Medical history information tracking method and device, computer equipment and readable storage medium

Abstract

The invention relates to a medical history information tracking method and device, computer equipment and a readable storage medium. The method comprises the following steps: acquiring a medical record image of a target object; performing layout analysis on the medical record image to obtain a plurality of layout partitions; performing text recognition on non-picture partitions in the plurality of layout partitions to obtain a medical record text; performing text analysis and information extraction on the medical record text to obtain structured medical record data; constructing a medical record graph database based on the structured medical record data; the medical history database is used for tracking medical history information. By adopting the method, the medical history information tracking efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of medical information systems, and in particular to a medical history information tracking method, apparatus, computer equipment and readable storage medium. Background Art

[0002] In traditional medical information systems, the collection of the target object's medical history mainly relies on two types. One is based on an interoperable electronic medical record system. After the doctor interviews the target object, the doctor extracts the corresponding fields in the electronic medical record and further edits and records them in combination with the current consultation situation; the other is a non-interoperable electronic medical record system. During the consultation process, the doctor consults the paper medical history materials provided by the target object or the cloud-based test reports provided by various hospitals and enters them manually. The above-mentioned medical history information collection method has the risk of losing medical history information when referring or visiting hospitals that have not yet interconnected electronic medical record systems, which is not conducive to the referral or transfer needs of the target object.

[0003] However, the completeness of paper medical history information depends on the target object's preservation of medical history data; the cloud report information provided by the hospital has different formats and uncertain retention periods, which leads to the doctor spending a lot of time to sort out the medical history data first and then manually enter it during the consultation. This series of operations may cause information loss, thus affecting the efficiency and quality of medical services. For the target object, if referral or transfer to another hospital is involved in the diagnosis and treatment process, it is necessary to completely preserve the medical history or clearly explain the previous course of disease and related test results, otherwise it will lead to the loss of some medical history information, and the corresponding tests and examinations will be re-performed or the treatment plan will be re-verified, which will increase the time and money cost of the target object to a certain extent and reduce the efficiency of medical treatment. In addition, the storage of its past medical history records mostly depends on the intercommunication within the hospital's electronic medical record system. The various types of medical history records that have not been effectively sorted out are not conducive to the target object's individual systematic medical history information storage. Therefore, the traditional electronic medical record system cannot fully display the target object's past medical history information when the system is not connected between hospitals, and there is a problem of low efficiency in tracking medical history information. Summary of the invention

[0004] Based on this, it is necessary to provide a medical history information tracking method, device, computer equipment, computer readable storage medium and computer program product to improve the efficiency of medical history information tracking in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for tracking medical history information, comprising:

[0006] Obtain medical record images of the target object;

[0007] Perform layout analysis on the medical record image to obtain multiple layout partitions;

[0008] Perform text recognition on non-image partitions in multiple layout partitions to obtain medical record text;

[0009] Perform text analysis and information extraction on medical record text to obtain structured medical record data;

[0010] A medical record graph database is constructed based on structured medical record data; the medical record graph database is used to track medical history information.

[0011] In one embodiment, text recognition is performed on non-image partitions in a plurality of layout partitions to obtain medical record text, including:

[0012] Performing text line detection on non-image partitions in multiple layout partitions to obtain text line detection results;

[0013] Performing a straightness score on each text line in the text line detection result to obtain straightness score results of each of the multiple text lines;

[0014] Determine a target text line whose straightness score result is lower than a preset score among the multiple text lines, and perform curvature transformation on the target text line;

[0015] Perform text recognition on the text lines after curvature transformation to obtain the medical record text.

[0016] In one embodiment, performing curvature transformation on a target text line includes:

[0017] Performing rectangular segmentation on the outer bounding box of the target text line to obtain multiple segmentation rectangles;

[0018] Determine the center point of the text line in each segmentation rectangle, and fit the text line center curve based on the text line center points of each of the multiple segmentation rectangles;

[0019] Based on multiple extreme value points in the center curve of the text line, the target text line is divided into multiple text segments;

[0020] Affine transformation is performed based on the inclination angles of the multiple text segments, and the obtained results after affine transformation are used as the text lines after curvature transformation.

[0021] In one embodiment, a straightness score is performed on each text line in the text line detection result to obtain straightness score results of multiple text lines, including:

[0022] For each text line in the text line detection result, determine the outer bounding box corresponding to the current text line;

[0023] Get the number of pixels in the outer bounding box corresponding to the current text line that are greater than a preset value, and the area of ​​the outer bounding box corresponding to the current text line;

[0024] The ratio of the number to the area is used as the straightness score result corresponding to the current text line.

[0025] In one embodiment, text line detection is performed on non-image partitions in multiple layout partitions to obtain text line detection results, including:

[0026] Extracting a first image feature of the medical record image, preprocessing the medical record image, and extracting a second image feature of the preprocessed medical record image;

[0027] fusing the first image feature and the second image feature to obtain a reference text line detection result;

[0028] Based on the reference text line detection result and the non-image partitions in the plurality of layout partitions, a text line detection result is determined.

[0029] In one embodiment, a layout analysis is performed on the medical record image to obtain multiple layout partitions, including:

[0030] The medical record image is sequentially preprocessed, normalized and resized to obtain a processed image; the preprocessing includes at least one of row color scale correction and tilt correction;

[0031] A preset neural network model is used to extract and classify features of the processed image to obtain multiple layout partitions.

[0032] In one embodiment, the medical history information tracking method further includes:

[0033] Get the query statement;

[0034] Vectorize the query statement to obtain a query vector;

[0035] Query the medical record database according to the query vector to obtain the query result;

[0036] Based on the query results, query statements and the preset language model, the medical history information tracking results generated by the preset language model are obtained.

[0037] In a second aspect, the present application also provides a medical history information tracking device, comprising:

[0038] An acquisition module, used to acquire a medical record image of a target object;

[0039] An analysis module, used for performing layout analysis on the medical record image to obtain multiple layout partitions;

[0040] A recognition module, used for performing text recognition on non-image partitions in multiple layout partitions to obtain medical record text;

[0041] The extraction module is used to perform text analysis and information extraction on the medical record text to obtain structured medical record data;

[0042] A construction module is used to build a medical record graph database based on structured medical record data; the medical record graph database is used to track medical history information.

[0043] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0044] Obtain medical record images of the target object;

[0045] Perform layout analysis on the medical record image to obtain multiple layout partitions;

[0046] Perform text recognition on non-image partitions in multiple layout partitions to obtain medical record text;

[0047] Perform text analysis and information extraction on medical record text to obtain structured medical record data;

[0048] A medical record graph database is constructed based on structured medical record data; the medical record graph database is used to track medical history information.

[0049] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0050] Obtain medical record images of the target object;

[0051] Perform layout analysis on the medical record image to obtain multiple layout partitions;

[0052] Perform text recognition on non-image partitions in multiple layout partitions to obtain medical record text;

[0053] Perform text analysis and information extraction on medical record text to obtain structured medical record data;

[0054] A medical record graph database is constructed based on structured medical record data; the medical record graph database is used to track medical history information.

[0055] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0056] Obtain medical record images of the target object;

[0057] Perform layout analysis on the medical record image to obtain multiple layout partitions;

[0058] Perform text recognition on non-image partitions in multiple layout partitions to obtain medical record text;

[0059] Perform text analysis and information extraction on medical record text to obtain structured medical record data;

[0060] A medical record graph database is constructed based on structured medical record data; the medical record graph database is used to track medical history information.

[0061] The above-mentioned medical history information tracking method, device, computer equipment, computer-readable storage medium and computer program product obtain medical record text by performing layout analysis and text recognition on the medical record image of the target object. Since the medical record image usually includes picture areas and non-picture areas, and the picture area may also include text, therefore, by performing layout analysis on the medical record image, it is helpful to identify the picture partitions and non-picture partitions in multiple layout partitions, and then performing text recognition on the non-picture partitions, eliminating the interference of text in the picture partitions, which is helpful to obtain medical record text that can effectively reflect the medical history of the target object; structured medical record data is obtained by performing text analysis and information extraction on the medical record text, and a medical record graph database is constructed based on the structured medical record data, which helps the target object to conveniently save medical history information, reduce the loss of medical history information, and ensure the integrity of medical history information; at the same time, in the form of a graph database, it is helpful to quickly track the historical medical history information of the target object, greatly improving the efficiency of tracking medical history information. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0063] FIG1 is a diagram of an application environment of a method for tracking medical history information in one embodiment;

[0064] FIG2 is a schematic diagram of a process flow of a method for tracking medical history information in one embodiment;

[0065] FIG3 is a schematic diagram of structured medical record data in one embodiment;

[0066] FIG4 is a schematic diagram of data processing of a pre-trained large language model in one embodiment;

[0067] FIG5 is a schematic diagram of text line detection results in one embodiment;

[0068] FIG6 is a schematic diagram of straightness scoring in one embodiment;

[0069] FIG7 is a schematic diagram of a text line detection step in one embodiment;

[0070] FIG8 is a schematic diagram of a process of color scale correction in one embodiment;

[0071] FIG9 is a schematic diagram of a process of tilt correction in one embodiment;

[0072] FIG10 is a schematic diagram of a flow chart of layout analysis in one embodiment;

[0073] FIG11 is a schematic diagram of medical history information query in one embodiment;

[0074] FIG12 is a schematic diagram of the composition of a medical history information tracking system in one embodiment;

[0075] FIG13 is a flow chart of a method for tracking medical history information in another embodiment;

[0076] FIG14 is a schematic diagram of a method for tracking medical history information in a specific embodiment 1;

[0077] FIG15 is a schematic diagram of a method for tracking medical history information in a specific embodiment 2;

[0078] FIG16 is a schematic diagram of a method for tracking medical history information in a specific embodiment 3;

[0079] FIG17 is a schematic diagram of a method for tracking medical history information in a specific embodiment 4;

[0080] FIG18 is a schematic diagram of a method for tracking medical history information in a specific embodiment 5;

[0081] FIG19 is a schematic diagram of a method for tracking medical history information in a specific embodiment 6;

[0082] FIG20 is a block diagram of a medical history information tracking device in one embodiment;

[0083] FIG. 21 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0085] The medical history information tracking method provided in the embodiment of the present application can be applied in the application environment shown in FIG. 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. This embodiment uses the method applied to the server 104 as an example. It can be understood that the method can also be applied to the terminal, and can also be applied to the system including the terminal and the server, and is implemented through the interaction between the terminal and the server. The server 104 obtains the medical record image of the target object; performs layout analysis on the medical record image to obtain multiple layout partitions; performs text recognition on the non-image partitions in the multiple layout partitions to obtain the medical record text; performs text analysis and information extraction on the medical record text to obtain structured medical record data; constructs a medical record image database based on the structured medical record data; and the medical record image database is used for medical history information tracking. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. The portable wearable device may be a smart watch, a smart bracelet, a head-mounted device, etc. The head-mounted device may be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0086] In an exemplary embodiment, as shown in FIG2 , a medical history information tracking method is provided, which is described by taking the method applied to the server 104 in FIG1 as an example, and includes the following steps 201 to 205. Among them:

[0087] Step 201, obtaining a medical record image of a target object.

[0088] The medical record image may be an image obtained by photographing or scanning the paper medical record of the target object. For example, a handheld electronic acquisition device may be used to respectively acquire images of multiple paper medical records provided by the target object to obtain multiple medical record images.

[0089] Medical record images can also be electronic medical records stored in the system, cloud-based test reports, etc.

[0090] Medical record images include a variety of examination data indicators, personal information of the target subject, medical hospital information, medical history information, diagnosis and treatment information, pathological examination information, doctor's orders and other information.

[0091] In some embodiments, the server backs up the acquired medical record images to a cloud data storage center, which helps avoid the loss of medical history information.

[0092] Step 202, performing layout analysis on the medical record image to obtain multiple layout partitions.

[0093] Among them, the medical record image usually includes multiple different types of content areas, such as headers, tables, texts, pictures, titles, etc. Layout analysis refers to partitioning each content area in the medical record image to obtain multiple layout partitions of multiple medical record images.

[0094] Exemplarily, the multiple layout partitions include a header partition, a table partition, a text partition, a picture partition, a title partition, and the like.

[0095] Step 203, performing text recognition on non-image partitions in the plurality of layout partitions to obtain medical record text.

[0096] Among them, non-image partitions refer to other partitions in multiple layout partitions except for image partitions, such as header partitions, table partitions, text partitions, title partitions, etc. Since image partitions may contain text or not, and the text in image partitions is often information irrelevant to medical history information such as the shooting time of the image and the shooting personnel information, if text recognition is performed on image partitions, the text in the image partitions may be recognized, thereby interfering with the medical history information of the target object. Therefore, the embodiment of the present application proposes to first perform layout analysis on the medical record image, and then perform text recognition on the non-image partitions in multiple layout partitions, and the obtained text recognition result is the medical record text. In this embodiment, a two-stage OCR (Optical Character Recognition) technology of text line detection and then text recognition is used for medical record text recognition, which is conducive to improving the accuracy of text recognition.

[0097] Step 204: perform text analysis and information extraction on the medical record text to obtain structured medical record data.

[0098] Among them, the medical record texts identified by text recognition technology are often plain texts without formatting. The medical record texts contain a large amount of text descriptions, including the target object's medical history, symptoms, diagnosis, treatment plan and other information. However, these medical record texts are often difficult to be directly understood and used by computer systems, which limits the application and analysis of medical record texts. Structured medical record data refers to structured data with a certain structure and format. Converting medical record texts into structured medical record data allows these data to be better processed and analyzed by computer systems, and can be more conveniently stored, retrieved, analyzed and shared, providing support for medical decision-making, clinical research, and medical quality assessment.

[0099] Exemplarily, the server uses a pre-trained large language model to perform text analysis on the medical record text, extract key information, including but not limited to the target subject's personal information, hospital information, medical history information, diagnosis and treatment information, pathological examination information, doctor's orders, etc., and uses it as structured medical record data. FIG3 is a schematic diagram of structured medical record data in one embodiment.

[0100] In some embodiments, the pre-trained large language model can be obtained by fine-tuning the basic large language model based on medical information, wherein the basic large language model can be any model that has undergone basic pre-training. FIG4 is a schematic diagram of data processing of a pre-trained large language model in one embodiment.

[0101] Step 205, construct a medical record graph database based on the structured medical record data; the medical record graph database is used for tracking medical history information.

[0102] Among them, the medical record graph database is a data management system based on node and edge storage units, for example, it can be Neo4j (a graph database), ArangoDB (a native multi-model database), TigerGraph (a high-performance graph database and analysis platform). The server determines the data attributes of each structured medical record data, takes each data attribute as a node, and takes the association relationship between the data attributes as an edge. The medical record graph database can be constructed by each node and edge. For example, data attributes can include patient information, hospital information, medical records, examination items, examination conclusions, medical advice, etc., and the association relationship can include visit, visit, examination, examination results, medical advice, etc., wherein patient information can include name (name), age (age), gender (gender), medical record number (patient_id), etc., hospital information can include name (name), address (address), number (id), etc., examination items can include name (name), type (type), examination value (value), reference value (reference), etc., medical records can include visit time (visit_date), visit type (visit_type), record number (record_id), etc.

[0103] In some embodiments, the server can convert the structured medical record data of the target object into corresponding nodes and edges, and add them to the existing graph database. The same nodes and edges can be merged, and different nodes and edges can be added to the existing graph database. As medical record data is continuously added, a more complete medical record graph database can be established. In this way, the medical record graph database can track personal medical history information for the target object, and can also track diagnosis and treatment information of patients for doctors.

[0104] Since the medical record graph database stores medical history information according to the association relationship between nodes, tracking medical history information based on the medical record graph database is conducive to improving the efficiency of medical history information tracking.

[0105] In the above-mentioned medical history information tracking method, the medical record text is obtained by performing layout analysis and text recognition on the medical record image of the target object. Since the medical record image usually includes picture areas and non-picture areas, and the picture area may also include text, therefore, performing layout analysis on the medical record image is conducive to identifying the picture partitions and non-picture partitions in multiple layout partitions, and then performing text recognition on the non-picture partitions, eliminating the interference of text in the picture partitions, which is conducive to obtaining a medical record text that can effectively reflect the medical history of the target object; structured medical record data is obtained by performing text analysis and information extraction on the medical record text, and a medical record graph database is constructed based on the structured medical record data, which helps the target object to conveniently save medical history information, reduce the loss of medical history information, and ensure the integrity of medical history information; at the same time, in the form of a graph database, it is conducive to quickly tracking the historical medical history information of the target object, greatly improving the efficiency of tracking medical history information.

[0106] In an exemplary embodiment, text recognition is performed on non-image partitions among multiple layout partitions to obtain medical record text, including: performing text line detection on non-image partitions among multiple layout partitions to obtain text line detection results; performing straightness scoring on each text line in the text line detection results to obtain straightness scoring results for each of the multiple text lines; determining a target text line among the multiple text lines whose straightness score result is lower than a preset score, and performing curvature transformation on the target text line; performing text recognition on the text line after the curvature transformation to obtain medical record text.

[0107] Among them, for the non-image partition in the medical record image, there are often multiple text lines, for example, it can be a text line in a text segment, or it can be a text line in a table. Text line detection is used to detect text lines in the non-image area of ​​the medical record image. The text line detection result can be a binary image of the same size as the medical record image, the position where the text line is located is the first pixel value, for example, the first pixel value is 255, and the position where the non-text line is located is the second pixel value, for example, the second pixel value is 0. As shown in Figure 5, it is a schematic diagram of the text line detection result in one embodiment.

[0108] In some embodiments, a machine learning model may be used to perform text line detection on non-image partitions in multiple layout partitions.

[0109] In actual scenarios, text lines in medical record images often appear as elongated rectangles, while paper medical records may have folds, or there may be shooting angle deviations during the shooting process of the electronic acquisition device. Therefore, each text line in the text line detection result may have a certain degree of deformation (refer to Figure 5). The embodiment of the present application proposes to perform a straightness score on each text line in the text line detection result. The straightness score is used to characterize the degree of deformation of the corresponding text line. The target text line with a score lower than the preset score is then subjected to a curvature transformation to reduce the degree of deformation of the target text line. Finally, text recognition is performed on the text line after the curvature transformation, which is beneficial to improving the accuracy of the medical record text.

[0110] Among them, since text recognition is essentially a sequence-to-sequence mapping, the medical record image containing text can be understood as a pixel sequence composed of a series of pixels, and the corresponding text is a text sequence, so text recognition is reduced to a sequence-to-sequence problem. The process of text recognition is as follows: input the text line detection result, for each text line, downsample the current text line, extract features, perform the labeling task on the extracted feature sequence, that is, assign a symbol to each feature, score the assigned symbols of the feature sequence, and finally select the assigned sequence with the highest score as the text output to obtain the medical record text.

[0111] In this embodiment, text line detection is performed on non-image partitions in multiple layout partitions, and each text line is scored for straightness, so as to identify the degree of deformation of each text line. Curvature transformation is performed on target text lines whose straightness scores are lower than preset scores, which can reduce the degree of deformation of the target text lines. In this way, text recognition is performed on the text lines after the curvature transformation, which is beneficial to improving the accuracy of the medical record text.

[0112] In an exemplary embodiment, a target text line is subjected to a curvature transformation, including: performing rectangular segmentation on an outer bounding box of the target text line to obtain a plurality of segmentation rectangles; determining a text line center point in each segmentation rectangle, and fitting a text line center curve based on the respective text line center points of the plurality of segmentation rectangles; dividing the target text line into a plurality of text segments based on a plurality of extreme value points in the text line center curve; and performing an affine transformation based on the respective inclination angles of the plurality of text segments, and obtaining a result after the affine transformation as a text line subjected to a curvature transformation.

[0113] The outer bounding box of the target text line refers to a rectangular box surrounding the target text line, for example, it may be a minimum rectangular box surrounding the target text line.

[0114] Rectangular segmentation refers to sliding and segmenting the outer bounding box according to a preset step size. Each slide segments a small rectangle from the outer bounding box of the target text line. After multiple rectangle segmentations, multiple segmented rectangles can be obtained.

[0115] The text line center point refers to the center point of the text line in each segmentation rectangle. The server performs curve fitting on multiple text line center points to obtain a text line center curve.

[0116] The extreme point refers to the point in the center curve of the text line with a slope of 0. The server divides the target text line into multiple text segments according to the extreme point. The part of the center curve of the text line adjacent to the extreme point corresponds to the text segment with deformation.

[0117] The tilt angle of the text segment is the tilt angle of the corresponding text segment part in the center curve of the text line. The server performs affine transformation on each corresponding text segment based on the tilt angle to obtain the result after affine transformation, which is a standard rectangle. Therefore, the text line after curvature transformation is a straight text line.

[0118] Exemplarily, the steps of curvature transformation are as follows:

[0119] Input the text line detection result to obtain the minimum outer bounding box [x_min, x_max, y_min, y_max] of each text line; calculate the center point of the text line, where a loop traverses from x_min to x_max with a step length of n, using y_min, y_max as the ordinates, and x and x+n in the previous step as the abscissas, to construct a differential rectangle, i.e., a segmentation rectangle, and calculate the centroid of all points with the first pixel value in the segmentation rectangle, i.e., the center point of the text line. Based on the polynomial fitting curve algorithm, the center curve of the text line is obtained by solving, and then the point where the first derivative of the center curve of the text line is equal to 0 is taken as the extreme point. If the number of extreme points is greater than 0, the target text line is divided into multiple text segments according to the extreme points. Calculate the slope of the center curve of the text line, and solve the inclination angle corresponding to each text segment, and then perform affine transformation on each text segment to obtain a standard rectangle, i.e., the text line after curvature transformation.

[0120] In this embodiment, the outer bounding box of the target text line is divided into rectangles, the text line center curve is fitted based on the text line center points of each of the multiple divided rectangles, the target text line is divided using multiple extreme points in the text line center curve, and an affine transformation is performed based on the inclination angles of each of the multiple text segments to obtain the text line after curvature transformation. This curvature transformation method effectively improves the text recognition accuracy when the paper medical record is bent.

[0121] In an exemplary embodiment, a straightness score is performed on each text line in the text line detection result to obtain straightness score results for each of the multiple text lines, including: for each text line in the text line detection result, determining an outer bounding box corresponding to the current text line; obtaining the number of pixel values ​​in the outer bounding box corresponding to the current text line that are greater than a preset value, and the area of ​​the outer bounding box corresponding to the current text line; and using the ratio of the number to the area as the straightness score result corresponding to the current text line.

[0122] The outer bounding box refers to a rectangular box surrounding the current text line, for example, it can be the smallest rectangular box surrounding the current text line. Figure 6 is a schematic diagram of a straightness score in one embodiment. The rectangular box in the figure is the outer bounding box corresponding to the current text line.

[0123] The text line detection result includes pixels belonging to text lines and pixels belonging to non-text lines. Pixels whose pixel values ​​are greater than a preset value are pixels belonging to text lines. For example, the preset value is 0.

[0124] The ratio of the number to the area is the straightness score result corresponding to the current text line. In some embodiments, the straightness score result is less than 0.6, indicating that the current text line is severely deformed.

[0125] In this embodiment, by obtaining the number of pixel values ​​in the outer bounding box corresponding to the current text line that are greater than a preset value and the area of ​​the outer bounding box, the ratio of the two is used as the straightness scoring result, which is helpful in determining the deformation degree of the current text line.

[0126] In an exemplary embodiment, text line detection is performed on non-picture partitions among multiple layout partitions to obtain text line detection results, including: extracting a first image feature of a medical record image, preprocessing the medical record image, and extracting a second image feature of the preprocessed medical record image; fusing the first image feature and the second image feature to obtain a reference text line detection result; and determining the text line detection result based on the reference text line detection result and the non-picture partitions among multiple layout partitions.

[0127] Among them, the first image feature refers to the feature of the medical record image, and the second image feature refers to the feature of the preprocessed medical record image. The method of preprocessing the medical record image includes at least one of color scale correction and tilt correction. Since there may be problems such as creases and text deformation in the medical record image, preprocessing the medical record image is beneficial to correct the problems such as creases and text deformation in the medical record image. However, preprocessing may also cause certain result deviations. If text line detection is performed directly based on the first image feature of the medical record image, or if text line detection is performed directly based on the second image feature of the preprocessed medical record image, it may cause missed detection and false detection. In order to ensure the accuracy of text line detection, the first image feature and the second image feature can be fused to improve the accuracy of the text line detection result.

[0128] The reference text line detection result includes text lines of the picture partition and the non-picture partition. Therefore, the server uses the part of the reference text line detection result that belongs to the non-picture partition as the text line detection result.

[0129] FIG7 is a schematic diagram of a text line detection step in an embodiment, wherein the medical record image and the preprocessed medical record image are respectively extracted features using a deep neural network, and then feature fusion is performed to finally obtain a text line detection result, which is much better than the traditional target detection algorithm.

[0130] In this embodiment, the detection accuracy of the text line is improved by extracting and fusing the features of the medical record image and the preprocessed medical record image and then performing semantic segmentation.

[0131] In an exemplary embodiment, a layout analysis is performed on a medical record image to obtain multiple layout partitions, including: preprocessing, normalizing and resizing the medical record image in sequence to obtain a processed image; the preprocessing includes at least one of row color scale correction and tilt correction; and a preset neural network model is used to extract features and classify the processed image to obtain multiple layout partitions.

[0132] Color scale correction refers to a processing method for restoring the color tone of the shooting scene to ensure that the brightness and contrast of the medical record image conform to the natural perception of the human eye. FIG8 is a flowchart of color scale correction in one embodiment. The color scale correction steps are as follows:

[0133] 1. Start: Start the execution of the algorithm.

[0134] 2. Initialization parameters:

[0135] Set rgb_min_output = 0 and rgb_max_output = 255, initialize channel_list to an empty list, and copy the medical record image raw_img to dst.

[0136] 3. Calculate the minimum and maximum values ​​of each channel: For each color channel (red, green, blue), calculate the minimum value channel_min_input and the maximum value channel_max_input of each channel.

[0137] 4. Calculate histogram: For each color channel, calculate its histogram hist_c and bin boundaries bin_edges.

[0138] 5. Adjust the minimum and maximum values ​​of the input: For each channel: calculate the cumulative histogram and determine the channel_max_input[c] value that satisfies sum>0.99, calculate the cumulative histogram in reverse order and determine the channel_min_input[c] value that satisfies sum>0.99.

[0139] 6. Calculate the output channel: For each channel: calculate the output value of each pixel value according to the formula and store it in channel_list.

[0140] 7. Apply color scale adjustment: For each channel of each pixel: set the pixel values ​​exceeding channel_max_input[c] to 255, and replace the remaining pixel values ​​with the calculated color scale adjusted value.

[0141] 8. Handle exceptions: If any exception is encountered during execution, the unprocessed raw_img is returned.

[0142] 9. Return the processed image: Return the image dst after color level adjustment.

[0143] 10. End.

[0144] Tilt correction refers to a method for restoring the tilted portion of a medical record image. FIG9 is a schematic diagram of a tilt correction process in an embodiment. The tilt correction is based on Hough transform, and the steps are as follows:

[0145] 1. The Hough parameter space is represented by two parameters: ρ: distance, vertical distance, θ: angle, the inclination angle of the straight line relative to the horizontal axis.

[0146] 2. Traverse each edge point in the edge detection result: For each edge point (x, y), calculate its corresponding (ρ, θ) value: ρ=x⋅cos(θ)+y⋅sin(θ); the counter in the Hough space corresponding to (ρ, θ) is increased by one, and the local maximum values ​​are found in the accumulator array. These maximum values ​​correspond to the straight lines in the image. These local maxima represent the detected straight line parameters (ρ, θ), and the detected straight line parameters (ρ, θ) are converted back to the straight line in the image space.

[0147] Normalization refers to normalizing the pixel values ​​of an image to [0,1]. Resizing refers to adjusting the size of an image to a preset size to ensure that the image fed into the neural network is of uniform size.

[0148] FIG10 is a flow chart of layout analysis in one embodiment. The server preprocesses the medical record image, then normalizes the preprocessed medical record image to obtain a normalized image, then resizes the normalized image to obtain a processed image, and uses a preset neural network model to extract features and classify the processed image to obtain multiple layout partitions. The preset neural network can be a neural network model with an encoder and decoder structure.

[0149] In this embodiment, the medical record image is preprocessed, normalized and resized in sequence to obtain a processed image, and then a preset neural network model is used to extract features and classify the processed image to obtain multiple layout partitions. Performing multiple processing on the medical record image is helpful to improve the accuracy of the layout partitions; using the preset neural network model to divide the layout partitions improves the efficiency and accuracy of the layout partitions.

[0150] In an exemplary embodiment, the medical history information tracking method also includes: obtaining a query statement; vectorizing the query statement to obtain a query vector; querying the medical record database based on the query vector to obtain a query result; and obtaining a medical history information tracking result generated by a preset language model based on the query result, the query statement and a preset language model.

[0151] The query statement refers to a statement input by the user for querying medical history information, which can be a natural language statement or a text statement. If it is a natural language statement, natural language processing is required to obtain the corresponding query text.

[0152] FIG11 is a schematic diagram of medical history information query in one embodiment. The server vectorizes the query statement and converts the query statement into a corresponding vector to obtain a query vector. In some embodiments, by comparing the similarity between the query vector and the vector corresponding to each data in the medical record database, the data in the medical record database whose similarity meets the preset conditions is used as the query result.

[0153] The server inputs the query result and the query statement into a preset language model to obtain the medical history information tracking result generated by the preset language model.

[0154] In this embodiment, the query statement is vectorized, and then the vector similarity comparison method is used to obtain preliminary retrieval results from the medical record database. The query results and the query statement are input into the preset language model to generate the medical history information tracking results. The medical history query is combined with an external database, namely the medical record database, so that the preset language model can generate more accurate results that are more suitable for the medical history query scenario, thereby improving the accuracy of the medical history information tracking results.

[0155] To explain the medical history information tracking method and effect in detail, the following is a most detailed example:

[0156] The tracking of medical history information in the diagnosis and treatment scenario is used as an example for explanation. The medical history information tracking method is applied to the medical history information tracking system. FIG12 is a schematic diagram of the composition of the medical history information tracking system in one embodiment. Before seeking medical treatment, the target subject uses an electronic collection device to collect previous paper medical history materials and uploads them to the system, or directly uploads electronic medical records. The system receives historical medical records in the form of images, and forms a medical record database through layout analysis, OCR recognition, and information extraction. It displays the target subject's historical diagnosis and treatment records and condition tracking to the doctor from different dimensions, providing effective information for the doctor's diagnosis and treatment.

[0157] The medical history information tracking system includes two major modules: an OCR module and a large language model module. Figure 13 is a flow chart of a medical history information tracking method in another embodiment. Among them, the OCR module is responsible for text recognition of the medical records in the image format uploaded by the target object, and outputs editable medical record text. The large language model module is used to perform text analysis and information extraction on the obtained medical record text to obtain key content such as patient personal information, medical hospital information, medical history information, diagnosis and treatment information, pathological examination information, and doctor's orders; format the content into structured medical record data in json format, and then convert it into a graph database for storage, and finally perform user information retrieval and target object medical history information display through the medical record graph database.

[0158] In a specific embodiment 1, as shown in FIG14, a schematic diagram of a medical history information tracking method in a specific embodiment 1. In the scenario of a patient's initial medical consultation, the patient takes a photo of the paper medical record through a handheld electronic collection device and authorizes it to be uploaded, which is converted by the system into a structured format that conforms to the doctor's personal diagnosis and treatment habits for the doctor to read. The construction of structured medical record data includes automatic extraction of key information, which includes but is not limited to positive indicator extraction, test results required for differential diagnosis, etc.

[0159] In a specific embodiment 2, as shown in Figure 15, a schematic diagram of a medical history information tracking method in a specific embodiment 2 is shown. In the scenario of a patient's follow-up visit, the patient takes a photo of the medical records required for the follow-up visit and authorizes the upload of the photo, which is converted into a structured format that conforms to the doctor's personal diagnosis and treatment habits by the system. On this basis, the doctor selects the content that needs to be paid attention to for comparison and reading to form a follow-up medical record.

[0160] In a specific embodiment three, as shown in FIG16, a schematic diagram of a medical history information tracking method in a specific embodiment three is shown. In the patient referral / consultation medical treatment scenario, the patient takes a photo of his personal medical record through a handheld electronic collection device and authorizes it to be uploaded. The system converts it into a structured format that conforms to the doctor's personal diagnosis and treatment habits for the attending doctor (i.e., Doctor A in FIG16) to read and form a medical record. After selecting the content and filling in the treatment opinion, a sharing link is formed and forwarded to the referring / consulting doctor (i.e., Doctor B in FIG16). After Doctor B completes the consultation, he decides whether to share Doctor A based on the actual situation.

[0161] In a specific embodiment 4, as shown in Figure 17, a schematic diagram of a medical history information tracking method in a specific embodiment 4. In the scenario of hospital remote consultation / Internet hospital follow-up pre-consultation medical treatment, the patient takes a photo of the personal medical record through a handheld electronic collection device and authorizes it to upload. The system automatically extracts key information (including but not limited to positive index extraction, test results required for differential diagnosis, etc.), obtains medical record data in a structured format, and forms a standard medical history format together with the patient's pre-consultation questionnaire answer results for the doctor to read. After the doctor receives the consultation, he fills in the treatment opinion to form a medical record, which can be sent back to the patient, the local hospital or stored in the Internet hospital as appropriate.

[0162] In a specific embodiment 5, as shown in FIG18, a schematic diagram of a medical history information tracking method in a specific embodiment 5 is shown. In the doctor research scenario, the doctor can set the required screening requirements and preview the results according to his research direction, and download the medical data of the desensitized patient personal information to form research data. The desensitized data can also be merged with the data of the doctors in the same team according to the actual situation to achieve the purpose of sharing research data.

[0163] In a specific embodiment six, as shown in FIG19, a schematic diagram of a medical history information tracking method in a specific embodiment six is ​​shown. On the premise of meeting the data volume requirements, a pre-diagnosis model that conforms to the doctor's personal diagnosis and treatment thinking is formed by a large model to assist the doctor in collecting the patient's outpatient pre-diagnosis medical history, and combined with the original medical record structuring function, a doctor's personal model is formed to assist the doctor in quickly forming outpatient medical records and treatment opinions. In addition, this embodiment can be used in different application scenarios. On the premise of complying with laws and regulations, it is combined with the scenario processes of the aforementioned embodiments one to four (follow-up visits, referrals, remote diagnosis and treatment, etc.) to form a telemedicine model.

[0164] The above-mentioned medical history information tracking method obtains the medical record text by performing layout analysis and text recognition on the medical record image of the target object. Since the medical record image usually includes picture areas and non-picture areas, and the picture area may also include text, therefore, performing layout analysis on the medical record image is conducive to identifying the picture partitions and non-picture partitions in multiple layout partitions, and then performing text recognition on the non-picture partitions, eliminating the interference of text in the picture partitions, which is conducive to obtaining a medical record text that can effectively reflect the medical history of the target object; structured medical record data is obtained by performing text analysis and information extraction on the medical record text, and a medical record graph database is constructed based on the structured medical record data, which helps the target object to conveniently save medical history information, reduce the loss of medical history information, and ensure the integrity of medical history information; at the same time, in the form of a graph database, it is conducive to quickly tracking the historical medical history information of the target object, greatly improving the efficiency of tracking medical history information. In addition, the medical record tracking function provides individuals with a medical record folder mode, which consists of main functions such as medical history collection, medical history recognition and editing, and medical history structured reproduction. It is intended to help the target object conveniently save medical history data, reduce the loss of key medical history data when changing medical scenarios, and ensure the integrity of previous medical records; the recognition of medical history images and the structuring of medical history information not only condense the previous medical history to form a structured summary, which is convenient for the attending doctor to quickly grasp the condition of the target object and improve the efficiency of diagnosis and treatment, but also provides doctors with a personalized diagnosis and treatment database to assist doctors in disease research; a graph database is constructed based on the medical history information of the target object and presented to the user. Since each target object has more than one The system indirectly solves the problem of intercommunication of medical history data among hospitals, builds a high-speed information channel for different diseases and different diagnosis and treatment methods, and fills the gap in the field of diagnosis and treatment information. In the text line detection process, a method for calculating irregular text lines based on projection transformation is proposed, which is suitable for deformable text line recognition in a large number of hand-shot scenarios. The online learning function is introduced to form a personalized intelligent assistant for individual doctors by learning the personal preferences of different doctors, which not only solves the problem of medical record organization for patients, but also improves the diagnosis and treatment efficiency of doctors. The collected medical history information is exported to both doctors and patients in a standardized format, which is convenient for subsequent medical services and research use by both parties.

[0165] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0166] Based on the same inventive concept, the embodiment of the present application also provides a medical history information tracking device for implementing the medical history information tracking method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more medical history information tracking device embodiments provided below can refer to the limitations of the medical history information tracking method above, and will not be repeated here.

[0167] In an exemplary embodiment, as shown in FIG20 , a medical history information tracking device 100 is provided, comprising: an acquisition module 110, an analysis module 120, an identification module 130, an extraction module 140 and a construction module 150, wherein:

[0168] An acquisition module 110 is used to acquire a medical record image of a target object;

[0169] An analysis module 120 is used to perform layout analysis on the medical record image to obtain a plurality of layout partitions;

[0170] A recognition module 130 is used to perform text recognition on non-image partitions in the plurality of layout partitions to obtain medical record text;

[0171] The extraction module 140 is used to perform text analysis and information extraction on the medical record text to obtain structured medical record data;

[0172] The construction module 150 is used to construct a medical record database based on the structured medical record data; the medical record database is used for tracking medical history information.

[0173] The above-mentioned medical history information tracking device obtains the medical record text by performing layout analysis and text recognition on the medical record image of the target object. Since the medical record image usually includes picture areas and non-picture areas, and the picture area may also include text, therefore, by performing layout analysis on the medical record image, it is helpful to identify the picture partitions and non-picture partitions in multiple layout partitions, and then performing text recognition on the non-picture partitions, eliminating the interference of text in the picture partitions, which is helpful to obtain the medical record text that can effectively reflect the medical history of the target object; structured medical record data is obtained by performing text analysis and information extraction on the medical record text, and a medical record graph database is constructed based on the structured medical record data, which helps the target object to conveniently save medical history information, reduce the loss of medical history information, and ensure the integrity of medical history information; at the same time, in the form of a graph database, it is helpful to quickly track the historical medical history information of the target object, greatly improving the efficiency of tracking medical history information.

[0174] In one embodiment, text recognition is performed on non-image partitions among multiple layout partitions to obtain medical record text, and the recognition module 130 is also used to: perform text line detection on non-image partitions among multiple layout partitions to obtain text line detection results; perform straightness scoring on each text line in the text line detection results to obtain straightness scoring results for each of the multiple text lines; determine a target text line among the multiple text lines whose straightness score result is lower than a preset score, and perform curvature transformation on the target text line; perform text recognition on the text line after the curvature transformation to obtain medical record text.

[0175] In one embodiment, the target text line is subjected to a curvature transformation, and the recognition module 130 is further used to: perform a rectangular segmentation on the outer bounding box of the target text line to obtain a plurality of segmentation rectangles; determine the center point of the text line in each segmentation rectangle, and fit a text line center curve based on the text line center points of each of the plurality of segmentation rectangles; divide the target text line into a plurality of text segments based on a plurality of extreme points in the text line center curve; perform an affine transformation based on the inclination angles of each of the plurality of text segments, and use the result of the affine transformation as the text line after the curvature transformation.

[0176] In one embodiment, a straightness score is performed on each text line in the text line detection result to obtain straightness score results for each of the multiple text lines. The recognition module 130 is also used to: determine, for each text line in the text line detection result, an outer bounding box corresponding to the current text line; obtain the number of pixel values ​​in the outer bounding box corresponding to the current text line that are greater than a preset value, and the area of ​​the outer bounding box corresponding to the current text line; and use the ratio of the number to the area as the straightness score result corresponding to the current text line.

[0177] In one embodiment, text line detection is performed on non-picture partitions in multiple layout partitions to obtain text line detection results, and the recognition module 130 is also used to: extract first image features of the medical record image, preprocess the medical record image, and extract second image features of the preprocessed medical record image; fuse the first image features and the second image features to obtain a reference text line detection result; determine the text line detection result based on the reference text line detection result and the non-picture partitions in the multiple layout partitions.

[0178] In one embodiment, a layout analysis is performed on the medical record image to obtain a plurality of layout partitions, and the analysis module 120 is further used to: preprocess, normalize and resize the medical record image in sequence to obtain a processed image; the preprocessing includes at least one of row color scale correction and tilt correction; and a preset neural network model is used to extract features and classify the processed image to obtain a plurality of layout partitions.

[0179] In one embodiment, the medical history information tracking device 100 also includes a query module, which is also used to: obtain a query statement; vectorize the query statement to obtain a query vector; query the medical record database based on the query vector to obtain a query result; based on the query result, the query statement and the preset language model, obtain the medical history information tracking result generated by the preset language model.

[0180] Each module in the above-mentioned medical history information tracking device can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.

[0181] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG21. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store medical record images, medical record texts, structured medical record data, and a medical record diagram database. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a medical history information tracking method is implemented.

[0182] Those skilled in the art will understand that the structure shown in FIG. 21 is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0183] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0184] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0185] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0186] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0187] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0188] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0189] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for tracking medical history information, characterized in that: The method comprises: Obtain medical record images of the target object; Performing layout analysis on the medical record image to obtain a plurality of layout partitions; Performing text recognition on the non-image partitions in the multiple layout partitions to obtain medical record text; Performing text analysis and information extraction on the medical record text to obtain structured medical record data; A medical record database is constructed based on the structured medical record data; the medical record database is used for tracking medical history information.

2. The method according to claim 1, characterized in that The step of performing text recognition on the non-image partitions in the plurality of layout partitions to obtain the medical record text includes: Performing text line detection on the non-image partitions in the multiple layout partitions to obtain text line detection results; Performing a straightness score on each text line in the text line detection result to obtain straightness score results of each of the plurality of text lines; Determine a target text line whose straightness score result is lower than a preset score among the multiple text lines, and perform curvature transformation on the target text line; Perform text recognition on the text lines after curvature transformation to obtain the medical record text.

3. The method according to claim 2, characterized in that The step of performing curvature transformation on the target text line includes: Performing rectangular segmentation on the outer bounding box of the target text line to obtain a plurality of segmentation rectangles; Determine the center point of the text line in each segmentation rectangle, and fit the text line center curve based on the text line center points of each of the multiple segmentation rectangles; Based on a plurality of extreme value points in the text line center curve, dividing the target text line into a plurality of text segments; Affine transformation is performed based on the inclination angles of the multiple text segments, and the obtained results after affine transformation are used as the text lines after curvature transformation.

4. The method according to claim 2, characterized in that: The step of performing a straightness score on each text line in the text line detection result to obtain straightness score results of each of the plurality of text lines comprises: For each text line in the text line detection result, determining an outer bounding box corresponding to the current text line; Obtaining the number of pixel values ​​in the outer bounding box corresponding to the current text line that are greater than a preset value, and the area of ​​the outer bounding box corresponding to the current text line; The ratio of the number to the area is used as a straightness score result corresponding to the current text line.

5. The method according to claim 2, characterized in that: The performing text line detection on the non-image partitions in the plurality of layout partitions to obtain text line detection results includes: Extracting a first image feature of the medical record image, preprocessing the medical record image, and extracting a second image feature of the preprocessed medical record image; fusing the first image feature and the second image feature to obtain a reference text line detection result; A text line detection result is determined based on the reference text line detection result and non-image partitions in the multiple layout partitions.

6. The method according to claim 1, characterized in that The performing layout analysis on the medical record image to obtain a plurality of layout partitions includes: Preprocessing, normalizing and resizing the medical record image in sequence to obtain a processed image; the preprocessing includes at least one of row color scale correction and tilt correction; A preset neural network model is used to extract and classify features of the processed image to obtain multiple layout partitions.

7. The method according to claim 1, characterized in that The method further comprises: Get the query statement; Vectorizing the query statement to obtain a query vector; Query the medical record database according to the query vector to obtain a query result; Based on the query result, the query statement and the preset language model, a medical history information tracking result generated by the preset language model is obtained.

8. A medical history information tracking device, characterized in that: The device comprises: An acquisition module, used to acquire a medical record image of a target object; An analysis module, used for performing layout analysis on the medical record image to obtain a plurality of layout partitions; A recognition module, used for performing text recognition on the non-image partitions in the plurality of layout partitions to obtain medical record text; An extraction module, used to perform text analysis and information extraction on the medical record text to obtain structured medical record data; A construction module is used to construct a medical record database based on the structured medical record data; the medical record database is used for tracking medical history information.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images