Standardized storage method, device and medium for physical examination report

Abstract

Embodiments of the present disclosure disclose a physical examination report standardization storage method, device, electronic equipment and medium. A specific embodiment of the method comprises: performing image enhancement processing on an obtained user physical examination report image set, and then performing physical examination item classification processing to obtain a first physical examination item report image set and a second physical examination item report image set; generating an image detection text information set; performing image text recognition on a user physical examination text region image set to obtain a physical examination report text recognition information set; performing text calibration on the physical examination report text recognition information set to obtain a physical examination report text calibration information set; generating a report standardization prompt phrase group set; performing content extraction on the physical examination report text calibration information set to obtain a report extraction information set; and matching and storing the report extraction information set to a user physical examination report storage system. The embodiment can improve the extraction accuracy and effect of physical examination report information, shorten the extraction time and reduce the waste of storage resources.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer technology, and particularly to a physical examination report standardization storage method and device, electronic equipment and medium. BACKGROUND

[0002] A physical examination report contains a large amount of information about different examination items related to the user's physical health. Different examination items have various data representations and format differences, so it is an important problem to uniformly and standardize the storage of physical examination reports. For the standardization storage of physical examination reports, the commonly used way is: using a traditional OCR (Optical Character Recognition) algorithm to perform text detection and recognition on the physical examination report image to obtain physical examination report text information. Then, using a predefined data extraction rule and template to extract the content of the physical examination report text information to obtain physical examination extraction information. Finally, storing the physical examination extraction information.

[0003] However, in practice, it is found that when the above method is used to standardize the storage of physical examination reports, the following technical problems often exist: 1. Since the traditional OCR algorithm is based on artificial features for text recognition, the accuracy of text recognition is low and the recognition generalization ability is poor in the complex and variable text and image interspersed text of the physical examination report, and the predefined data extraction rule and template are artificially defined, which has certain subjectivity and professionalism, cannot adapt to dynamic changes, resulting in low accuracy and efficiency of physical examination report content extraction, a large amount of redundant and erroneous data in the extracted content, and waste of storage resources and prolonged extraction time.

[0004] The above information disclosed in this BACKGROUND section is only for the purpose of enhancing the understanding of the background of the present disclosure and, therefore, can include information that does not form the prior art known to those of ordinary skill in the art in the country. SUMMARY

[0005] The summary of the present disclosure is intended to introduce the concepts in a simplified form, which will be described in detail in the specific embodiments section. The summary of the present disclosure is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0006] Some embodiments of the present disclosure propose a physical examination report standardization storage method, device, electronic equipment and medium to solve one or more of the technical problems mentioned in the above BACKGROUND section.

[0007] In a first aspect, some embodiments of the present disclosure provide a physical examination report standardization storage method, comprising: obtaining a user physical examination report image set and a physical examination report standardization rule information set, wherein the user physical examination report image set comprises a user physical examination image area image and a user physical examination text area image; performing image enhancement processing on the user physical examination report image set to obtain an enhanced user physical examination report image set; performing physical examination item classification processing on the enhanced user physical examination report image set to obtain a first physical examination item report image set and a second physical examination item report image set; generating an image detection text information set according to a user physical examination image area image set included in the first physical examination item report image set; performing image text recognition on a user physical examination text area image set included in the first physical examination item report image set to obtain a physical examination report text recognition information set; performing text calibration on the physical examination report text recognition information set according to the image detection text information set to obtain a physical examination report text calibration information set; generating a report standardization prompt phrase group set according to the physical examination report standardization rule information set and the user physical examination report image set; performing content extraction on the physical examination report text calibration information set according to a physical examination report content extraction large language model and the report standardization prompt phrase group set to obtain a report extraction information set; and matching and storing the report extraction information set to a user physical examination report storage system, and performing storage load optimization on the user physical examination report storage system.

[0008] In a second aspect, some embodiments of the present disclosure provide a physical examination report standardization storage device, comprising: an acquisition unit configured to acquire a user physical examination report image set and a physical examination report standardization rule information set, wherein the user physical examination report image comprises a user physical examination image area image and a user physical examination text area image; an image enhancement unit configured to perform image enhancement processing on the user physical examination report image set to obtain an enhanced user physical examination report image set; a physical examination item classification unit configured to perform physical examination item classification processing on the enhanced user physical examination report image set to obtain a first physical examination item report image set and a second physical examination item report image set; a first generation unit configured to generate an image detection text information set according to a user physical examination image area image set included in the first physical examination item report image set; an image text recognition unit configured to perform image text recognition on a user physical examination text area image set included in the first physical examination item report image set to obtain a physical examination report text recognition information set; a text calibration unit configured to perform text calibration on the physical examination report text recognition information set according to the image detection text information set to obtain a physical examination report text calibration information set; a second generation unit configured to generate a report standardization prompt phrase group set according to the physical examination report standardization rule information set and the user physical examination report image set; a content extraction unit configured to perform content extraction on the physical examination report text calibration information set according to a physical examination report content extraction large language model and the report standardization prompt phrase group set to obtain a report extraction information set; and a storage unit configured to match and store the report extraction information set to a user physical examination report storage system and perform storage load optimization on the user physical examination report storage system.

[0009] In a third aspect, some embodiments of the present disclosure provide an electronic device, comprising: one or more processors; and a storage device having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the first aspect.

[0010] In a fourth aspect, some embodiments of the present disclosure provide a computer readable medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method described in any implementation manner of the first aspect.

[0011] The above various embodiments of the present disclosure have the following beneficial effects: the physical examination report standardization storage method of some embodiments of the present disclosure can improve the extraction accuracy and effect of physical examination report information, shorten the extraction time and reduce the waste of storage resources. Specifically, the reasons for the low accuracy and efficiency of related physical examination report content extraction, the existence of a large amount of redundant error data in the extracted content, and the waste of storage resources are as follows: because the traditional OCR algorithm is based on artificial features for text recognition, the accuracy of text recognition of complex and variable physical examination reports and image interspersed text is low and the recognition generalization ability is poor, and the pre-defined data extraction rules and templates are artificially defined and have certain subjectivity and professionalism, which cannot adapt to dynamic changes, resulting in low accuracy and efficiency of physical examination report content extraction, a large amount of redundant error data in the extracted content, waste of storage resources, and prolonged extraction time. Based on this, the physical examination report standardization storage method of some embodiments of the present disclosure can first obtain a user physical examination report image set and a physical examination report standardization rule information set, wherein the user physical examination report image includes a user physical examination image area image and a user physical examination text area image. Here, the user physical examination report image set and the physical examination report standardization rule information set are used for subsequent content extraction. Secondly, the above-mentioned user physical examination report image set is subjected to image enhancement processing to obtain an enhanced user physical examination report image set. Here, the image quality of the user physical examination report can be improved. Thirdly, the above-mentioned enhanced user physical examination report image set is subjected to physical examination item classification processing to obtain a first physical examination item report image set and a second physical examination item report image set. Here, image classification facilitates subsequent processing of different types of user physical examination reports to improve content extraction accuracy. Subsequently, according to the user physical examination image area image set included in the above-mentioned first physical examination item report image set, an image detection text information set is generated. Here, the image detection text information set is the result of recognizing the user physical examination image area image, which is used to supplement and correct the subsequently generated text information. Then, the user physical examination text area image set included in the above-mentioned first physical examination item report image set is subjected to image text recognition to obtain a physical examination report text recognition information set. Here, automatic text recognition can improve the accuracy and effect of recognition, and facilitate subsequent content extraction. After that, according to the above-mentioned image detection text information set, the above-mentioned physical examination report text recognition information set is subjected to text calibration to obtain a physical examination report text calibration information set. Here, the accuracy and comprehensiveness of the physical examination report can be improved. Then, according to the above-mentioned physical examination report standardization rule information set and the above-mentioned user physical examination report image set, a report standardization prompt phrase group set is generated. Here, the adaptability of the prompt phrase to the physical examination report standardization storage scene is improved, the accuracy of the output of subsequent content extraction using a large language model is improved, and the probability of the expected output is improved.Then, according to the content of the physical examination report, the large language model and the report standardization prompt set are extracted, and the content of the physical examination report text is extracted to obtain the report extraction information set. Here, the large language model is guided by the prompt to extract the content, which can improve the accuracy and efficiency of automatic content extraction and improve the data quality of the report extraction information. Finally, the report extraction information set is matched and stored in the user physical examination report storage system, and the storage load of the user physical examination report storage system is optimized. Here, the waste of storage resources of the user physical examination report storage system is reduced, and the load of the system is reduced. Therefore, the physical examination report standardization storage method can improve the extraction accuracy and effect of the physical examination report information, shorten the extraction time and reduce the waste of storage resources. BRIEF DESCRIPTION OF DRAWINGS

[0012] The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by describing in detail some embodiments thereof with reference to the attached drawings. The same or similar components have the same or similar reference labels. It should be understood that the drawings are schematic and elements and features are not necessarily drawn to scale.

[0013] Figure 1 is a flowchart of some embodiments of the physical examination report standardization storage method according to the present disclosure;

[0014] Figure 2 is a schematic diagram of a system page for standardizing storage of attribute values corresponding to internal echo attribute information of breast nodules in breast nodule physical examination items in some embodiments of the physical examination report standardization storage method according to the present disclosure;

[0015] Figure 3 is a schematic diagram of a system page for standardizing storage of attribute values corresponding to various attribute information in general detection physical examination items in some embodiments of the physical examination report standardization storage method according to the present disclosure;

[0016] Figure 4 is a schematic diagram of a system page for storing user physical examination reports by a user physical examination report storage system in some embodiments of the physical examination report standardization storage method according to the present disclosure;

[0017] Figure 5 is a structural schematic diagram of some embodiments of the physical examination report standardization storage device according to the present disclosure;

[0018] Figure 6 is a structural schematic diagram of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0019] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be interpreted as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be more thoroughly and completely understood. It should be understood that the drawings and embodiments of the present disclosure are only for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0020] In addition, it should be further noted that only parts related to the present application are shown in the drawings for ease of description. The embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0021] It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0022] It should be noted that the adjectives "one", "multiple" mentioned in the present disclosure are illustrative and not limiting, and those skilled in the art should understand that unless otherwise explicitly stated in the context, it should be understood as "one or more".

[0023] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of the messages or information.

[0024] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0025] Figure 1 The flow 100 of some embodiments of the physical examination report standardization storage method according to the present disclosure is shown. The physical examination report standardization storage method includes the following steps:

[0026] Step 101, obtaining a set of user physical examination report images and a set of physical examination report standardization rule information.

[0027] In some embodiments, the execution subject (e.g., an electronic device) of the medical report standardization storage method described above can obtain a user medical report image set and a medical report standardization rule information set through wired or wireless connection, wherein the user medical report image set includes a user medical imaging area image and a user medical text area image. The user medical report image in the user medical report image set can be an image showing the health of each organ of the user's body in the form of an image. The medical report standardization rule information in the medical report standardization rule information set can be a rule information defined by a person and output in a format extracted from the user medical report image. The user medical report image set can be a medical report of a third-party medical institution collected through a system interface. The medical report standardization rule information set can be obtained from a local database. The user medical imaging area image can be an image of a region where a medical image taken during the user's medical examination is placed. The user medical imaging text area image can be an image of a region where text appears in the user medical report image.

[0028] Step 102, image enhancement processing is performed on the user medical report image set to obtain an enhanced user medical report image set.

[0029] In some embodiments, the execution subject can perform image enhancement processing on the user medical report image set to obtain an enhanced user medical report image set.

[0030] In some optional implementations of some embodiments, the image enhancement processing on the user medical report image set to obtain an enhanced user medical report image set can include the following steps:

[0031] First, for each user medical imaging area image in the user medical imaging area image set corresponding to the user medical report image set, the following image fusion steps are performed:

[0032] Sub-step 1, pyramid reconstruction processing is performed on the user medical imaging area image to obtain a reconstructed imaging image sequence. The reconstructed imaging image sequence can be a sequence of images of low-frequency components and high-frequency detail components of multiple sizes obtained after down-sampling.

[0033] In practice, the execution subject can first use a Gaussian pyramid to perform Gaussian blur down-sampling on the user medical imaging area image to obtain a multi-scale cropped imaging image sequence. Then, a Laplacian pyramid is used to perform up-sampling difference processing on the multi-scale cropped imaging image sequence to obtain a Laplacian imaging image sequence as the reconstructed imaging image sequence.

[0034] Sub-step 2, determining an image fusion weight matrix of the reconstructed image sequence according to a scene illumination map sequence corresponding to the reconstructed image sequence. The scene illumination map can be a texture image based on a spherical coordinate system, used to simulate the illumination and reflection of the environment corresponding to the user's medical report image. The scene illumination map can be an image in which the maximum value of the three-channel brightness in the reconstructed image sequence is selected as the initial illumination estimation.

[0035] As an example, the execution subject can determine the image fusion weight matrix of the reconstructed image sequence according to the scene illumination map corresponding to the reconstructed image sequence by using an image energy minimization function. The image fusion weight matrix can be obtained by solving the image energy minimization function by a multi-resolution conjugate gradient method. The image energy minimization function can be:

[0036]

[0037] wherein, represents the reconstructed image. T represents the scene illumination map. M represents the image fusion weight matrix parameter. represents the difference filter of the reconstructed image. μ represents the smoothing constant. represents the square of the quadratic norm of the reconstructed image and the scene illumination map, representing the similarity between the reconstructed image and the scene illumination map. represents the first norm of the reconstructed image, representing the smoothing processing of the reconstructed image.

[0038] Sub-step 3, generating an image exposure rate value according to the image fusion weight matrix and the reconstructed image sequence. The image exposure rate value can represent the exposure degree of the image.

[0039] As an example, the execution subject can first filter at least one reconstructed image corresponding to the scene illumination map from the reconstructed image sequence by the normalized pixel value of each scene illumination map in the scene illumination map sequence, wherein the normalized pixel value of the corresponding scene illumination map is less than or equal to a preset pixel value threshold. The preset pixel value threshold can be a preset value. For example, the preset pixel value threshold can be 0.5. Then, the three-channel brightness geometric mean value function of the at least one reconstructed image is determined by a bidirectional texture luminance transformation function. The three-channel brightness geometric mean value function can be a function containing an image exposure rate parameter. The bidirectional texture luminance transformation function can be

[0040]

[0041] γ=k a .

[0042] wherein g(P, k) represents a bilateral texture luminance transform function. P represents a pixel value matrix normalized from each pixel value in a pixel value matrix of a reconstructed image image. k represents an image exposure rate parameter. β represents a control global gain coefficient, used to adjust the overall brightness of the reconstructed image image. γ represents a control gamma coefficient of a non-linear transform, affecting the degree of non-linear stretching / compression of the pixel value, and the value other than 1 represents that the bilateral texture luminance function is nonlinear to the camera. a represents the direction and degree of the influence of the exposure rate on the gamma coefficient, and when the exposure rate is greater than 0, a is -0.3293, and the gamma coefficient is negative. The above function represents a logarithmic compression of the pixel value, which is suitable for enhancing the details of the low exposure area. As the exposure rate increases, the gamma coefficient decreases, and the degree of nonlinearity decreases, which can avoid excessive enhancement of the exposure area. b represents the step of the change of the gain coefficient, and when b is greater than 0, the exposure rate increases, the exponential increases, and the global gain coefficient rapidly rises to improve the overall brightness. When the exposure rate decreases, the global gain coefficient decreases to avoid the dark part of the image being too dark. When the relevant parameters of the user physical examination image area image are not obtained, a is -0.3293, and b is 1.1258.

[0043] Finally, the above three-channel luminance geometric mean function is input into the image entropy function corresponding to the above reconstructed image image sequence, and the maximum image enhancement is solved to obtain the image exposure rate value.

[0044] Substep 4, according to the above image exposure rate value, the image mapping processing is performed on the above reconstructed image image sequence to obtain a multi-exposure image image sequence. Among them, the multi-exposure image image in the above multi-exposure image image sequence can be a reconstructed image image containing different exposure degrees. As an example, the above execution subject can input the above image exposure rate value into the bilateral texture luminance transform function to perform image mapping processing on the above reconstructed image image sequence to obtain a multi-exposure image image sequence.

[0045] Substep 5, according to the above image exposure rate value and the above image fusion weight matrix, the image fusion is performed on the above multi-exposure image image sequence and the above reconstructed image image sequence to obtain a fused image image.

[0046] As an example, the above execution subject can perform weighted summation on the above image exposure rate value and the image fusion weight matrix to obtain a fused image image.

[0047] Secondly, the image noise reduction processing is performed on the user physical examination text area image set corresponding to the above user physical examination report image set to obtain a noise-reduced text area image set.

[0048] In a third step, the above-mentioned denoised text region image set is subjected to a correction transformation process to obtain a corrected text region image set. The correction transformation process can include rotation correction and perspective transformation.

[0049] In a fourth step, the above-mentioned corrected text region image set and the obtained fused image set are determined as an enhanced user physical examination report image set.

[0050] In step 103, the enhanced user physical examination report image set is subjected to a physical examination item classification process to obtain a first physical examination item report image set and a second physical examination item report image set.

[0051] In some embodiments, the execution subject can perform a physical examination item classification process on the enhanced user physical examination report image set to obtain a first physical examination item report image set and a second physical examination item report image set. The first physical examination item report image in the first physical examination item report image set can be an image of a physical examination report with long text and multiple examination attributes. The long text can be text with more than 50 words. For example, the first physical examination item report image can be an image of a breast ultrasound examination room sub-physical examination item report. The multiple examination attributes can include at least one of the following: breast nodule boundary condition, breast nodule shape, nodule aspect ratio, breast nodule internal echo, breast nodule blood flow signal, breast nodule direction, and nodule property. The second physical examination item report image in the second physical examination item report image set can be an image of a physical examination report with short text or a few sentences. For example, the second physical examination item report image can be a general physical examination item report including user's height, weight, blood pressure, and other sub-items.

[0052] In practice, the execution subject can first perform a physical examination item segmentation on each enhanced user physical examination report image in the enhanced user physical examination report image set to obtain a physical examination item image group set. Then, the physical examination item image group set is classified according to the number of text words to obtain a first physical examination item report image set and a second physical examination item report image set.

[0053] In step 104, an image detection text information set is generated according to the user physical examination image region image set included in the first physical examination item report image set.

[0054] In some embodiments, the execution subject can generate an image detection text information set according to the user physical examination image region image set included in the first physical examination item report image set. The image detection text information in the image detection text information set can be text information describing the user physical examination image region image obtained by model intelligent recognition of the user physical examination image region image.

[0055] In some optional implementations of some embodiments, the above-mentioned generating the image detection text information set according to the user physical examination image area image set included in the first physical examination item report image set can include the following steps:

[0056] Firstly, for each user physical examination image area image in the user physical examination image area image set included in the first physical examination item report image set, the following generating steps are performed:

[0057] Substep 1: performing visual feature extraction on the user physical examination image area image to obtain an image visual feature vector. The image visual feature vector can represent color, texture, edge, shape, direction and other feature information of the user physical examination image area image. The visual feature extraction can be performed by using a ResNet-50 model.

[0058] Substep 2: inputting the image visual feature vector into a visual residual attention mechanism layer to obtain a visual residual attention feature vector, wherein the visual residual attention mechanism layer includes a visual local attention mechanism and a visual global attention mechanism. The visual attention feature vector can represent a vector with different weights for different parts of the user physical examination image area image. The visual local attention mechanism can be a model composed of a convolution layer and a nonlinear activation function in series, and the attention weight is determined by a sigmoid function. The visual local attention mechanism can be a deep neural network model including a first convolution layer, a ReLU (Rectified Linear Unit) activation function, a second convolution layer, a ReLU activation function, a third convolution layer, a Softmax activation function, a matrix multiplication of the image visual feature vector and the feature vector output by the Softmax activation function, and a pixel-by-pixel feature addition of the feature vector output by the matrix multiplication. The visual global attention mechanism can be a model that adds a parallel two-residual convolution layer to the visual local attention mechanism, and performs feature splicing on the two obtained feature vectors.

[0059] Substep 3: determining the Hadamard matrix product of the image visual feature vector and the visual residual attention feature vector to obtain a visual attention feature vector.

[0060] Substep 4, input the preset pathological attribute relationship graph into the spatial attention mechanism layer to obtain a set of pathological attribute abnormal feature vectors, wherein the preset pathological attribute relationship graph includes a plurality of abnormal pathological formation relationship graphs and a pathological attribute relationship graph corresponding to each abnormal pathology. The preset pathological attribute relationship graph can be an existing medical-related knowledge graph. The abnormal pathological formation relationship graph can be a knowledge graph composed of body parts with abnormal conditions in the user's medical image region image.

[0061] Substep 5, according to the preset pathological attribute feature vector, the visual attention feature vector and the set of pathological attribute abnormal feature vectors are fused to obtain a fusion embedding feature vector. The preset pathological attribute feature vector can represent the association between the abnormal image part in the user's medical image region image and the attribute information included.

[0062] Substep 6, performing graph-level attention operation on the fusion embedding feature vector to obtain a visual hierarchical feature vector. The visual hierarchical feature vector can represent the abnormal information, attribute information and context feature information between the previous time step in the fusion embedding feature vector. In practice, the execution subject can first input the fusion embedding feature vector into the GAT (Graph Attention Network) to obtain a fusion attention feature vector. Then, the fusion attention feature vector and the fusion attention feature vector of the previous time step are subjected to secondary attention operation to obtain a fusion context feature vector as the visual hierarchical feature vector.

[0063] Substep 7, input the visual hierarchical feature vector into the text generation decoder to generate a set of image detection text information. The text generation decoder can be a Transformer model containing multiple levels of multi-head attention mechanism.

[0064] Step 105, performing image text recognition on the user's medical text region image set included in the first medical project report image set to obtain a set of medical report text recognition information.

[0065] In some embodiments, the execution subject can perform image text recognition on the user's medical text region image set included in the first medical project report image set to obtain a set of medical report text recognition information. The medical report text recognition information in the set of medical report text recognition information can be information that displays the text in the user's medical text region image in text form.

[0066] In some optional implementations of some embodiments, the above-mentioned image text recognition on the user physical examination text region image set included in the above-mentioned first physical examination item report image set to obtain the physical examination report text recognition information set can include the following steps:

[0067] Firstly, for each user physical examination text region image in the user physical examination text region image set, the following decoding steps are performed:

[0068] Substep 1, a text box detection model is used to detect the text box of the user physical examination text region image to obtain a text region bounding box group. The text region bounding box in the text region bounding box group can be a rectangular box representing the position and range of the text in the user physical examination text region image. The text box detection model can be a deep neural network model for detecting the text region of the input user physical examination text region image. The text box detection model can be a model in which a channel and spatial attention mechanism is added to a YOLO v7-tiny (You Only Look Once v7-tiny) model.

[0069] Substep 2, the text region bounding box group is subjected to a character repair process to obtain a repaired text bounding box group. The character repair process can be a character repair process using an inflation and erosion method.

[0070] Substep 3, the repaired text bounding box group is subjected to adaptive feature extraction to obtain a text visual feature vector group. The text visual feature vector in the text visual feature vector group can represent the detailed information and text semantic information of the image of the repaired text bounding box. The adaptive feature extraction can be a feature extraction that first crops the image size of the repaired text bounding box group to adapt to the output of the convolutional network of the feature extraction.

[0071] Substep 4, the text visual feature vector group is input into a bidirectional long short-term memory neural network to obtain a text time sequence feature vector group. The text time sequence feature vector in the text time sequence feature vector group can represent the time sequence dependency information of the text characters.

[0072] Substep 5, the text time sequence feature vector group is subjected to feature sequence label prediction to obtain a character probability distribution feature vector group. The character probability distribution feature vector in the character probability distribution feature vector group can represent the probability value of each feature vector being recognized as a character. The feature sequence label prediction can be a label prediction using a fully connected layer and a Softmax activation function.

[0073] Sub-step 6. Perform text sequence decoding on the character probability distribution feature vector set to obtain the physical examination report text recognition information. The text sequence decoding can be performed by using a CTC (Connectionist Temporal Classification) algorithm.

[0074] Step 106. Perform text calibration on the physical examination report text recognition information set according to the image detection text information set to obtain a physical examination report text calibration information set.

[0075] In some embodiments, the execution subject can perform text calibration on the physical examination report text recognition information set according to the image detection text information set to obtain a physical examination report text calibration information set. The physical examination report text calibration information in the physical examination report text calibration information set can be text information obtained by filling and modifying the physical examination report text recognition information.

[0076] As an example, the execution subject can first perform attribute value matching between each image detection text information in the image detection text information set and the corresponding physical examination report text recognition information in the physical examination report text recognition information set to obtain a text matching result set. Then, in response to determining that the text matching result represents different attribute values, the execution subject can obtain re-detection result information of the physical examination item with different attribute values by different medical personnel, and determine the re-detection result information as the physical examination report text calibration information. Finally, in response to determining that the text matching result represents a mismatch, the execution subject can determine the image detection text information as the physical examination report text calibration information.

[0077] Step 107. Generate a report standardization prompt phrase group set according to the physical examination report standardization rule information set and the user physical examination report image set.

[0078] In some embodiments, the execution subject can generate a report standardization prompt phrase group set according to the physical examination report standardization rule information set and the user physical examination report image set. The report standardization prompt phrase in the report standardization prompt phrase group set can be text information for guiding a large language model to output results in a preset output format.

[0079] As an example, the execution subject can input the physical examination report standardization rule information set and the user physical examination report image set into a large language model to obtain a report standardization prompt phrase group set. The large language model can be a GPT model (Generative Pre-trained Transformer).

[0080] In the process of adopting the technical solutions to solve the above technical problem one, the following technical problem two is often accompanied: due to the large language model contains a large number of parameters, how to train or train part of the parameters, so that the large language model has pertinence, the information extraction of the physical examination report is more pertinence, improve the accuracy and efficiency of extraction. In view of the above technical problem two, the conventional solution is generally: through the artificially constructed prompt language model to guide the large language model to extract the content of the physical examination report. However, the above conventional solution still has the following problems: because the artificially constructed prompt language model needs to be mapped through a large number of templates and the output results of the large language model to try continuously, it is time-consuming and the model output accuracy is low, resulting in low accuracy and efficiency of subsequent content extraction, the large language model needs to repeatedly extract the content, improves the system load and wastes a lot of training resources. And the inventor considers the shortcomings of the above conventional solution, and combines the advantages / technical status of the automatic generation of prompt language technology owned by the company where the inventor is located, and we decide to adopt the following solution:

[0081] In some optional implementations of some embodiments, the above generating the report standardization prompt language group set according to the above physical examination report standardization rule information set and the above user physical examination report image set can include the following steps:

[0082] Firstly, the physical examination item information extraction is performed on the physical examination report text calibration information set corresponding to the above user physical examination report image set, and the physical examination item field set is obtained. The physical examination item field in the physical examination item field set can be the field of the item name or item number of the physical examination performed by the user. For example, the physical examination item field can be but not limited to at least one of the following: ophthalmology, internal medicine, stomatology, blood routine.

[0083] Secondly, according to the above physical examination item field set, at least one physical examination report standardization rule information corresponding to each user physical examination report image in the above user physical examination report image set is determined as a target standardization rule information group, and a target standardization rule information group set is obtained.

[0084] As an example, the above execution subject can first determine the report physical examination item group set of the physical examination report text calibration information set corresponding to the above user physical examination report image set. Then, through the report physical examination item group set, the physical examination item matching is performed on the user physical examination report image set and the physical examination report standardization rule information set, at least one physical examination report standardization rule information of each user physical examination report image is obtained, and a target standardization rule information group set is obtained.

[0085] Thirdly, for each physical examination report text calibration information in the above physical examination report text calibration information set, the following prompt language generation step is performed:

[0086] Sub-step 1, the above physical examination report text calibration information is segmented to obtain a physical examination report segmented paragraph text group. The physical examination report segmented paragraph text in the physical examination report segmented paragraph text group can be a paragraph text segmented according to a sentence.

[0087] Sub-step 2, dependency syntax analysis is performed on the above physical examination report segmented paragraph text group and the initial input prompt, to obtain a paragraph dependency syntax tree group and an initial prompt dependency syntax tree, wherein the initial input prompt includes a target standardization rule information group. The initial input prompt can be a relatively simple prompt constructed by a user, which includes a preset extraction or output constraint. The paragraph dependency syntax tree in the paragraph dependency syntax tree group can be a tree structure constructed according to the grammatical relationship of each segmented word in the physical examination report segmented paragraph text. The initial prompt dependency syntax tree can be a tree structure composed according to the grammatical relationship of each prompt word in the initial input prompt. The dependency syntax analysis can be a syntax analysis performed by using a HanLP (Han Language Processing) natural language tool.

[0088] Sub-step 3, dependency graph conversion is performed on the above paragraph dependency syntax tree group and the initial prompt dependency syntax tree, to obtain a paragraph dependency graph group and an initial prompt dependency graph. The paragraph dependency graph in the paragraph dependency graph group can represent the relationship between each segmented word in the form of a graph. The initial prompt dependency graph can represent the association relationship between each prompt analysis in the form of a graph. The dependency graph conversion can be a graph conversion performed by using a DGL (Deep Graph Library) graph neural network library. It should be noted that, in order to avoid the situation that the segmented words of HanLP and the segmented words output by the DGL model are different, it is determined whether the dependency tree is in conflict with the model when the dependency tree is constructed, and a table of conflicts between the two is maintained. When there is a conflict, a minimum granularity segmentation method is adopted to ensure that the coding of the dependency tree is correct.

[0089] Sub-step 4, performing graph convolution processing on the paragraph dependency graph set and the initial prompt dependency graph to obtain a paragraph semantic syntax feature vector set and an initial prompt semantic syntax feature vector set. The paragraph semantic syntax feature vector in the paragraph semantic syntax feature vector set can represent information of various syntactic relationships (such as nominal subject, direct object, possession modifier, preposition modifier, and object of preposition) between the segmented word nodes and the neighbor segmented word nodes included in the paragraph dependency graph. The initial prompt semantic syntax feature vector in the initial prompt semantic syntax feature vector set can represent information of various syntactic relationships between the prompt segmented word nodes and the neighbor prompt segmented word nodes included in the initial prompt dependency graph. The graph convolution processing can be inputting the paragraph dependency graph set and the initial prompt dependency graph into a RGCN (Relational Graph Convolutional Network) model.

[0090] Sub-step 5, inputting the paragraph semantic syntax feature vector set and the initial prompt semantic syntax feature vector set into a time sequence extraction network, a multi-layer perception, and a layer normalization layer in sequence to obtain a paragraph semantic syntax feature vector set and an initial prompt semantic syntax feature vector set. The paragraph semantic syntax feature vector in the paragraph semantic syntax feature vector set can represent semantic and syntactic information of each segmented word node. The initial prompt semantic syntax feature vector in the initial prompt semantic syntax feature vector set can represent semantic and syntactic information of the prompt segmented word node. The time sequence extraction network can be a recurrent neural network for extracting time sequence information of the paragraph semantic syntax feature vector set and the initial prompt semantic syntax feature vector set respectively. For example, the time sequence extraction network can be a bidirectional long short-term memory neural network. The multi-layer perception can be an MLP (Multi Layer Perceptron) containing two linear layers and a ReLU activation function. The normalization layer can be a neural network layer that can make the loss value more stable and prevent gradient vanishing and explosion problems.

[0091] Sub-step 6: The paragraph semantic and syntactic feature vector set and the initial prompt semantic and syntactic feature vector set are subjected to graph attention pooling processing to obtain a report prompt feature vector set. The report prompt feature vector in the report prompt feature vector set can represent information of attention weights of feature information fused with semantics and syntax. In practice, the execution subject can first determine a set of attention weight values of the paragraph semantic and syntactic feature vector set and the initial prompt semantic and syntactic feature vector set through a multi-layer perception set. Then, the set of attention weight values is averaged after adding a second-order adjacency matrix to obtain a set of average attention weight values. Subsequently, the first preset number of semantic and syntactic feature vectors sorted from large to small according to the set of average attention weight values are screened from the paragraph semantic and syntactic feature vector set and the initial prompt semantic and syntactic feature vector set for pooling mask processing to obtain the report prompt feature vector set. The preset number can be the number of word segmentation of the finally generated prompt.

[0092] Sub-step 7: The report prompt feature vector set is input into a text generation model to obtain a report prompt set. The report prompt in the report prompt set can be a prompt obtained by fusing the semantics and syntax of the set of standardized rule information of the physical examination report and the set of user physical examination report images. The text generation model can be a deep neural network model that maps the input report prompt feature vector to text. For example, the text generation model can be a BERT (Bidirectional Encoder Representations from Transformers) model.

[0093] Sub-step 8: The report prompt set is subjected to thought chain bundle search optimization to obtain an optimized report prompt set. The optimized report prompt in the optimized report prompt set can be a prompt that performs intermediate reasoning on the subsequent content extraction process to simplify the reasoning task in each step. The thought chain bundle search optimization can be in the form of a thought tree, drawing on the idea of bundle search, expanding the search space of the prompt, and enabling the physical examination report content extraction large language model to generate multiple potential prompts in each iteration, i.e., branches of the thought tree. The prompts with poor effects are eliminated, and the top 5 prompts with the best effects are selected to enter the next round of optimization. Multiple iterations, i.e., the depth of the thought tree, deepen the levels of model exploration and gradually optimize the prompts automatically.

[0094] Step 9, according to the prompt feedback information corresponding to the optimized report prompt group, the report prompt is optimized, and the report standardized prompt group is obtained. The prompt feedback information in the prompt feedback information group can be the feedback information that the content extraction large language model extracts through the prompt extraction.

[0095] The above technical solution, combined with "step 108" and "step 109", and related content as an invention point of an embodiment of the present disclosure, solves the second technical problem mentioned in the background art: due to the need for human-constructed prompts to be mapped through a large number of templates and large language model output results for continuous attempts, time-consuming and low model output accuracy, resulting in low accuracy and efficiency of subsequent content extraction, large language model needs to repeatedly extract content, improves system load and wastes a lot of training resources. The factors that lead to low accuracy and efficiency of subsequent content extraction, large language model needs to repeatedly extract content, improve system load and waste a lot of training resources are often as follows: due to the need for human-constructed prompts to be mapped through a large number of templates and large language model output results for continuous attempts, time-consuming and low model output accuracy. If the above factors are solved, the accuracy and efficiency of subsequent content extraction can be improved, the number of content extraction of large language model can be reduced, the system load can be reduced, and the waste of training resources can be reduced. In order to achieve this effect, the present disclosure first determines at least one standardized report rule information corresponding to each user medical report image, so as to be more suitable for medical content extraction task. Secondly, the dependency syntax tree and the dependency graph are constructed, and the syntax information of the paragraph and the initial prompt can be extracted. Then, the dependency graph is subjected to graph convolution, time sequence extraction, multi-layer perception, layer normalization, attention mechanism and text generation, which fuses the semantic information of multiple syntax relationships of neighbor word segmentation nodes, and the inherent sequence of natural language and the context relationship of words in natural text, which can improve the performance of the report prompt. The graph attention pooling determines the important nodes through the weight value, which can reduce the data amount of the node feature vector and improve the generation efficiency. Finally, the report prompt group is subjected to thought chain bundle search optimization and reflection optimization, which can further improve the performance of the prompt and the applicability of content extraction, improve the accuracy of the prompt, reduce the training times of the medical report content extraction large language model, reduce the system load and resource consumption, and improve the prompt word generation efficiency.

[0096] Step 108, according to the medical report content extraction large language model and the report standardized prompt group set, the medical report text calibration information set is subjected to content extraction, and the report extraction information set is obtained.

[0097] In some embodiments, the execution subject can extract a large language model according to the content of the physical examination report and the report standardization prompt set to extract the report extraction information set from the physical examination report text calibration information set. The physical examination report content extraction large language model can be an input content extraction prompt to guide the large language model to extract the physical examination item attribute information and attribute value from the physical examination report text calibration information set. For example, the physical examination report content extraction large language model can be a Chinese LLaMA2-7B (Chinese Large Language Model Meta AI 2-7B) model. Chinese indicates a large language model for Chinese language. 7B indicates that the model has 700 million parameters. The report extraction information in the report extraction information set can be the information of the extracted physical examination item attribute information and the corresponding attribute value in the physical examination report text calibration information. For example, the report extraction information can be information of multiple attribute values representing no echo, low echo, and high echo information corresponding to the breast nodule internal echo attribute information. As shown in Figure 2 and Figure 3 Figure 2 shows a system page for storing the standardized extraction of the attribute values corresponding to the breast nodule internal echo attribute information according to the rules. Figure 3 shows a system page for storing the extraction of each attribute value in the general examination item according to the physical examination report standardization rule information.

[0098] As an example, the execution subject can input the report standardization prompt set and the physical examination report text calibration information set into the physical examination report content extraction large language model to obtain the report extraction information set.

[0099] In some optional implementations of some embodiments, the content extraction of the physical examination report text calibration information set according to the physical examination report content extraction large language model and the report standardization prompt set to obtain the report extraction information set can include the following steps:

[0100] First, obtain the physical examination report knowledge base and model fine-tuning sample corresponding to the physical examination report text calibration information set. The physical examination report knowledge base can be an existing knowledge graph related to the physical examination report. The model fine-tuning sample can be a physical examination report image used to fine-tune the parameters of the physical examination report content extraction large language model. The model fine-tuning sample is a sample containing labeled data with correct content extraction.

[0101] Second, according to the physical examination report knowledge base, knowledge edit the physical examination report content extraction large language model to obtain a knowledge edited content extraction model. ​

[0102] As an example, the execution subject can utilize a MEMIT (Mass-Editing Memory in a Transformer) knowledge editing algorithm to extract a large language model from the physical examination report content according to a physical examination report knowledge base, and obtain a knowledge-edited content extraction model.

[0103] In the third step, the initial weight matrix of the large language model of the physical examination report content is decomposed to obtain an amplitude vector and a direction matrix. The amplitude direction can represent the control weight update step size, which is a scalar parameter. The direction matrix can represent the direction of weight update under low-rank decomposition.

[0104] In the fourth step, the direction matrix is updated by low-rank matrix according to the model fine-tuning sample and the knowledge-edited content extraction model, to obtain an updated direction matrix.

[0105] As an example, the execution subject can first determine the first low-rank matrix and the second low-rank matrix of the direction matrix by using the LoRA (Low-Rank Adaptation) algorithm. Then, the mean square error loss function value of the knowledge-edited content extraction model is determined by using the model fine-tuning sample. Finally, the first low-rank matrix and the second low-rank matrix are iterated by using the mean square error loss function value and the Adam optimizer until the mean square error loss function value is less than or equal to the preset loss threshold, to obtain the updated direction matrix.

[0106] In the fifth step, the amplitude vector is updated by back propagation according to the model fine-tuning sample and the knowledge-edited content extraction model, to obtain an updated amplitude matrix.

[0107] As an example, the execution subject can iterate the amplitude vector by using the mean square error loss function value and the Adam optimizer until the mean square error loss function value is less than or equal to the preset loss threshold, to obtain the updated amplitude matrix.

[0108] In the sixth step, the knowledge-edited content extraction model is fine-tuned according to the updated direction matrix and the updated amplitude matrix, to obtain a fine-tuned content extraction model.

[0109] As an example, the execution subject can first determine the matrix multiplication of the updated direction matrix and the updated amplitude matrix as a model parameter matrix. Then, the original parameter matrix of the knowledge-edited content extraction model is replaced by the model parameter matrix to obtain the fine-tuned content extraction model.

[0110] In the seventh step, the report standardization prompt phrase group set and the fine-tuned content extraction model are used to perform multi-round question and answer content extraction on the physical examination report text calibration information set to obtain a report extraction information set. The multi-round question and answer content extraction can be performed by using the report standardization prompt phrase group to guide the fine-tuned content extraction model to extract content step by step.

[0111] In step 109, the report extraction information set is matched and stored in the user physical examination report storage system, and the storage load of the user physical examination report storage system is optimized.

[0112] In some embodiments, the report extraction information set can be matched and stored in the user physical examination report storage system, and the storage load of the user physical examination report storage system can be optimized. The user physical examination report storage system can be a server cluster for storing the extracted report extraction information set. The storage load optimization can be performed by using a heuristic algorithm. As shown in FIG. 11, a system page for storing the attribute information of the physical examination items corresponding to the physical examination report image of the user in the user physical examination report storage system is displayed. Figure 4 Figure 4 The left part is the user physical examination report, and the right part is the stored area part displayed in the user physical examination report storage system.

[0113] In the process of solving the above technical problem one by using the technical solution, the following technical problem three is often accompanied: how to store a large amount of structured data extracted from the standardized content of the physical examination report to balance the load and storage efficiency of the user physical examination report storage system. In view of the above technical problem three, the conventional solution is generally to use hash sharding and hash indexing to optimize the storage load of the user physical examination report storage system. However, the conventional solution still has the following problems: since hash sharding divides data according to fixed rules, it cannot adapt to the dynamic changes of the report extraction information set, and there is a problem that the data is distributed in the same shard, causing high node load. In addition, the hash index needs to be queried across multiple shards for full table scan queries, which consumes a lot of system server communication, resulting in high load and low storage efficiency of the user physical examination report storage system. The inventors considered the shortcomings of the conventional solution and combined the advantages / technical status of the storage load optimization technology owned by the company where the inventors work, and decided to use the following solution:

[0114] In some optional implementations of some embodiments, the matching and storing of the report extraction information set in the user physical examination report storage system and the optimization of the storage load of the user physical examination report storage system can include the following steps:

[0115] ​In the first step, the report extraction information set is converted to obtain a report conversion extraction information set. In practice, the execution subject can first convert the punctuation symbols in the report extraction information set into Chinese symbols. Then, the report extraction information set is segmented by using the line break in the Chinese symbols and stored in an ordered set (list set). Finally, the ordered set is assembled into a triple set containing the number, attribute information, and attribute value according to the length of the subscript in the ordered set, as the report conversion extraction information set. The subscript length can be 3 or 4.

[0116] In the second step, the examination item number set included in the report conversion extraction information set is used to determine the storage location information set of the report conversion extraction information set in the user examination report storage system. The storage location information in the storage location information set can represent the logical storage address of the report conversion extraction information set in the user examination report storage system.

[0117] In the third step, the data access probability of each storage data corresponding to each storage location information set in the storage location information set is determined by using the historical data query statement set to obtain a data access probability set as the initial access probability set of the report conversion extraction information set. The data access probability in the data access probability set can represent the access frequency of the storage data. The data access probability can be the ratio of the number of query statements containing each storage data in the historical access query statement to the amount of historical access query statements. The historical data query statement in the historical data query statement set can be a query statement for accessing the storage data in the user examination report storage system before the current time.

[0118] In the fourth step, the data association probability matrix of the storage data set having an association relationship with the report conversion extraction information set is determined according to the storage location information set and the initial access probability set. Each element in the data association probability matrix can represent the association degree of the storage data and the report conversion extraction information. The association relationship can include the existence of the primary-foreign key relationship and the co-occurrence relationship in the query statement.

[0119] As an example, the execution subject can first determine each of the probability values of the storage value corresponding to the storage location information set as a preset correlation value, because the storage location information set is determined by the physical examination item, and there is a primary key consistency problem. The preset correlation value can be a preset probability value. For example, the preset correlation value can be 1. Second, the correlation probability value of at least one storage data of the existence appearance relationship of each storage data corresponding to the report conversion extraction information is determined by the storage location information set. The preset correlation value and the initial access probability corresponding to the initial access probability set are weighted and summed to obtain a data correlation probability matrix.

[0120] In the fifth step, according to the data correlation probability matrix, the fuzzy storage data cluster to which each report conversion extraction information in the report conversion extraction information set belongs is determined to obtain a fuzzy storage data cluster set. Each fuzzy storage data cluster in the fuzzy storage data cluster set includes a report clustering center, a report membership matrix, and a report storage node address. The fuzzy storage data cluster in the fuzzy storage data cluster set can be a data set composed of report conversion extraction information obtained by fuzzy C-means clustering based on the data correlation probability matrix. The report storage node address can be the address of the system server.

[0121] As an example, the execution subject can utilize the fuzzy C-means clustering algorithm to determine the fuzzy storage data cluster to which each report conversion extraction information in the report conversion extraction information set belongs according to the data correlation probability matrix to obtain a fuzzy storage data cluster set. The data correlation probability matrix is used as an initialized membership matrix. The objective function of the fuzzy C-means clustering algorithm can be the original fuzzy C-means algorithm objective function plus an access cost term and a penalty term. The access cost term can include access frequency cost, correlation cost, and cross-node communication delay cost. The penalty term can be a penalty term for strong correlation between report conversion extraction information and storage data but large difference in membership.

[0122] In the sixth step, according to the fuzzy storage data cluster set, a report predictive index tree, a report balanced multi-query index tree, and a report leaf index tree of the report conversion extraction information set are determined. The report predictive index tree can be an index tree for predicting the shard range of the report conversion extraction information set. The report balanced multi-query index tree can be an index tree for determining the accurate boundary of a range query to avoid errors in the report predictive index tree and to handle complex conditions. The report leaf index tree can be an index tree for recording the report clustering center, the report membership matrix, and the report storage node address.

[0123] As an example, the execution subject can first convert the report in the fuzzy storage data cluster set into a report conversion extraction information set and input it into a data shard prediction model to obtain a report predictive index tree. The data shard prediction model can be a multi-layer perceptron that predicts the data shard of the report conversion extraction information set by extracting the association relationship between the historical query statement and the corresponding data shard of the historical query statement. Then, a report balanced multi-path query index tree of the fuzzy storage data cluster set is constructed using a B+ tree construction method. Finally, the report cluster center, the report membership matrix, and the report storage node are stored in the leaf node to determine the report leaf index tree.

[0124] Step 7: According to the report predictive index tree, the report balanced multi-path query index tree, and the report leaf index tree, the database index tree corresponding to the user physical examination report storage system is indexed tree combined to obtain a report hybrid index tree. The report hybrid index tree can be a tree structure used to locate the storage location of data.

[0125] As an example, the execution subject can take the report predictive index tree as the root node of the report hybrid index tree, take the report balanced multi-path query index tree as the middle layer of the report hybrid index tree, and take the report leaf index tree as the leaf layer of the report hybrid index tree.

[0126] Step 8: A server storage node network diagram of the user physical examination report storage system is generated, wherein each edge weight of the server storage node network diagram represents the communication delay between nodes. The server storage node network diagram can represent the communication connection relationship between the servers of the user physical examination report storage system.

[0127] Step 9: According to the report hybrid index tree and the server storage node network diagram, the user physical examination report storage system is load balanced to determine the server storage node set where the report conversion extraction information set is located, and the data storage of the report conversion extraction information set is completed.

[0128] As an example, the execution subject can use a reinforcement learning strategy to map each shard included in the report hybrid index tree to the server storage node network diagram to load balance the user physical examination report storage system, determine the server storage node set where the report conversion extraction information set is located, and complete the data storage of the report conversion extraction information set.

[0129] The technical scheme and related content thereof serve as one of the invention points of the embodiments of the present disclosure, and solve the third technical problem mentioned in the background. That is, because the hash partitioning adopts a fixed rule to divide data, it cannot adapt to the dynamic changes of the report extraction information set, and there is a problem of high node load caused by the distribution of data in the same partition, and a problem of high system server communication consumption caused by the need for cross-partition query for full table scan query of the hash index, which leads to high load and low storage efficiency of the user health report storage system. The factors leading to high load and low storage efficiency of the user health report storage system are usually as follows: because the hash partitioning adopts a fixed rule to divide data, it cannot adapt to the dynamic changes of the report extraction information set, and there is a problem of high node load caused by the distribution of data in the same partition, and a problem of high system server communication consumption caused by the need for cross-partition query for full table scan query of the hash index. If the above factors are solved, the effect of reducing the complexity of the user health report storage system and improving the storage efficiency can be achieved. In order to achieve this effect, the present disclosure first determines the fuzzy storage data cluster set through the determined data correlation probability matrix, dynamically adjusts the data partitioning of the report conversion extraction information set, and avoids the load imbalance problem caused by the static rule. Then, three index trees are constructed and combined, the report predictive index tree dynamically optimizes the query path, which can reduce full scan and time, the report balanced multi-path query index tree can provide stable and reliable accurate query to make up for the error and complex query of the report predictive index tree, and the report leaf index tree can accurately physically store and locate to ensure the accuracy and efficiency of data physical location. Finally, the server storage node network diagram and the report hybrid index tree are generated to perform load balancing processing, which can improve the storage efficiency of the user health report storage system and reduce the system load.

[0130] Optionally, the execution subject can further perform the following steps after 109:

[0131] Firstly, at least one health report standardization rule information matched with the second health project report image set is screened out from the health report standardization rule information set.

[0132] Secondly, image text recognition is performed on the second health project report image set to obtain a health report text information set. The health report text information in the health report text information set can be text information obtained by character recognition on the text in the image.

[0133] Thirdly, the medical examination report standardization rule information and the medical examination report text information set are matched and extracted to obtain a user medical examination item information set and a corresponding user medical examination item attribute information set, wherein the user medical examination item attribute information includes a user medical examination item attribute and a user medical examination item attribute value. The user medical examination item attribute can be the name of a medical examination item or a sub-item. For example, the user medical examination item attribute can be the systolic pressure of general examination. The user medical examination item attribute value can be the value corresponding to the name of the medical examination item or the sub-item. For example, the user medical examination item attribute value can be the systolic pressure of 112.

[0134] Fourthly, the user medical examination item attribute information set is matched and stored in the user medical examination report storage system according to the user medical examination item information set.

[0135] Further referring to Figure 5 , as an implementation of the method shown in the above figures, the disclosure provides some embodiments of a medical examination report standardization storage device, which corresponds to the method embodiments shown in the above figures, and the medical examination report standardization storage device can be applied in various electronic devices. Figure 1

[0136] As shown in Figure 5 ​As shown, a physical examination report standardization storage device 500 includes an acquisition unit 501, an image enhancement unit 502, a physical examination item classification unit 503, a first generation unit 504, an image text recognition unit 505, a text calibration unit 506, a second generation unit 507, a content extraction unit 508, and a storage unit 509. The acquisition unit 501 is configured to acquire a user physical examination report image set and a physical examination report standardization rule information set, wherein the user physical examination report image set includes a user physical examination image area image and a user physical examination text area image. The image enhancement unit 502 is configured to perform image enhancement processing on the user physical examination report image set to obtain an enhanced user physical examination report image set. The physical examination item classification unit 503 is configured to perform physical examination item classification processing on the enhanced user physical examination report image set to obtain a first physical examination item report image set and a second physical examination item report image set. The first generation unit 504 is configured to generate an image detection text information set according to the user physical examination image area image set included in the first physical examination item report image set. The image text recognition unit 505 is configured to perform image text recognition on the user physical examination text area image set included in the first physical examination item report image set to obtain a physical examination report text recognition information set. The text calibration unit 506 is configured to perform text calibration on the physical examination report text recognition information set according to the image detection text information set to obtain a physical examination report text calibration information set. The second generation unit 507 is configured to generate a report standardization prompt phrase group set according to the physical examination report standardization rule information set and the user physical examination report image set. The content extraction unit 508 is configured to perform content extraction on the physical examination report text calibration information set according to a physical examination report content extraction large language model and the report standardization prompt phrase group set to obtain a report extraction information set. The storage unit 509 is configured to match and store the report extraction information set to a user physical examination report storage system and perform storage load optimization on the user physical examination report storage system.

[0137] It can be understood that the units described in the physical examination report standardization storage device 500 correspond to the respective steps in the method described above. Figure 1 The operations, features, and advantages described above for the method also apply to the physical examination report standardization storage device 500 and the units included therein, and will not be repeated here.

[0138] Reference is made below to Figure 6 which shows a structural schematic diagram of an electronic device (e.g., an electronic device) 600 suitable for implementing some embodiments of the present disclosure. Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0139] AsFigure 6 As shown, the electronic device 600 can include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601 that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 602 or loaded into a random access memory (RAM) 603 from a storage device 608. Various programs and data required for the operation of the electronic device 600 are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0140] Generally, the following devices can be connected to the I / O interface 605: input devices 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 608 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 609. The communication devices 609 can allow the electronic device 600 to communicate wirelessly or wired with other devices to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that all of the illustrated devices are not required, and more or fewer devices can alternatively be implemented. Figure 6 Each block shown in the flowcharts can represent a device, or a plurality of devices, as needed.

[0141] In particular, processes described above with reference to the flowcharts can be implemented as a computer software program according to some embodiments of the present disclosure. For example, some embodiments of the present disclosure include a computer program product including a computer program carried on a computer readable medium, the computer program containing program codes for performing the methods shown in the flowcharts. In some such embodiments, the computer program can be downloaded and installed from a network through the communication devices 609, or installed from the storage devices 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-described functions defined in the methods of some embodiments of the present disclosure are performed.

[0142] It is an aspect of the present disclosure that the computer-readable medium described above can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium may, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any combination thereof. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In some embodiments of the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such a propagated data signal can take a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium that is not a storage medium and that can be used to carry or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained in the computer-readable medium can be transmitted by any suitable medium, including, but not limited to, a wire, an optical fiber, an RF (radio frequency) or the like, or any suitable combination thereof.

[0143] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (Hyper Text Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0144] The computer-readable medium described above can be included in the electronic device described above; or can exist separately from the electronic device and be not assembled in the electronic device. The computer-readable medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to implement the steps of steps 101 to 109.

[0145] Computer program code for carrying out operations of some embodiments of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0146] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in some cases, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and / or flow diagrams and combinations of blocks in the block diagrams and / or flow diagrams can be implemented by a dedicated hardware-based system that carries out specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0147] The units described in some embodiments of the present disclosure can be implemented by means of software, or can be implemented by hardware. The described units can also be provided in a processor, for example, it can be described that a processor includes an acquisition unit, an image enhancement unit, a physical examination item classification unit, a first generation unit, an image text recognition unit, a text calibration unit, a second generation unit, a content extraction unit and a storage unit. Among them, the name of these units does not constitute a limitation to the unit itself in some cases, for example, the acquisition unit can also be described as "a unit for acquiring a user physical examination report image set and a physical examination report standardization rule information set".

[0148] The functionality described herein above can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program- specific Integrated Circuits (ASICs), Program- specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0149] The above description is merely exemplary of the disclosure and the application made use of the principles of the technology. It is to be understood that the application scope of the embodiments of the disclosure is not limited to the specific combinations of technical features described above, and should also cover other technical solutions formed by any combination of the above technical features or equivalent features thereof without departing from the inventive concept. For example, the technical solutions formed by replacing the above features with technical features having similar functions disclosed in the embodiments of the disclosure (but not limited to) with each other.

Claims

1. A physical examination report standardization storage method, comprising: obtaining a user physical examination report image set and a physical examination report standardization rule information set, wherein the user physical examination report image set comprises a user physical examination image area image and a user physical examination text area image; performing image enhancement processing on the user physical examination report image set to obtain an enhanced user physical examination report image set; performing physical examination item classification processing on the enhanced user physical examination report image set to obtain a first physical examination item report image set and a second physical examination item report image set; generating an image detection text information set according to a user physical examination image area image set included in the first physical examination item report image set, comprising: for each user physical examination image area image in the user physical examination image area image set included in the first physical examination item report image set, performing the following generation steps: performing visual feature extraction on the user physical examination image area image to obtain an image visual feature vector; inputting the image visual feature vector into a visual residual attention mechanism layer to obtain a visual residual attention feature vector, wherein the visual residual attention mechanism layer comprises a visual local attention mechanism and a visual global attention mechanism; determining a Hadamard matrix product of the image visual feature vector and the visual residual attention feature vector to obtain a visual attention feature vector; inputting a preset pathological attribute relationship graph into a spatial attention mechanism layer to obtain a pathological attribute abnormal feature vector set, wherein the preset pathological attribute relationship graph comprises a relationship graph of multiple abnormal pathologies and a pathological attribute relationship graph corresponding to each abnormal pathology; performing feature embedding fusion on the visual attention feature vector and the pathological attribute abnormal feature vector set according to a preset pathological attribute feature vector to obtain a fusion embedding feature vector; performing graph hierarchical attention operation on the fusion embedding feature vector to obtain a visual hierarchical feature vector; inputting the visual hierarchical feature vector into a text generation decoder to generate an image detection text information set; performing image text recognition on a user physical examination text area image set included in the first physical examination item report image set to obtain a physical examination report text recognition information set; performing text calibration on the physical examination report text recognition information set according to the image detection text information set to obtain a physical examination report text calibration information set; According to the physical examination report standardization rule information set and the user physical examination report image set, a report standardization prompt phrase group set is generated, including: performing physical examination item information extraction on the physical examination report text calibration information set corresponding to the user physical examination report image set to obtain a physical examination item field set; determining at least one physical examination report standardization rule information corresponding to each user physical examination report image in the user physical examination report image set as a target standardization rule information group according to the physical examination item field set, to obtain a target standardization rule information group set; for each physical examination report text calibration information in the physical examination report text calibration information set, the following prompt phrase generation steps are performed: segmenting the physical examination report text calibration information to obtain a physical examination report segmented paragraph text group; performing dependency syntax analysis on the physical examination report segmented paragraph text group and an initial input prompt phrase to obtain a paragraph dependency syntax tree group and an initial prompt dependency syntax tree, wherein the initial input prompt phrase includes the target standardization rule information group; performing dependency graph conversion on the paragraph dependency syntax tree group and the initial prompt dependency syntax tree to obtain a paragraph dependency graph group and an initial prompt dependency graph; performing graph convolution processing on the paragraph dependency graph group and the initial prompt dependency graph to obtain a paragraph dependency node feature vector group set and an initial prompt node feature vector group; sequentially inputting the paragraph dependency node feature vector group set and the initial prompt node feature vector group into a time sequence extraction network, a multi-layer perception and a layer normalization layer to obtain a paragraph semantic syntax feature vector group set and an initial prompt semantic syntax feature vector group; performing graph attention pooling processing on the paragraph semantic syntax feature vector group set and the initial prompt semantic syntax feature vector group to obtain a report prompt phrase feature vector group; inputting the report prompt phrase feature vector group into a text generation model to obtain a report prompt phrase group; performing thought chain bundle search optimization on the report prompt phrase group to obtain an optimized report prompt phrase group; performing reflection optimization on the optimized report prompt phrase according to the prompt phrase feedback information group corresponding to the optimized report prompt phrase group to obtain a report standardization prompt phrase group, wherein the prompt phrase feedback information group is feedback information that the content extracted by the physical examination report content extraction large language model through prompt phrase extraction is abnormal; According to the physical examination report content extraction large language model and the report standardization prompt phrase group set, content extraction is performed on the physical examination report text calibration information set to obtain a report extraction information set; The report extraction information set is matched and stored to the user physical examination report storage system, and the user physical examination report storage system is stored for load optimization.

2. The method of claim 1, wherein, The method further comprises: screening at least one physical examination report standardization rule information matching the second physical examination item report image set from the physical examination report standardization rule information set; performing image text recognition on the second physical examination item report image set to obtain a physical examination report text information set; The at least one physical examination report standardization rule information and the physical examination report text information set are matched and extracted for physical examination item content, to obtain a user physical examination item information set and a corresponding user physical examination item attribute information set, wherein the user physical examination item attribute information includes a user physical examination item attribute and a user physical examination item attribute value; According to the user physical examination item information set, the user physical examination item attribute information set is matched and stored into the user physical examination report storage system.

3. The method of claim 1, wherein, The image enhancement processing is performed on the user physical examination report image set to obtain an enhanced user physical examination report image set, including: For each user physical examination image area image in the user physical examination image area image set corresponding to the user physical examination report image set, the following image fusion steps are performed: The user physical examination image area image is subjected to pyramid reconstruction processing to obtain a reconstructed image image sequence; According to the scene light map sequence corresponding to the reconstructed image image sequence, an image fusion weight matrix of the reconstructed image image sequence is determined; According to the image fusion weight matrix and the reconstructed image image sequence, an image exposure rate value is generated; According to the image exposure rate value, image mapping processing is performed on the reconstructed image image sequence to obtain a multi-exposure image image sequence; According to the image exposure rate value and the image fusion weight matrix, image fusion is performed on the multi-exposure image image sequence and the reconstructed image image sequence to obtain a fused image image; The image noise reduction processing is performed on the user physical examination text area image set corresponding to the user physical examination report image set to obtain a noise-reduced text area image set; The noise-reduced text area image set is subjected to correction transformation processing to obtain a corrected text area image set; The corrected text area image set and the obtained fused image image set are determined as the enhanced user physical examination report image set.

4. The method of claim 1, wherein, The user physical examination text area image set included in the first physical examination item report image set is subjected to image text recognition to obtain a physical examination report text recognition information set, including: For each user physical examination text area image in the user physical examination text area image set, the following decoding steps are performed: A text box detection model is used to detect the text box of the user physical examination text area image to obtain a text area bounding box group; The text area bounding box group is subjected to character repair processing to obtain a repaired text bounding box group; The repaired text bounding box group is subjected to adaptive feature extraction to obtain a text visual feature vector group; The text visual feature vector group is input into a bidirectional long short-term memory neural network to obtain a text time sequence feature vector group; The text time sequence feature vector group is subjected to feature sequence label prediction to obtain a character probability distribution feature vector group; The character probability distribution feature vector group is subjected to text sequence decoding to obtain physical examination report text recognition information.

5. The method of claim 1, wherein, The report standardization prompt phrase group set is used to extract the content of the physical examination report text calibration information set to obtain a report extraction information set, including: obtain a physical examination report knowledge base and a model fine-tuning sample corresponding to the physical examination report text calibration information set; edit the physical examination report content extraction large language model according to the physical examination report knowledge base to obtain a knowledge-edited content extraction model; decompose an initial weight matrix of the physical examination report content extraction large language model to obtain an amplitude vector and a direction matrix; update the direction matrix according to the model fine-tuning sample and the knowledge-edited content extraction model to obtain an updated direction matrix; update the amplitude vector according to the model fine-tuning sample and the knowledge-edited content extraction model to obtain an updated amplitude matrix; fine-tune the knowledge-edited content extraction model according to the updated direction matrix and the updated amplitude matrix to obtain a fine-tuned content extraction model; perform multi-round question and answer content extraction on the physical examination report text calibration information set by using the report standardization prompt phrase group set and the fine-tuned content extraction model to obtain a report extraction information set.

6. A physical examination report standardization storage device, comprising: an acquisition unit configured to acquire a user physical examination report image set and a physical examination report standardization rule information set, wherein the user physical examination report image set includes a user physical examination image area image and a user physical examination text area image; an image enhancement unit configured to perform image enhancement processing on the user physical examination report image set to obtain an enhanced user physical examination report image set; a physical examination item classification unit configured to perform physical examination item classification processing on the enhanced user physical examination report image set to obtain a first physical examination item report image set and a second physical examination item report image set; The first generation unit is configured to generate image detection text information set according to the user physical examination image area image set included in the first physical examination item report image set, including: for each user physical examination image area image in the user physical examination image area image set included in the first physical examination item report image set, performing the following generation steps: performing visual feature extraction on the user physical examination image area image to obtain an image visual feature vector; inputting the image visual feature vector into a visual residual attention mechanism layer to obtain a visual residual attention feature vector, wherein the visual residual attention mechanism layer includes a visual local attention mechanism and a visual global attention mechanism; determining the Hadamard matrix product of the image visual feature vector and the visual residual attention feature vector to obtain a visual attention feature vector; inputting a preset pathological attribute relationship graph into a spatial attention mechanism layer to obtain a pathological attribute abnormal feature vector set, wherein the preset pathological attribute relationship graph includes a relationship graph of multiple abnormal pathologies and a pathological attribute relationship graph corresponding to each abnormal pathology; according to a preset pathological attribute feature vector, performing feature embedding fusion on the visual attention feature vector and the pathological attribute abnormal feature vector set to obtain a fusion embedding feature vector; performing graph hierarchical attention operation on the fusion embedding feature vector to obtain a visual hierarchical feature vector; inputting the visual hierarchical feature vector into a text generation decoder to generate an image detection text information set; The image text recognition unit is configured to perform image text recognition on the user physical examination text area image set included in the first physical examination item report image set to obtain a physical examination report text recognition information set; The text calibration unit is configured to perform text calibration on the physical examination report text recognition information set according to the image detection text information set to obtain a physical examination report text calibration information set; The second generation unit is configured to generate a report standardization prompt group set according to the physical examination report standardization rule information set and the user physical examination report image set, including: performing physical examination item information extraction on a physical examination report text calibration information set corresponding to the user physical examination report image set to obtain a physical examination item field set; determining at least one physical examination report standardization rule information corresponding to each user physical examination report image in the user physical examination report image set as a target standardization rule information group according to the physical examination item field set to obtain a target standardization rule information group set; for each physical examination report text calibration information in the physical examination report text calibration information set, performing the following prompt generation steps: segmenting the physical examination report text calibration information to obtain a physical examination report segmented paragraph text group; performing dependency syntax analysis on the physical examination report segmented paragraph text group and an initial input prompt to obtain a paragraph dependency syntax tree group and an initial prompt dependency syntax tree, wherein the initial input prompt includes the target standardization rule information group; performing dependency graph conversion on the paragraph dependency syntax tree group and the initial prompt dependency syntax tree to obtain a paragraph dependency graph group and an initial prompt dependency graph; performing graph convolution processing on the paragraph dependency graph group and the initial prompt dependency graph to obtain a paragraph dependency node feature vector group set and an initial prompt node feature vector group; inputting the paragraph dependency node feature vector group set and the initial prompt node feature vector group into a time sequence extraction network, a multi-layer perception and a layer normalization layer in sequence to obtain a paragraph semantic syntax feature vector group set and an initial prompt semantic syntax feature vector group; performing graph attention pooling processing on the paragraph semantic syntax feature vector group set and the initial prompt semantic syntax feature vector group to obtain a report prompt feature vector group; inputting the report prompt feature vector group into a text generation model to obtain a report prompt group; performing thought chain bundle search optimization on the report prompt group to obtain an optimized report prompt group; performing reflection optimization on the optimized report prompt according to a prompt feedback information group corresponding to the optimized report prompt group, to obtain a report standardization prompt group, wherein the prompt feedback information group is feedback information that content extracted by a physical examination report content extraction large language model through prompt extraction is abnormal; The content extraction unit is configured to perform content extraction on the physical examination report text calibration information set according to a physical examination report content extraction large language model and the report standardization prompt group set to obtain a report extraction information set; The storage unit is configured to match and store the report extraction information set to a user physical examination report storage system, and perform storage load optimization on the user physical examination report storage system.

7. An electronic device, comprising: one or more processors; storage having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, so that the one or more processors implement the method of any one of claims 1-5.

8. A computer readable medium having stored thereon a computer program, wherein, The computer program is executed by the processor to implement the method of any one of claims 1-5. The computer program is executed by the processor to implement the method of any one of claims 1-5.

Images