Physical examination report standardized storage method and device, electronic equipment and medium

Abstract

The embodiment of the invention discloses a physical examination report standardized storage method and device, electronic equipment and a medium. A specific embodiment of the method comprises the following steps: carrying out image enhancement processing on an obtained user physical examination report image set, and then carrying out physical examination item classification processing to obtain a first physical examination item report image set and a second physical examination item report image set; generating an image detection text information set; performing image text recognition on the user physical examination text region image set to obtain a physical examination report text recognition information set; performing text calibration on the physical examination report text identification information set to obtain a physical examination report text calibration information set; generating a report standardized prompt word group set; performing content extraction on the physical examination report text calibration information set to obtain a report extraction information set; and matching and storing the report extraction information set to a user physical examination report storage system. According to the embodiment, the extraction accuracy and effect of the physical examination report information can be improved, the extraction time is shortened, and waste of storage resources is reduced.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a method, device, electronic device, and medium for standardized storage of physical examination reports. Background Art

[0002] Physical examination reports contain a large amount of information related to different examination items related to the user's physical health. Different examination items have different data representations and formats, so it is important to standardize the processing and storage of physical examination reports. For standardized storage of physical examination reports, the commonly used method is to use traditional OCR (Optical Character Recognition) algorithms to detect and recognize text on the physical examination report image to obtain the physical examination report text information. Then, using predefined data extraction rules and templates, the content of the physical examination report text information is extracted to obtain the physical examination extracted information. Finally, the physical examination extracted information is stored.

[0003] However, it has been found in practice that when the above method is used to store physical examination reports in a standardized manner, the following technical problems often occur: Since the traditional OCR algorithm is based on artificial features for text recognition, the accuracy of text recognition in physical examination reports with complex and changeable text and images is low and the recognition generalization ability is poor. In addition, the predetermined data extraction rules and templates are manually defined, and there is a certain degree of subjectivity and professionalism, which cannot adapt to dynamic changes, resulting in low accuracy and efficiency in the extraction of physical examination report content, and a large amount of redundant and erroneous data in the extracted content, resulting in a waste of storage resources and extended extraction time.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the Invention

[0005] The content of this disclosure is used to briefly introduce concepts that will be described in detail in the detailed description section below. The content of this disclosure is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0006] Some embodiments of the present disclosure provide a standardized storage method, apparatus, electronic device, and medium for physical examination reports to solve one or more of the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present disclosure provide a method for storing standardized physical examination reports, comprising: obtaining a user physical examination report image set and a physical examination report standardization rule information set, wherein the user physical examination report image includes: a user physical examination image area image and a user physical examination text area image; performing image enhancement processing on the above-mentioned user physical examination report image set to obtain an enhanced user physical examination report image set; performing physical examination item classification processing on the above-mentioned enhanced user physical examination report image set to obtain a first physical examination item report image set and a second physical examination item report image set; generating an image detection text information set based on the user physical examination image area image set included in the above-mentioned first physical examination item report image set; performing image enhancement processing on the above-mentioned first physical examination item report image set to obtain a first physical examination item report image set and a second physical examination item report image set; performing image detection processing on the above-mentioned first physical examination item report image set The image set of user physical examination text area images included in the image set is used for image text recognition to obtain a physical examination report text recognition information set; based on the above-mentioned image detection text information set, the above-mentioned physical examination report text recognition information set is subjected to text calibration to obtain a physical examination report text calibration information set; based on the above-mentioned physical examination report standardization rule information set and the above-mentioned user physical examination report image set, a report standardization prompt word group set is generated; based on the physical examination report content extraction large language model and the above-mentioned report standardization prompt word group set, the above-mentioned physical examination report text calibration information set is subjected to content extraction to obtain a report extraction information set; the above-mentioned report extraction information set is matched and stored in the user physical examination report storage system, and the storage load of the user physical examination report storage system is optimized.

[0008] In a second aspect, some embodiments of the present disclosure provide a physical examination report standardization storage device, comprising: an acquisition unit, configured to acquire a user physical examination report image set and a physical examination report standardization rule information set, wherein the user physical examination report image comprises: a user physical examination image area image and a user physical examination text area image; an image enhancement unit, configured to perform image enhancement processing on the above-mentioned user physical examination report image set to obtain an enhanced user physical examination report image set; a physical examination item classification unit, configured to perform physical examination item classification processing on the above-mentioned enhanced user physical examination report image set to obtain a first physical examination item report image set and a second physical examination item report image set; a first generation unit, configured to generate an image detection text information set based on the user physical examination image area image set included in the above-mentioned first physical examination item report image set; an image text recognition unit, configured to perform image enhancement processing on the above-mentioned first physical examination item report image set Image text recognition is performed on the user physical examination text area image set included in the physical examination project report image set to obtain a physical examination report text recognition information set; a text calibration unit is configured to perform text calibration on the above-mentioned physical examination report text recognition information set based on the above-mentioned image detection text information set to obtain a physical examination report text calibration information set; a second generation unit is configured to generate a report standardization prompt word group set based on the above-mentioned physical examination report standardization rule information set and the above-mentioned user physical examination report image set; a content extraction unit is configured to extract a large language model and the above-mentioned report standardization prompt word group set based on the physical examination report content, perform content extraction on the above-mentioned physical examination report text calibration information set to obtain a report extraction information set; a storage unit is configured to match and store the above-mentioned report extraction information set in a user physical examination report storage system, and perform storage load optimization on the user physical examination report storage system.

[0009] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0010] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.

[0011] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the standardized storage method for physical examination reports of some embodiments of the present disclosure can improve the extraction accuracy and effect of physical examination report information, shorten the extraction time and reduce the waste of storage resources. Specifically, the reasons for the low extraction accuracy and efficiency of the relevant physical examination report content, the presence of a large amount of redundant and erroneous data in the extracted content, and the waste of storage resources are: since the traditional OCR algorithm is based on artificial features for text recognition, the accuracy of text recognition in the complex and changeable text and images interspersed in the physical examination report is low and the recognition generalization ability is poor, and the predetermined data extraction rules and templates are artificially defined, with certain subjectivity and professionalism, and cannot adapt to dynamic changes, resulting in low extraction accuracy and efficiency of the physical examination report content, the presence of a large amount of redundant and erroneous data in the extracted content, resulting in waste of storage resources and extended extraction time. Based on this, the standardized storage method for physical examination reports of some embodiments of the present disclosure can first obtain a user physical examination report image set and a physical examination report standardization rule information set, wherein the user physical examination report image includes: a user physical examination image area image and a user physical examination text area image. Here, the user medical examination report image set and the medical examination report standardization rule information set are used for subsequent content extraction. Secondly, the above-mentioned user medical examination report image set is subjected to image enhancement processing to obtain an enhanced user medical examination report image set. Here, the image quality of the user medical examination report can be improved. Again, the above-mentioned enhanced user medical examination report image set is subjected to medical examination item classification processing to obtain a first medical examination item report image set and a second medical examination item report image set. Here, image classification facilitates subsequent targeted processing of different types of user medical examination reports to improve the accuracy of content extraction. Subsequently, based on the user medical examination image area image set included in the above-mentioned first medical examination item report image set, an image detection text information set is generated. Here, the image detection text information set is the result of identifying the user medical examination image area image recognition, which is used to supplement and correct the text information generated subsequently. Then, the user medical examination text area image set included in the above-mentioned first medical examination item report image set is subjected to image text recognition to obtain a medical examination report text recognition information set. Here, automated text recognition can improve the accuracy and effect of recognition, and facilitate subsequent content extraction. Afterwards, based on the image detection text information set, the above-mentioned physical examination report text recognition information set is subjected to text calibration to obtain the physical examination report text calibration information set. This can improve the accuracy and comprehensiveness of the physical examination report. Then, based on the above-mentioned physical examination report standardization rule information set and the above-mentioned user physical examination report image set, a report standardized prompt group set is generated. This improves the adaptability of the prompts to the physical examination report standardized storage scenario, and improves the accuracy of the output of subsequent content extraction using a large language model and the probability of the output being as expected.Then, based on the physical examination report content extraction large language model and the above-mentioned report standardized prompt group set, the above-mentioned physical examination report text calibration information set is subjected to content extraction to obtain a report extraction information set. Here, by guiding the large language model to extract content through prompts, the accuracy and efficiency of automated content extraction can be improved, and the data quality of the report extraction information can be improved. Finally, the above-mentioned report extraction information set is matched and stored in the user physical examination report storage system, and the storage load of the user physical examination report storage system is optimized. Here, the waste of storage resources of the user physical examination report storage system is reduced, and the load of the system is reduced. It can be concluded that the standardized storage method for physical examination reports can improve the extraction accuracy and effect of physical examination report information, shorten the extraction time and reduce the waste of storage resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0013] Figure 1 is a flowchart of some embodiments of the method for standardized storage of physical examination reports according to the present disclosure;

[0014] Figure 2 This is a schematic diagram of a system page for standardized storage of attribute values corresponding to internal echo attribute information of breast nodules in a breast nodule physical examination item in some embodiments of the method for standardized storage of physical examination reports disclosed herein;

[0015] Figure 3 This is a schematic diagram of a system page for standardized storage of attribute values corresponding to various attribute information in general detection and physical examination items in some embodiments of the physical examination report standardized storage method disclosed herein;

[0016] Figure 4 1 is a schematic diagram of a system page for storing user physical examination reports in a user physical examination report storage system according to some embodiments of the method for standardized storage of physical examination reports disclosed herein;

[0017] Figure 5 is a schematic structural diagram of some embodiments of a standardized storage device for physical examination reports according to the present disclosure;

[0018] Figure 6 is a schematic structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0019] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0020] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.

[0021] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0022] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0023] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0024] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0025] Figure 1 The process 100 of some embodiments of the method for storing a medical examination report in a standardized manner according to the present disclosure is shown. The method for storing a medical examination report in a standardized manner includes the following steps:

[0026] Step 101: Obtain a user's physical examination report image set and a physical examination report standardization rule information set.

[0027] In some embodiments, the execution subject of the above-mentioned physical examination report standardization storage method (such as an electronic device) can obtain the user physical examination report image set and the physical examination report standardization rule information set through a wired connection method or a wireless connection method, wherein the user physical examination report image includes: a user physical examination image area image and a user physical examination text area image. Among them, the user physical examination report image in the above-mentioned user physical examination report image set can be an image showing the health status of each organ of the user's body in the form of an image. The physical examination report standardization rule information in the above-mentioned physical examination report standardization rule information set can be artificially defined, and the rule information of the output format after the content is extracted from the above-mentioned user physical examination report image. The acquisition of the above-mentioned user physical examination report image set can be a physical examination report of a third-party medical institution collected through a system interface. The above-mentioned physical examination report standardization rule information set can be obtained from a local database. The above-mentioned user physical examination image area image can be an area image of the medical image taken when the user is undergoing a physical examination. The above-mentioned user physical examination image text area image can be an image of the area where text appears in the user physical examination report image.

[0028] Step 102: Perform image enhancement processing on the user physical examination report image set to obtain an enhanced user physical examination report image set.

[0029] In some embodiments, the execution entity may perform image enhancement processing on the user physical examination report image set to obtain an enhanced user physical examination report image set.

[0030] In some optional implementations of some embodiments, performing image enhancement processing on the user physical examination report image set to obtain the enhanced user physical examination report image set may include the following steps:

[0031] In the first step, for each user physical examination image region image in the user physical examination image region image set corresponding to the user physical examination report image set, the following image fusion steps are performed:

[0032] Sub-step 1: Perform pyramid reconstruction on the user's physical examination image region to obtain a reconstructed image sequence. The reconstructed image sequence may be an image sequence of multi-sized low-frequency components and high-frequency detail components obtained by up-down reconstruction.

[0033] In practice, the execution entity may first use a Gaussian pyramid to perform Gaussian blur downsampling on the user's physical examination image region to obtain a multi-scale cropped image sequence. Then, using a Laplacian pyramid, the multi-scale cropped image sequence may be upsampled and differentially processed to obtain a Laplacian image sequence as the reconstructed image sequence.

[0034] Sub-step 2: Determine an image fusion weight matrix for the reconstructed image sequence based on a scene illumination map sequence corresponding to the reconstructed image sequence. The scene illumination map may be a texture image based on a spherical coordinate system that simulates the illumination and reflection of the environment corresponding to the user's physical examination report image. The scene illumination map may be an image that uses the maximum brightness value of three channels in the reconstructed image sequence as an initial illumination estimate.

[0035] As an example, the execution subject may use an image energy minimization function to determine the image fusion weight matrix of the reconstructed image sequence based on the scene illumination map corresponding to the reconstructed image sequence. The image fusion weight matrix is obtained by solving the image energy minimization function using a multi-resolution conjugate gradient method. The image energy minimization function may be:

[0036]

[0037] in, Represents the reconstructed image. T represents the scene illumination map. M represents the image fusion weight matrix parameter. represents the difference filter used to reconstruct the image. μ represents the smoothing constant. It represents the square of the quadratic norm of the reconstructed image and the scene illumination map, and represents the similarity between the reconstructed image and the scene illumination map. Represents a linear normal form of the reconstructed image, representing the smoothing process of the reconstructed image.

[0038] Sub-step 3: Generate an image exposure value based on the image fusion weight matrix and the reconstructed image sequence, wherein the image exposure value can represent the exposure degree of the image.

[0039] As an example, the execution subject may first select at least one reconstructed image image from the reconstructed image sequence by using the normalized pixel values of each scene illumination map in the scene illumination map sequence, wherein the normalized pixel value of the corresponding scene illumination map is less than or equal to a preset pixel value threshold. The preset pixel value threshold may be a pre-set value. For example, the preset pixel value threshold may be 0.5. Then, the three-channel brightness geometric mean function of the at least one reconstructed image image is determined by using a bidirectional texture brightness transformation function. The three-channel brightness geometric mean function may be a function including an image exposure parameter. The bidirectional texture brightness transformation function may be

[0040]

[0041] γ=k a .

[0042] Where g(P, k) represents the bidirectional texture brightness transformation function. P represents the normalized pixel value matrix of the reconstructed image. k represents the image exposure parameter. β controls the global gain coefficient, which is used to adjust the overall brightness of the reconstructed image. γ controls the gamma coefficient, which affects the degree of nonlinear stretching / compression of pixel values. A value other than 1 indicates that the bidirectional texture brightness function is nonlinear for the camera. a represents the direction and degree of the exposure's effect on the gamma coefficient. When the exposure is greater than 0, a is -0.3293, and the gamma coefficient is negative. This function performs logarithmic compression of pixel values, which is suitable for enhancing details in underexposed areas. As the exposure increases, the gamma coefficient decreases, reducing the degree of nonlinearity and preventing over-enhancement in overexposed areas. b represents the step size of the gain coefficient. When b is greater than 0, the exposure increases, the exponential increases, and the global gain coefficient increases rapidly, improving overall brightness. When the exposure decreases, the global gain coefficient decreases, preventing dark areas from being overly dark. When the relevant parameters of the user's physical examination image area are not obtained, the value of a is -0.3293 and the value of b is 1.1258.

[0043] Finally, the three-channel brightness geometric mean function is input into the image entropy function corresponding to the reconstructed image sequence, and the image enhancement is maximized to obtain the image exposure value.

[0044] Sub-step 4: Performing image mapping on the reconstructed image sequence based on the image exposure ratio value to obtain a multiple-exposure image sequence. The multiple-exposure images in the multiple-exposure image sequence may include reconstructed images with different exposure levels. As an example, the execution entity may input the image exposure ratio value into a bidirectional texture-luminance transformation function to perform image mapping on the reconstructed image sequence to obtain the multiple-exposure image sequence.

[0045] Sub-step 5: performing image fusion on the multiple exposure image sequence and the reconstructed image sequence according to the image exposure rate value and the image fusion weight matrix to obtain a fused image.

[0046] As an example, the execution entity may perform weighted summation on the image exposure rate value and the image fusion weight matrix to obtain a fused image.

[0047] In the second step, image noise reduction is performed on the user physical examination text region image set corresponding to the user physical examination report image set to obtain a noise-reduced text region image set.

[0048] The third step is to perform correction transformation processing on the above-mentioned denoised text region image set to obtain a corrected text region image set. The above-mentioned correction transformation processing may include: rotation correction and perspective transformation.

[0049] In the fourth step, the corrected text region image set and the obtained fused image set are determined as the enhanced user physical examination report image set.

[0050] Step 103 , classify the enhanced user physical examination report image set by physical examination items to obtain a first physical examination item report image set and a second physical examination item report image set.

[0051] In some embodiments, the execution entity may perform physical examination item classification processing on the enhanced user physical examination report image set to obtain a first physical examination item report image set and a second physical examination item report image set. The first physical examination item report image in the first physical examination item report image set may be an image of a physical examination report whose physical examination item examination result is a long text and multiple examination attributes. The long text may be a text with more than 50 words in a paragraph. For example, the first physical examination item report image may include an image of a physical examination report of a sub-physical examination item in a breast ultrasound examination room. The multiple examination attributes may include but are not limited to at least one of the following: breast nodule boundary condition, breast nodule morphology, nodule aspect ratio, breast nodule internal echo, breast nodule blood flow signal, breast nodule direction, and nodule nature. The second physical examination item report image in the second physical examination item report image set may be an image of a physical examination report whose physical examination result is a short text or a few sentences. For example, the second physical examination item report image may be a physical examination report of a general physical examination item including sub-items such as the user's height, weight, and blood pressure.

[0052] In practice, the execution entity may first segment each enhanced user medical examination report image in the enhanced user medical examination report image set by medical examination item to obtain a set of medical examination item images. Then, the set of medical examination item images may be classified based on the number of words in the text to obtain a first set of medical examination item report images and a second set of medical examination item report images.

[0053] Step 104 : generating an image detection text information set based on the user physical examination image area image set included in the first physical examination item report image set.

[0054] In some embodiments, the execution entity may generate an image detection text information set based on the user physical examination image area image set included in the first physical examination item report image set. The image detection text information in the image detection text information set may be text information describing the user physical examination image area image obtained by performing model intelligent recognition on the user physical examination image area image.

[0055] In some optional implementations of some embodiments, generating the image detection text information set based on the user physical examination image area image set included in the first physical examination item report image set may include the following steps:

[0056] In the first step, for each user physical examination image region image in the user physical examination image region image set included in the first physical examination item report image set, the following generation steps are performed:

[0057] Sub-step 1: Perform visual feature extraction on the user's physical examination image region to obtain an image visual feature vector. The image visual feature vector may represent multiple feature information of the user's physical examination image region, such as color, texture, edges, shape, and orientation. The visual feature extraction may be performed using a ResNet-50 model.

[0058] Sub-step 2: Input the above-mentioned image visual feature vector into the visual residual attention mechanism layer to obtain a visual residual attention feature vector, wherein the above-mentioned visual residual attention mechanism layer includes: a visual local attention mechanism and a visual global attention mechanism. The above-mentioned visual attention feature vector can represent a vector that uses different weights for different parts of the user's physical examination image area. The above-mentioned visual local attention mechanism can be a model consisting of a series of convolutional layers and nonlinear activation functions, and determining the attention weight through a sigmoid function. The above-mentioned visual local attention mechanism can be a deep neural network model including a first convolutional layer, a ReLU (Rectified Linear Unit) activation function, a second convolutional layer, a ReLU activation function, a third convolutional layer, a Softmax activation function, a matrix multiplication of the image visual feature vector and the feature vector output by the Softmax activation function, and a pixel-by-pixel feature addition of the image visual feature vector and the feature vector output by the matrix multiplication. The above-mentioned visual global attention mechanism can be a model that adds a parallel two residual convolutional layers on the basis of the above-mentioned visual local attention mechanism, and performs feature splicing on the two obtained feature vectors.

[0059] Sub-step 3: Determine the Hadamard matrix product of the above-mentioned image visual feature vector and the above-mentioned visual residual attention feature vector to obtain the visual attention feature vector.

[0060] In sub-step 4, the preset pathology attribute relationship graph is input into the spatial attention mechanism layer to obtain a set of pathology attribute abnormality feature vectors. The preset pathology attribute relationship graph includes a relationship graph formed by multiple abnormal pathologies and a pathology attribute relationship graph corresponding to each abnormal pathology. The preset pathology attribute relationship graph can be an existing medical-related knowledge graph. The relationship graph formed by abnormal pathologies can be a knowledge graph of body parts exhibiting abnormalities in the user's physical examination image.

[0061] Sub-step 5: Based on a preset pathology attribute feature vector, feature embedding and fusion are performed on the visual attention feature vector and the pathology attribute abnormality feature vector set to obtain a fused embedded feature vector. The preset pathology attribute feature vector may represent the association between the abnormal image portion and the attribute information included in the user's physical examination image region.

[0062] Sub-step 6, performs a graph-level attention operation on the above-mentioned fused embedded feature vector to obtain a visual-level feature vector. The above-mentioned visual-level feature vector can represent the feature information of the abnormal information, attribute information and context between the previous time step in the fused embedded feature vector. In practice, the above-mentioned execution subject can first input the above-mentioned fused embedded feature vector into GAT (Graph Attention Network) to obtain a fused attention feature vector. Then, the above-mentioned fused attention feature vector and the fused attention feature vector of the previous time step are subjected to a secondary attention operation to obtain a fused context feature vector as the visual-level feature vector.

[0063] Sub-step 7: Input the visual level feature vector into a text generation decoder to generate an image detection text information set. The text generation decoder can be a Transformer model with a multi-level multi-head attention mechanism.

[0064] Step 105 : performing image text recognition on the user physical examination text area image set included in the first physical examination item report image set to obtain a physical examination report text recognition information set.

[0065] In some embodiments, the execution entity may perform image text recognition on the user physical examination text area image set included in the first physical examination item report image set to obtain a physical examination report text recognition information set. The physical examination report text recognition information in the physical examination report text recognition information set may be information displaying the text in the user physical examination text area image in text form.

[0066] In some optional implementations of some embodiments, performing image text recognition on the user physical examination text area image set included in the first physical examination item report image set to obtain the physical examination report text recognition information set may include the following steps:

[0067] In the first step, for each user physical examination text area image in the above user physical examination text area image set, perform the following decoding steps:

[0068] Sub-step 1: Use a text box detection model to perform text box detection on the above-mentioned user physical examination text area image to obtain a text area bounding box group. The text area bounding box in the above-mentioned text area bounding box group can be a rectangular box used to represent the position and range of the text in the user physical examination text area image. The text box detection model can be a deep neural network model that performs text area detection on the input user physical examination text area image. The above-mentioned text box detection model can be a model that adds channel and spatial attention mechanisms to the YOLO v7-tiny (You Only Look Once v7-tiny, the 7th lightweight version of YOLO) model.

[0069] Sub-step 2: Perform text patching on the text region bounding box group to obtain a patched text bounding box group. The text patching may be performed using a dilation and erosion method.

[0070] Sub-step 3: Performing adaptive feature extraction on the patched text bounding box group to obtain a text visual feature vector group. The text visual feature vectors in the text visual feature vector group can represent image detail information and text semantic information of the patched text bounding box. The adaptive feature extraction can include first cropping the patched text bounding box group to fit the feature extraction output of the convolutional network.

[0071] Sub-step 4: Input the above text visual feature vector set into a bidirectional long short-term memory neural network to obtain a text temporal feature vector set. The text temporal feature vectors in the above text temporal feature vector set can represent the temporal dependency information of the text characters.

[0072] Sub-step 5: Perform feature sequence label prediction on the text time series feature vector group to obtain a character probability distribution feature vector group. The character probability distribution feature vectors in the character probability distribution feature vector group can represent the probability of each feature vector being recognized as a character. The feature sequence label prediction can be performed using a fully connected layer and a softmax activation function.

[0073] Sub-step 6: performing text sequence decoding on the character probability distribution feature vector group to obtain the physical examination report text recognition information. The text sequence decoding may be performed using a CTC (Connectionist Temporal Classification) algorithm.

[0074] Step 106 : performing text calibration on the physical examination report text recognition information set according to the image detection text information set to obtain a physical examination report text calibration information set.

[0075] In some embodiments, the execution entity may perform text calibration on the physical examination report text recognition information set based on the image detection text information set to obtain a physical examination report text calibration information set. The physical examination report text calibration information in the physical examination report text calibration information set may be text information obtained by filling in and modifying the physical examination report text recognition information.

[0076] As an example, the execution entity may first perform attribute value matching on each image detection text information in the image detection text information set and the corresponding physical examination report text recognition information in the physical examination report text recognition information set to obtain a text matching result set. Then, in response to determining that the above text matching result indicates that the attribute values are different, re-test result information of physical examination items with different attribute values by different medical personnel is obtained, and the re-test result information is determined as the physical examination report text calibration information. Finally, in response to determining that the above text matching result indicates that no match is found, the image detection text information is determined as the physical examination report text calibration information.

[0077] Step 107: Generate a report standardized prompt word set according to the physical examination report standardized rule information set and the user physical examination report image set.

[0078] In some embodiments, the execution entity may generate a set of standardized report prompts based on the set of standardized medical examination report rule information and the set of user medical examination report images. The standardized report prompts in the set of standardized report prompts may be text information that guides the large language model to output results in a preset output format.

[0079] As an example, the execution entity may input the medical examination report standardization rule information set and the user medical examination report image set into a large language model to obtain a set of standardized report prompts. The large language model may be a GPT model (Generative Pre-trained Transformer).

[0080] In the process of adopting technical solutions to solve the above-mentioned technical problem one, the following technical problem two is often accompanied: Since the large language model contains a large number of parameters, how to make the large language model targeted without basically training or training some parameters, so that the information extraction of the physical examination report is more targeted, and the accuracy and efficiency of the extraction are improved. For the above-mentioned technical problem two, the conventional solution is generally: to guide the large language model to extract the content of the physical examination report through the prompts of the artificially constructed large language model. However, the above-mentioned conventional solution still has the following problems: since the artificially constructed prompts need to be mapped through a large number of templates and the output results of the large language model for continuous attempts, it is time-consuming and the model output accuracy is low, resulting in low accuracy and efficiency of subsequent content extraction. The large language model needs to extract content repeatedly, which increases the system load and wastes a lot of training resources. The inventors took into account the shortcomings of the above-mentioned conventional solutions, and combined with the advantages / technical status of the automatic generation of prompts technology owned by the inventor's company, we decided to adopt the following solution:

[0081] In some optional implementations of some embodiments, generating a set of standardized report prompts based on the set of standardized physical examination report rule information and the set of user physical examination report images may include the following steps:

[0082] The first step is to extract medical examination item information from the medical examination report text calibration information set corresponding to the above-mentioned user medical examination report image set to obtain a medical examination item field set. The medical examination item field in the above-mentioned medical examination item field set can be a field for the name or item number of the medical examination performed by the user. For example, the above-mentioned medical examination item field can be, but is not limited to, at least one of the following: ophthalmology, internal medicine, dentistry, and blood routine.

[0083] The second step is to determine at least one physical examination report standardization rule information corresponding to each user physical examination report image in the user physical examination report image set based on the physical examination item field set, as the target standardization rule information group, and obtain the target standardization rule information group set.

[0084] As an example, the execution entity may first determine the report medical examination item set of the medical examination report text calibration information set corresponding to the user medical examination report image set. Then, using the report medical examination item set, the user medical examination report image set and the medical examination report standardization rule information set are matched for medical examination items to obtain at least one medical examination report standardization rule information set for each user medical examination report image, thereby obtaining a target standardization rule information set.

[0085] In the third step, for each piece of medical examination report text calibration information in the above medical examination report text calibration information set, the following prompt generation steps are performed:

[0086] Sub-step 1: segment the medical examination report text calibration information to obtain a medical examination report segmented paragraph text group. The medical examination report segmented paragraph text in the medical examination report segmented paragraph text group may be paragraph text obtained by segmenting according to sentences.

[0087] Sub-step 2, performing dependency syntactic analysis on the above-mentioned physical examination report segmented paragraph text group and the initial input prompt to obtain a paragraph dependency syntax tree group and an initial prompt dependency syntax tree, wherein the above-mentioned initial input prompt includes a target standardization rule information group. The above-mentioned initial input prompt can be a relatively simple prompt constructed by the user containing preset extraction or output constraints. The paragraph dependency syntax tree in the above-mentioned paragraph dependency syntax tree group can be a tree structure constructed according to the grammatical relationship of each segmented word in the physical examination report segmented paragraph text. The above-mentioned initial prompt dependency syntax tree can be a tree structure composed of the grammatical relationship of each prompt word in the initial input prompt. The above-mentioned dependency syntax analysis can be a syntax analysis performed using the HanLP (Han Language Processing) natural language tool.

[0088] Sub-step 3, performing dependency graph conversion on the above paragraph dependency syntax tree group and the above initial prompt dependency syntax tree to obtain a paragraph dependency graph group and an initial prompt dependency graph. Among them, the paragraph dependency graph in the above paragraph dependency graph group can be a graph that represents the relationship between each segmented word. The above initial prompt dependency graph can be a graph that represents the association relationship between each prompt analysis. The above dependency graph conversion can be a graph conversion using the DGL (Deep Graph Library) graph neural network library. It should be noted that in order to avoid the situation where the word segmentation of HanLP is different from the word segmentation output by the DGL model, when constructing the dependency tree, it will be judged whether there is a conflict with the model, and a table of conflicts between the two will be maintained. When there is a conflict, the minimum granularity word segmentation method will be used to ensure that the encoding of the dependency tree is correct.

[0089] Sub-step 4, performing graph convolution processing on the above paragraph dependency graph group and the above initial prompt dependency graph to obtain a paragraph dependency node feature vector group set and an initial prompt node feature vector group. Among them, the paragraph dependency node feature vectors in the above paragraph dependency node feature vector group set can characterize the information of multiple grammatical associations (such as nominal subjects, direct objects, possessive modifiers, prepositional modifiers, objects of prepositions, etc.) between the segmented word and its neighboring segmented word node group included in the paragraph dependency graph. The initial prompt node feature vectors in the above initial prompt node feature vector group can characterize the information of multiple grammatical relationships between the prompt word and the neighboring prompt word node group included in the above initial prompt dependency graph. The above graph convolution processing can be inputting the above paragraph dependency graph group and the above initial prompt dependency graph into an RGCN (Relational Graph Convolutional Network) model.

[0090] Sub-step 5, inputting the above paragraph dependency node feature vector set and the above initial prompt node feature vector set into the time series extraction network, the multi-layer perceptron and the layer normalization layer in sequence, to obtain the paragraph semantic grammatical feature vector set and the initial prompt semantic grammatical feature vector set. Among them, the paragraph semantic grammatical feature vector in the above paragraph semantic grammatical feature vector set can represent the semantic and grammatical information of each segmentation node. The initial prompt semantic grammatical feature vector in the above initial prompt semantic grammatical feature vector group can represent the semantic and grammatical information of the prompt word node. The above time series extraction network can be a recurrent neural network that extracts the time series information of the paragraph dependency node feature vector set and the initial prompt node feature vector group respectively. For example, the above time series extraction network can be a bidirectional long short-term memory neural network. The above multi-layer perceptron can be an MLP (Multi Layer Perceptron) comprising two linear layers and a ReLU activation function. The above normalization layer (Layer Normalization) can make the loss value drop more smoothly and prevent the neural network layer from the gradient disappearance and explosion problems.

[0091] Sub-step 6, performing graph attention pooling processing on the above paragraph semantic grammatical feature vector set and the above initial prompt semantic grammatical feature vector set to obtain a report prompt feature vector set. The report prompt feature vector in the above report prompt feature vector set can represent the information of the attention weight of the feature information of the fusion of semantics and grammar. In practice, the above execution subject can first determine the attention weight value set of the above paragraph semantic grammatical feature vector set and the above initial prompt semantic grammatical feature vector set through a multi-layer perception set. Then, after adding a second-order adjacency matrix to the above attention weight value set, the average is taken to obtain an attention weight mean value set. Afterwards, the semantic grammatical feature vectors that are ranked in the front of the above paragraph semantic grammatical feature vector set and the above initial prompt semantic grammatical feature vector set according to the attention weight mean value set are selected from the above paragraph semantic grammatical feature vector set and the above initial prompt semantic grammatical feature vector set according to the attention weight mean value set and sorted from large to small are pooled and masked to obtain a report prompt feature vector set. The above preset number can be the number of word segments of the prompt finally generated.

[0092] Sub-step 7: Input the above-mentioned report prompt feature vector group into a text generation model to obtain a report prompt group. The report prompts in the above-mentioned report prompt group can be prompts that are generated by integrating the semantics and syntax of the physical examination report standardization rule information set and the user physical examination report image set. The text generation model can be a deep neural network model that performs text mapping on the input report prompt feature vector. For example, the text generation model can be a BERT (Bidirectional Encoder Representations from Transformers) model.

[0093] Sub-step 8 is to perform thought chain bundle search optimization on the above-mentioned report prompt group to obtain an optimized report prompt group. Among them, the optimized report prompts in the above-mentioned optimized report prompt group can be prompts for performing intermediate reasoning on the subsequent content extraction process to simplify the reasoning task in each step. The above-mentioned thought chain bundle search optimization can be achieved by adopting a thought tree approach, drawing on the idea of beam search, expanding the search space of prompts, so that the large language model for extracting the content of the physical examination report generates multiple potential prompts in each round of iteration, which are branches of the thought tree, and eliminating prompts with poor results, and selecting the top 5 prompts with the best results to enter the next round of optimization. The use of multiple rounds of iteration, that is, the depth of the thought tree, deepens the level of model exploration and gradually and automatically optimizes the prompts.

[0094] Sub-step 9: Reflectively optimize the optimized report prompts based on the corresponding prompt feedback information set of the optimized report prompts to obtain a standardized report prompt set. The prompt feedback information in the prompt feedback information set may be feedback indicating anomalies in the content extracted by the large language model for extracting physical examination report content through the prompts.

[0095] The above technical solution, combined with "step 108" and "step 109", and their related contents as an inventive point of an embodiment of the present disclosure, solves the second technical problem mentioned in the background technology: "Since the manually constructed prompts need to be mapped through a large number of templates and the output results of the large language model for continuous attempts, it is time-consuming and the model output accuracy is low, resulting in low subsequent content extraction accuracy and efficiency. The large language model needs to repeatedly extract content, increasing the system load and wasting a large amount of training resources." The factors that lead to low subsequent content extraction accuracy and efficiency, the need for the large language model to repeatedly extract content, increasing the system load and wasting a large amount of training resources are often as follows: Since the manually constructed prompts need to be mapped through a large number of templates and the output results of the large language model for continuous attempts, it is time-consuming and the model output accuracy is low. If the above factors are solved, the effect of improving the subsequent content extraction accuracy and efficiency, reducing the number of content extraction times of the large language model, reducing the system load and reducing the waste of training resources can be achieved. To achieve this effect, the present disclosure first determines at least one medical examination report standardization rule information corresponding to each user's medical examination report image so as to be more suitable for the medical examination content extraction task. Secondly, a dependency syntax tree and dependency graph are constructed to extract grammatical information about the paragraph and initial prompt. Then, the dependency graph is subjected to graph convolution, temporal extraction, multi-layer perceptron, layer normalization, attention mechanism, and text generation. This incorporates semantic information from multiple grammatical relationships between neighboring word segments, as well as the inherent sequentiality of natural language and the contextual relationships between words in natural text, improving the performance of report prompts. Graph attention pooling identifies important nodes by weighting them, reducing the amount of data in node feature vectors and improving generation efficiency. Finally, thought chain search optimization and reflection optimization are performed on the report prompt group, further improving the performance of the prompts and the applicability of content extraction. This improves the accuracy of the prompts, reduces the number of training cycles for the large language model used to extract content from physical examination reports, reduces system load and resource consumption, and improves the efficiency of prompt generation.

[0096] Step 108 : extracting a large language model and a report standardized prompt phrase set based on the content of the physical examination report, performing content extraction on the physical examination report text calibration information set, and obtaining a report extraction information set.

[0097] In some embodiments, the execution subject may extract content from the physical examination report text calibration information set based on the physical examination report content extraction large language model and the report standardized prompt group set to obtain a report extraction information set. The physical examination report content extraction large language model may be a model that inputs content extraction prompts to guide the large language model to extract physical examination item attribute information and attribute values from the physical examination report text calibration information set. For example, the physical examination report content extraction large language model may be a Chinese LLaMA2-7B (Chinese Large Language ModelMeta AI 2-7B) model. Chinese represents a large language model for the Chinese language. 7B represents a model with 7 billion parameters. The report extraction information in the report extraction information set may be information on the physical examination item attribute information and corresponding attribute values in the extracted physical examination report text calibration information. For example, the report extraction information may be information on multiple attribute values corresponding to the internal echo attribute information of breast nodules, representing no echo, low echo, and high echo information. Figure 2 and Figure 3 As shown, Figure 2 The system page shows the storage of attribute values corresponding to the internal echo attribute information of breast nodules extracted for breast physical examination items after being standardized and extracted according to rules. Figure 3 This page shows the system page where each attribute value in general examination items is extracted and stored according to the standardized rule information of the physical examination report.

[0098] As an example, the execution entity may input the report standardized prompt phrase set and the physical examination report text calibration information set into a physical examination report content extraction large language model to obtain a report extraction information set.

[0099] In some optional implementations of some embodiments, extracting the large language model based on the physical examination report content and the report standardized prompt phrase set, performing content extraction on the physical examination report text calibration information set to obtain the report extraction information set may include the following steps:

[0100] The first step is to obtain a medical examination report knowledge base and model fine-tuning samples corresponding to the medical examination report text calibration information set. The medical examination report knowledge base can be an existing knowledge graph related to medical examination reports. The model fine-tuning samples can be medical examination report images used to fine-tune the parameters of the large language model for extracting medical examination report content. The model fine-tuning samples are samples of annotated data containing correct content extraction.

[0101] In the second step, based on the above-mentioned physical examination report knowledge base, the above-mentioned physical examination report content extraction large language model is knowledge edited to obtain the content extraction model after knowledge editing.

[0102] As an example, the above-mentioned execution entity can use the MEMIT (Mass-Editing Memory in a Transformer) knowledge editing algorithm to perform knowledge editing on the large language model for extracting the content of the physical examination report based on the physical examination report knowledge base to obtain a content extraction model after knowledge editing.

[0103] The third step is to decompose the initial weight matrix of the large language model extracted from the medical examination report content to obtain a magnitude vector and a direction matrix. The magnitude direction is a scalar parameter that controls the weight update step size. The direction matrix represents the direction of the weight update under low-rank decomposition.

[0104] The fourth step is to perform low-rank matrix updates on the above direction matrices based on the fine-tuning samples of the above model and the content extraction model after knowledge editing to obtain the updated direction matrix.

[0105] As an example, the execution entity may first use the LoRA (Low-Rank Adaptation) algorithm to determine the first low-rank matrix and the second low-rank matrix of the directional matrix. Then, the model fine-tuning sample is used to determine the mean square error loss function value of the content extraction model after knowledge editing. Finally, the mean square error loss function value and the Adam optimizer are used to iterate the first low-rank matrix and the second low-rank matrix until the mean square error loss function value is less than or equal to the preset loss threshold, thereby obtaining the updated directional matrix.

[0106] The fifth step is to fine-tune the sample according to the above model and the content extraction model after the above knowledge editing, and perform backpropagation update on the above amplitude vector to obtain the updated amplitude matrix.

[0107] As an example, the execution entity may use the mean square error loss function value and the Adam optimizer to iterate the amplitude vector until the mean square error loss function value is less than or equal to a preset loss threshold, thereby obtaining an updated amplitude matrix.

[0108] In the sixth step, the content extraction model after knowledge editing is fine-tuned according to the updated direction matrix and the updated amplitude matrix to obtain a fine-tuned content extraction model.

[0109] As an example, the execution entity may first determine the matrix multiplication of the updated direction matrix and the updated amplitude matrix as the model parameter matrix. Then, the model parameter matrix is used to replace the original parameter matrix of the content extraction model after knowledge editing to obtain the fine-tuned content extraction model.

[0110] In the seventh step, using the standardized report prompts and the fine-tuned content extraction model, multiple rounds of question-and-answer content extraction are performed on the calibrated physical examination report text information set to obtain a report extraction information set. This multiple rounds of question-and-answer content extraction can be performed by using the standardized report prompts to gradually guide the fine-tuned content extraction model to perform content extraction.

[0111] Step 109: Match and store the report extraction information set in the user physical examination report storage system, and optimize the storage load of the user physical examination report storage system.

[0112] In some embodiments, the execution subject may match and store the report extraction information set to the user physical examination report storage system, and optimize the storage load of the user physical examination report storage system. The user physical examination report storage system may be a server cluster for storing the extracted report extraction information set. The storage load optimization may be a load optimization performed using a heuristic algorithm. Figure 4 As shown, a system page for storing the attribute information of the physical examination items corresponding to the user's physical examination report image into the user's physical examination report storage system is displayed. Figure 4 The left part is the user's physical examination report, and the right part is the area after storage displayed in the user's physical examination report storage system.

[0113] In the process of adopting technical solutions to solve the above-mentioned technical problem one, the following technical problem three is often accompanied: how to store a large amount of structured data after standardized extraction of physical examination report content to balance the load and storage efficiency of the user physical examination report storage system. In response to the above-mentioned technical problem three, the conventional solution is generally: adopt hash sharding and hash indexing to optimize the storage load of the user physical examination report storage system. However, the above-mentioned conventional solution still has the following problems: since hash sharding uses fixed rules to divide data, it cannot adapt to the dynamic changes of the report extraction information set, and there is a data concentration distributed in the same shard, resulting in excessive node load, and the hash index is a full table scan query that needs to be queried across multiple shards, and the system server communication consumption is large, resulting in a high load on the user physical examination report storage system and low storage efficiency. The inventors took into account the shortcomings of the above-mentioned conventional solutions and combined with the advantages / technical status of the storage load optimization technology owned by the inventor's company, we decided to adopt the following solution:

[0114] In some optional implementations of some embodiments, the above-mentioned matching and storing of the report extraction information set in the user physical examination report storage system and optimizing the storage load of the user physical examination report storage system may include the following steps:

[0115] The first step is to perform data conversion on the above-mentioned report extraction information set to obtain a report conversion extraction information set. In practice, the above-mentioned execution entity can first convert the punctuation marks included in the above-mentioned report extraction information set into Chinese symbols. Then, using the line breaks in the Chinese symbols, the above-mentioned report extraction information set is segmented and stored in an ordered set (list set). Finally, according to the length of the subscript in the above-mentioned ordered set, the above-mentioned ordered set is assembled into a triple set containing a number, attribute information, and attribute value as a report conversion extraction information set. Among them, the subscript length can be 3 or 4.

[0116] The second step is to use the medical examination item number set included in the report conversion extraction information set to determine a storage location information set of the report conversion extraction information set in the user medical examination report storage system. The storage location information in the storage location information set may represent a logical storage address of the report conversion extraction information set in the user medical examination report storage system.

[0117] The third step is to determine the data access probability of each storage data corresponding to each storage location information set in the above storage location information set through the historical data query statement set, and obtain the data access probability set as the initial access probability set of the above report conversion extraction information set. The data access probability in the above data access probability set can represent the access frequency of the storage data. The above data access probability can be the ratio of the number of query statements containing each storage data in the historical access query statement to the number of historical access query statements. The historical data query statements in the above historical data query statement set can be query statements that access the storage data in the user physical examination report storage system before the current time.

[0118] Step 4: Based on the storage location information set and the initial access probability set, a data association probability matrix is determined for the storage data sets associated with the report conversion and extraction information set. Each element in the data association probability matrix may represent the degree of association between the storage data and the report conversion and extraction information. The associations may include primary and foreign key relationships and co-occurrence relationships in query statements.

[0119] As an example, the execution subject may first determine the probability values of each storage value corresponding to the storage location information set as a preset association value, since the storage location information set is determined by the physical examination items to have a problem of primary key consistency. The preset association value may be a pre-set probability value. For example, the preset association value may be 1. Secondly, through the storage location information set, the association probability value of at least one storage data having an appearance relationship corresponding to each storage data of the report conversion extraction information is determined to be the preset association value. Finally, the preset association values corresponding to the primary and foreign key relationships corresponding to the storage location information set and the initial access probabilities corresponding to the initial access probability set are weighted and summed to obtain a data association probability matrix.

[0120] Step 5: Based on the data association probability matrix, determine the fuzzy storage data cluster to which each report conversion extraction information in the report conversion extraction information set belongs, thereby obtaining a fuzzy storage data cluster set. Each fuzzy storage data cluster in the fuzzy storage data cluster set includes a report cluster center, a report membership matrix, and a report storage node address. The fuzzy storage data clusters in the fuzzy storage data cluster set may be data sets composed of report conversion extraction information obtained by fuzzy C-means clustering using the data association probability matrix. The report storage node address may be the address of a system server.

[0121] As an example, the execution subject may utilize the fuzzy C-means clustering algorithm to determine the fuzzy storage data cluster to which each report conversion extraction information in the report conversion extraction information set belongs based on the data association probability matrix, thereby obtaining a fuzzy storage data cluster set. The data association probability matrix serves as an initialized membership matrix. The objective function of the fuzzy C-means clustering algorithm may be to add access cost terms and penalty terms to the objective function of the original fuzzy C-means algorithm. The access cost terms may include access frequency cost, correlation cost, and cross-node communication delay cost. The penalty term may be a penalty term when the report conversion extraction information and the storage data are highly correlated but have large differences in membership.

[0122] Step 6: Based on the above-mentioned fuzzy storage data class cluster set, determine the report predictive index tree, report balanced multi-way query index tree and report leaf index tree of the above-mentioned report conversion extraction information set. The above-mentioned report predictive index tree can be an index tree that predicts the shard range of the above-mentioned report conversion extraction information set. The above-mentioned report balanced multi-way query index tree can be an index tree used to determine the precise boundary of the range query to avoid errors in the report predictive index tree and handle complex conditions. The above-mentioned report leaf index tree can be an index tree used to record the report cluster center, the report membership matrix and the report storage node address.

[0123] As an example, the execution entity may first input the report conversion extraction information set from the fuzzy storage data cluster set into a data sharding prediction model to obtain a report predictive index tree. The data sharding prediction model may be a multi-layer perceptron that extracts the association between historical query statements and the data blocks corresponding to the historical query statements to predict the data blocks of the report conversion extraction information set. Then, a B+ tree construction method is used to construct a report balanced multi-way query index tree for the fuzzy storage data cluster set. Finally, the report cluster center, report membership matrix, and report storage node are stored in the leaf node, which is determined as the report leaf index tree.

[0124] In step 7, based on the report predictive index tree, the report balanced multi-way query index tree, and the report leaf index tree, the database index tree corresponding to the user physical examination report storage system is combined to obtain a report composite index tree. The report composite index tree may be a tree structure used to locate the storage location of the data.

[0125] As an example, the above-mentioned execution entity can use the reporting predictive index tree as the root node of the reporting hybrid index tree, the above-mentioned reporting balanced multi-way query index tree as the middle layer of the reporting hybrid index tree, and the above-mentioned reporting leaf index tree as the leaf layer of the reporting hybrid index tree.

[0126] Step 8: Generate a server storage node network diagram of the user physical examination report storage system, wherein the edge weights included in the server storage node network diagram represent inter-node communication delays. The server storage node network diagram may represent a diagram of the communication connection relationship between the various servers of the user physical examination report storage system.

[0127] The ninth step is to perform load balancing on the user physical examination report storage system based on the report mixed index tree and the server storage node network diagram to determine the server storage node set where the report conversion and extraction information set is located, and complete the data storage of the report conversion and extraction information set.

[0128] As an example, the above-mentioned execution entity can use reinforcement learning strategies to map the various shards included in the above-mentioned report hybrid index tree to the server storage node network diagram to perform load balancing on the above-mentioned user physical examination report storage system, determine the server storage node set where the above-mentioned report conversion extraction information set is located, and complete the data storage of the report conversion extraction information set.

[0129] The above technical solution and its related contents, as an inventive point of an embodiment of the present disclosure, solve the third technical problem mentioned in the background technology: "Since hash sharding uses fixed rules to divide data, it cannot adapt to the dynamic changes of the report extraction information set, and there is a data set that is concentrated in the same shard, resulting in excessive node load, and the hash index is a full table scan query that needs to be queried across multiple shards, the system server communication consumption is large, resulting in a high load on the user physical examination report storage system and low storage efficiency." The factors that lead to a high load and low storage efficiency of the user physical examination report storage system are often as follows: Since hash sharding uses fixed rules to divide data, it cannot adapt to the dynamic changes of the report extraction information set, and there is a data set that is concentrated in the same shard, resulting in excessive node load, and the hash index is a full table scan query that needs to be queried across multiple shards, the system server communication consumption is large. If the above factors are solved, the complexity of the user physical examination report storage system can be reduced and the storage efficiency can be improved. In order to achieve this effect, the present disclosure first determines the fuzzy storage data cluster set by determining the data association probability matrix, and can dynamically adjust the data sharding of the report conversion extraction information set to avoid the load imbalance problem caused by static rules. Then, after constructing the three index trees, they combine them. The predictive index tree dynamically optimizes query paths, reducing full disk scans and time. The balanced multi-path query index tree provides stable, reliable, and accurate queries to compensate for the errors and complex queries of the predictive index tree. The leaf index tree allows for precise physical storage location, ensuring the accuracy and efficiency of data physical location. Finally, load balancing is performed using the generated server storage node network diagram and the hybrid index tree, improving the storage efficiency of the user's physical examination report storage system and reducing system load.

[0130] Optionally, after step 109, the execution entity may further perform the following steps:

[0131] The first step is to filter out at least one physical examination report standardization rule information that matches the second physical examination item report image set from the physical examination report standardization rule information set.

[0132] The second step is to perform image text recognition on the second physical examination item report image set to obtain a physical examination report text information set. The physical examination report text information in the physical examination report text information set may be text information obtained by performing character recognition on the text in the image.

[0133] The third step is to match and extract the physical examination item content of the at least one physical examination report standardization rule information and the physical examination report text information set to obtain a user physical examination item information set and a corresponding user physical examination item attribute information set, wherein the user physical examination item attribute information includes: user physical examination item attributes and user physical examination item attribute values. The user physical examination item attributes can be the name of a physical examination item or sub-item. For example, the user physical examination item attributes can be the systolic blood pressure of a general examination. The user physical examination item attribute values can be the values corresponding to the names of the physical examination items or sub-items. For example, the user physical examination item attribute values can be a systolic blood pressure of 112.

[0134] The fourth step is to match the user physical examination item attribute information set according to the user physical examination item information set and store it in the user physical examination report storage system.

[0135] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a standardized storage device for physical examination reports. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the standardized storage device for physical examination reports can be specifically applied to various electronic devices.

[0136] like Figure 5As shown, a medical examination report standardization storage device 500 includes: an acquisition unit 501, an image enhancement unit 502, a medical examination item classification unit 503, a first generation unit 504, an image text recognition unit 505, a text calibration unit 506, a second generation unit 507, a content extraction unit 508, and a storage unit 509. The acquisition unit 501 is configured to acquire a user medical examination report image set and a medical examination report standardization rule information set, wherein the user medical examination report image includes a user medical examination image area image and a user medical examination text area image. The image enhancement unit 502 is configured to perform image enhancement processing on the user medical examination report image set to obtain an enhanced user medical examination report image set. The medical examination item classification unit 503 is configured to perform medical examination item classification processing on the enhanced user medical examination report image set to obtain a first medical examination item report image set and a second medical examination item report image set. The first generation unit 504 is configured to generate an image detection text information set based on the user medical examination image area image set included in the first medical examination item report image set. The image text recognition unit 505 is configured to perform image text recognition on the user physical examination text area image set included in the above-mentioned first physical examination item report image set to obtain a physical examination report text recognition information set. The text calibration unit 506 is configured to perform text calibration on the above-mentioned physical examination report text recognition information set based on the above-mentioned image detection text information set to obtain a physical examination report text calibration information set. The second generation unit 507 is configured to generate a report standardization prompt group set based on the above-mentioned physical examination report standardization rule information set and the above-mentioned user physical examination report image set. The content extraction unit 508 is configured to extract the large language model based on the physical examination report content and the above-mentioned report standardization prompt group set, perform content extraction on the above-mentioned physical examination report text calibration information set to obtain a report extraction information set. The storage unit 509 is configured to match and store the above-mentioned report extraction information set in the user physical examination report storage system, and optimize the storage load of the user physical examination report storage system.

[0137] It is understood that the units recorded in the physical examination report standard storage device 500 and the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the medical examination report standardization storage device 500 and the units contained therein, and will not be described in detail here.

[0138] Reference below Figure 6 , which shows a structural schematic diagram of an electronic device (eg, an electronic device) 600 suitable for implementing some embodiments of the present disclosure. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0139] like Figure 6 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the electronic device 600 are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0140] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead. Figure 6 Each block shown in the figure may represent one device, or may represent multiple devices as needed.

[0141] In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network via the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of some embodiments of the present disclosure are performed.

[0142] It should be noted that in some embodiments of the present disclosure, the computer-readable medium mentioned above may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0143] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0144] The computer-readable medium may be included in the electronic device, or may exist independently and not be incorporated into the electronic device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device implements steps 101 to 109.

[0145] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0146] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0147] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, they may be described as: a processor including an acquisition unit, an image enhancement unit, a physical examination item classification unit, a first generation unit, an image text recognition unit, a text calibration unit, a second generation unit, a content extraction unit, and a storage unit. The names of these units do not, in some cases, constitute a limitation on the units themselves. For example, the acquisition unit may also be described as a "unit for acquiring a user physical examination report image set and a physical examination report standardization rule information set."

[0148] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0149] The above description is only an illustration of some preferred embodiments of the present disclosure and the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A standardized storage method for physical examination reports, comprising: Acquire a user physical examination report image set and a physical examination report standardization rule information set, wherein the user physical examination report image includes: a user physical examination image area image and a user physical examination text area image; performing image enhancement processing on the user physical examination report image set to obtain an enhanced user physical examination report image set; Performing physical examination item classification processing on the enhanced user physical examination report image set to obtain a first physical examination item report image set and a second physical examination item report image set; generating an image detection text information set based on the user physical examination image area image set included in the first physical examination item report image set; Performing image text recognition on the user physical examination text area image set included in the first physical examination item report image set to obtain a physical examination report text recognition information set; Performing text calibration on the physical examination report text recognition information set according to the image detection text information set to obtain a physical examination report text calibration information set; Generate a report standardized prompt word set according to the physical examination report standardized rule information set and the user physical examination report image set; Extracting a large language model based on the content of the physical examination report and the report standardized prompt phrase set, performing content extraction on the physical examination report text calibration information set to obtain a report extraction information set; The report extraction information set is matched and stored in a user physical examination report storage system, and the storage load of the user physical examination report storage system is optimized.

2. The method according to claim 1, wherein The method further comprises: Filtering out at least one piece of medical examination report standardization rule information that matches the second medical examination item report image set from the medical examination report standardization rule information set; Performing image text recognition on the second physical examination item report image set to obtain a physical examination report text information set; Performing physical examination item content matching and extraction on the at least one physical examination report standardization rule information and the physical examination report text information set to obtain a user physical examination item information set and a corresponding user physical examination item attribute information set, wherein the user physical examination item attribute information includes: user physical examination item attributes and user physical examination item attribute values; According to the user physical examination item information set, the user physical examination item attribute information set is matched and stored in the user physical examination report storage system.

3. The method according to claim 1, wherein The performing image enhancement processing on the user physical examination report image set to obtain an enhanced user physical examination report image set includes: For each user physical examination image region image in the user physical examination image region image set corresponding to the user physical examination report image set, the following image fusion steps are performed: Performing pyramid reconstruction processing on the user's physical examination image area image to obtain a reconstructed image sequence; Determining an image fusion weight matrix for the reconstructed image sequence according to a scene illumination map sequence corresponding to the reconstructed image sequence; generating an image exposure rate value according to the image fusion weight matrix and the reconstructed image sequence; performing image mapping processing on the reconstructed image sequence according to the image exposure rate value to obtain a multiple exposure image sequence; performing image fusion on the multiple exposure image sequence and the reconstructed image sequence according to the image exposure rate value and the image fusion weight matrix to obtain a fused image; Performing image noise reduction processing on the user physical examination text region image set corresponding to the user physical examination report image set to obtain a noise-reduced text region image set; Performing correction transformation processing on the denoised text region image set to obtain a corrected text region image set; The corrected text region image set and the obtained fused image set are determined as the enhanced user physical examination report image set.

4. The method according to claim 1, wherein The performing image text recognition on the user physical examination text area image set included in the first physical examination item report image set to obtain a physical examination report text recognition information set includes: For each user physical examination text area image in the user physical examination text area image set, the following decoding steps are performed: Using a text box detection model, perform text box detection on the user physical examination text area image to obtain a text area boundary box group; Performing text patching on the text region bounding box group to obtain a patched text bounding box group; performing adaptive feature extraction on the patched text bounding box group to obtain a text visual feature vector group; Inputting the text visual feature vector set into a bidirectional long short-term memory neural network to obtain a text temporal feature vector set; Performing feature sequence label prediction on the text time series feature vector group to obtain a character probability distribution feature vector group; The character probability distribution feature vector group is subjected to text sequence decoding to obtain the physical examination report text recognition information.

5. The method according to claim 1, wherein The large language model and the report standardized prompt phrase set are extracted based on the content of the physical examination report, and content extraction is performed on the physical examination report text calibration information set to obtain a report extraction information set, including: Obtaining a physical examination report knowledge base and model fine-tuning samples corresponding to the physical examination report text calibration information set; Performing knowledge editing on the physical examination report content extraction large language model according to the physical examination report knowledge base to obtain a knowledge-edited content extraction model; Decomposing the initial weight matrix of the large language model extracted from the physical examination report content to obtain an amplitude vector and a direction matrix; According to the model fine-tuning sample and the knowledge edited content extraction model, the direction matrix is respectively updated with a low-rank matrix to obtain an updated direction matrix; Based on the model fine-tuning sample and the knowledge edited content extraction model, back-propagation update is performed on the amplitude vector to obtain an updated amplitude matrix; Fine-tuning the content extraction model after knowledge editing according to the updated direction matrix and the updated amplitude matrix to obtain a fine-tuned content extraction model; The standardized report prompt phrase set and the fine-tuned content extraction model are used to perform multiple rounds of question-and-answer content extraction on the physical examination report text calibration information set to obtain a report extraction information set.

6. The method according to claim 1, wherein The generating of the image detection text information set based on the user physical examination image area image set included in the first physical examination item report image set includes: For each user physical examination image area image in the user physical examination image area image set included in the first physical examination item report image set, the following generating steps are performed: Extracting visual features from the user's physical examination image area to obtain an image visual feature vector; Inputting the image visual feature vector into a visual residual attention mechanism layer to obtain a visual residual attention feature vector, wherein the visual residual attention mechanism layer includes: a visual local attention mechanism and a visual global attention mechanism; Determine the Hadamard matrix product of the image visual feature vector and the visual residual attention feature vector to obtain a visual attention feature vector; Inputting a preset pathology attribute relationship graph into the spatial attention mechanism layer to obtain a pathology attribute abnormal feature vector set, wherein the preset pathology attribute relationship graph includes: a relationship graph formed by multiple abnormal pathologies and a pathology attribute relationship graph corresponding to each abnormal pathology; According to a preset pathological attribute feature vector, the visual attention feature vector and the pathological attribute abnormality feature vector set are subjected to feature embedding fusion to obtain a fused embedded feature vector; Performing a graph-level attention operation on the fused embedded feature vector to obtain a visual-level feature vector; The visual level feature vector is input into a text generation decoder to generate an image detection text information set.

7. A standardized storage device for physical examination reports, comprising: An acquiring unit is configured to acquire a user physical examination report image set and a physical examination report standardization rule information set, wherein the user physical examination report image includes: a user physical examination image area image and a user physical examination text area image; an image enhancement unit configured to perform image enhancement processing on the user physical examination report image set to obtain an enhanced user physical examination report image set; a physical examination item classification unit configured to perform physical examination item classification processing on the enhanced user physical examination report image set to obtain a first physical examination item report image set and a second physical examination item report image set; A first generating unit is configured to generate an image detection text information set according to the user physical examination image area image set included in the first physical examination item report image set; an image text recognition unit configured to perform image text recognition on the user physical examination text area image set included in the first physical examination item report image set to obtain a physical examination report text recognition information set; a text calibration unit configured to perform text calibration on the physical examination report text recognition information set according to the image detection text information set to obtain a physical examination report text calibration information set; A second generating unit is configured to generate a report standardized prompt word set according to the physical examination report standardized rule information set and the user physical examination report image set; a content extraction unit configured to extract a large language model and the report standardized prompt phrase set according to the content of the physical examination report, perform content extraction on the physical examination report text calibration information set, and obtain a report extraction information set; The storage unit is configured to match and store the report extraction information set in the user physical examination report storage system, and optimize the storage load of the user physical examination report storage system.

8. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images