Prescription image recognition method, system, device and storage medium

Abstract

The present invention discloses a prescription picture recognition method, system, device and storage medium, the method comprising obtaining a prescription picture corresponding to a prescription to be recognized, performing text detection on the prescription picture to obtain a text area image; performing feature extraction, feature denoising and feature fusion on the text area image to obtain a target feature map corresponding to the prescription to be recognized; performing image distillation processing on the target feature map to obtain an initial text recognition result; performing named entity recognition and serialization on the initial text recognition result in turn to obtain a target text recognition result of the prescription to be recognized. The present invention realizes effective processing of noise in prescription pictures, not only improves the accuracy and robustness of recognition, but also enhances the understanding of text semantics and the recognition ability of named entities, and can better meet the needs of medical information management.

Description

Technical Field

[0001] The present invention relates to the field of medical information and artificial intelligence, and in particular to a prescription image recognition method, system, device and storage medium. Background Art

[0002] With the advancement of medical informatization and the popularization of paperless office in hospitals, the electronic management of prescription images has become increasingly important. However, in practical applications, prescription images are often interfered by various noises, such as illegible handwriting, uneven lighting, and blurred images. These noise interferences seriously affect the accuracy and reliability of automatic recognition of prescription images.

[0003] At present, image recognition technology that deals with noise interference mainly relies on image preprocessing and enhancement techniques, such as image denoising and super-resolution reconstruction. However, most of these methods are performed in the image preprocessing stage and fail to be closely integrated with the subsequent feature extraction and recognition process, making it difficult to significantly improve the robustness of the recognition system as a whole. In addition, when processing complex medical prescription images, existing methods still face the problems of insufficient robustness to noise, limited semantic understanding of text information, and especially poor performance in named entity recognition.

[0004] Therefore, the existing prescription image recognition methods have the problems of low recognition accuracy and insufficient robustness when faced with noise interference. There is an urgent need for a prescription image recognition method that can effectively resist noise in order to improve the accuracy and stability of prescription image recognition. Summary of the invention

[0005] In view of this, the present invention provides a prescription picture recognition method, system, device and storage medium to solve the problems of low recognition accuracy and insufficient robustness when facing noise interference using the existing methods.

[0006] The present invention provides a prescription picture recognition method, the method comprising:

[0007] Obtain a prescription image corresponding to the prescription to be identified, perform text detection on the prescription image, and obtain a text area image;

[0008] Performing feature extraction, feature denoising and feature fusion on the text area image to obtain a target feature map corresponding to the prescription to be identified;

[0009] Performing image distillation processing on the target feature map to obtain an initial text recognition result;

[0010] The initial text recognition result is sequentially subjected to named entity recognition and serialization to obtain a target text recognition result of the prescription to be recognized.

[0011] Optionally, the performing feature extraction, feature denoising and feature fusion on the text area image to obtain a target feature map corresponding to the prescription to be identified includes:

[0012] Using a ResNet residual network, extracting features from the text area image to obtain a low-resolution feature map;

[0013] Using a super-resolution denoising module, the low-resolution feature map is subjected to feature denoising to obtain a target high-resolution feature map;

[0014] The CBAM convolutional block attention module is used to perform feature fusion on the low-resolution feature map and the target high-resolution feature map to obtain the target feature map corresponding to the prescription to be identified.

[0015] Optionally, the ResNet residual network includes multiple convolutional layers;

[0016] The ResNet residual network is used to extract features from the text area image to obtain a low-resolution feature map, including:

[0017] The first convolutional layer of the ResNet residual network is used to extract features of the text area image to obtain the low-resolution feature map.

[0018] Optionally, the super-resolution denoising module includes an adaptive upsampler and a no-local processing layer;

[0019] The method of using a super-resolution denoising module to perform feature denoising on the low-resolution feature map to obtain a target high-resolution feature map includes:

[0020] Using the adaptive upsampler, upsampling the low-resolution feature map to obtain an original high-resolution feature map;

[0021] The original high-resolution feature map is denoised by using the no-local non-local processing layer to obtain the target high-resolution feature map.

[0022] Optionally, the using the adaptive upsampler to upsample the low-resolution feature map includes:

[0023] Select a two-dimensional feature position in the low-resolution feature map, and upsample the selected two-dimensional feature position using an adaptive upsampler based on a preset offset, to calculate a plurality of sampling positions at the selected two-dimensional feature position;

[0024] Assume that the selected two-dimensional feature position is (h', w'), and the formula for calculating the kth sampling position at the two-dimensional feature position (h', w') is as follows:

[0025] ;

[0026] Among them, p k is the kth sampling position at the two-dimensional feature position (h', w'), h' and w' are the height and width corresponding to the two-dimensional feature position (h', w'), respectively. and are the preset offsets of the kth sampling position at the two-dimensional feature position (h', w') in the height dimension and the width dimension respectively;

[0027] Using a bilinear interpolation method, a sampling feature value corresponding to each sampling position at the selected two-dimensional feature position is obtained from the low-resolution feature map;

[0028] For the selected two-dimensional feature position (h', w'), calculate the kth sampling position p k The corresponding sampling eigenvalue formula is:

[0029] ;

[0030] Among them, L1(p k ) is the kth sampling position p at the two-dimensional feature position (h', w') k The sampling eigenvalues ​​of is a bilinear interpolation function, L1 is the low-resolution feature map;

[0031] Performing weighted summation on the sampling eigenvalues ​​at all sampling positions at the selected two-dimensional feature position to obtain the original high-resolution eigenvalue at the selected two-dimensional feature position;

[0032] For the selected two-dimensional feature position (h', w'), the formula for calculating the corresponding original high-resolution feature value is:

[0033] ;

[0034] Among them, UL1(h',w') is the original high-resolution eigenvalue at the two-dimensional feature position (h',w'), K 2 is the total number of channels of the low-resolution feature map, W k (h', w') is the preset weight corresponding to the kth sampling position at the two-dimensional feature position (h', w');

[0035] Traverse each two-dimensional feature position in the low-resolution feature map, and obtain the original high-resolution feature value at each two-dimensional feature position in the same way; and obtain the original high-resolution feature map based on the original high-resolution feature values ​​at all two-dimensional feature positions.

[0036] Optionally, the using the no-local non-local processing layer to denoise the original high-resolution feature map to obtain the target high-resolution feature map includes:

[0037] Three learnable embedding transformation matrices are predefined;

[0038] Using the three embedding transformation matrices, respectively perform linear embedding transformation on the original high-resolution feature value of each pixel in the original high-resolution feature map to obtain three feature vectors corresponding to each pixel;

[0039] For pixel i in the original high-resolution feature map, the formula for calculating the three feature vectors corresponding to pixel i is specifically:

[0040] ;

[0041] in, , and are the three eigenvectors corresponding to pixel i, , and They are the three embedding transformation matrices, UL1 i is the original high-resolution feature value corresponding to pixel i in the original high-resolution feature map;

[0042] Calculate the similarity weight between each pixel and the neighboring pixels in the original high-resolution feature map and the normalization factor corresponding to each pixel;

[0043] For pixel i in the original high-resolution feature map, let the neighborhood of pixel i be S i , the neighborhood pixel j satisfies j∈S i , the formula for calculating the similarity weight between pixel i and neighboring pixel j is:

[0044] ;

[0045] in, Similarity weight between pixel i and neighboring pixel j, UL1 j is the original high-resolution feature value corresponding to the neighborhood pixel j in the original high-resolution feature map, is the feature vector The transposed vector of is an exponential function with the natural constant e as base;

[0046] For pixel i in the original high-resolution feature map, the formula for calculating the normalization factor corresponding to pixel i is specifically:

[0047] ;

[0048] in, is the normalization factor corresponding to pixel i, and UL1 is the original high-resolution feature map;

[0049] According to the similarity weight between each pixel and the neighboring pixels in the original high-resolution feature map and the normalization factor corresponding to each pixel, a non-local operation is performed on each pixel in the original high-resolution feature map to obtain a non-local operation result corresponding to each pixel;

[0050] For pixel i in the original high-resolution feature map, the formula for calculating the non-local operation result corresponding to pixel i is specifically:

[0051] ;

[0052] Among them, HL1 i is the non-local operation result corresponding to pixel i, g(UL1 j ) is a feature vector of neighborhood pixel j;

[0053] According to the non-local operation results of all pixels, the target high-resolution feature map is obtained.

[0054] Optionally, the CBAM convolutional block attention module includes a channel attention submodule and a spatial attention submodule;

[0055] The method of using the CBAM convolutional block attention module to perform feature fusion on the low-resolution feature map and the target high-resolution feature map to obtain the target feature map corresponding to the prescription to be identified includes:

[0056] Performing feature dimension reduction on the target high-resolution feature map according to a preset convolution block, and performing feature splicing on the low-resolution feature map and the target high-resolution feature map after feature dimension reduction to obtain a spliced ​​feature map;

[0057] The expression of the splicing feature map is specifically: F Concat = Concat [ Conv 3 × 3 ( HL 1 ), L 1 ] ;

[0058] Among them, F Concat is the splicing feature map, is the splicing operation function, is a 3×3 convolution operation function, HL1 is the target high-resolution feature map, and L1 is the low-resolution feature map;

[0059] Inputting the concatenated feature map into the channel attention submodule to generate a channel attention map; fusing the channel attention map and the concatenated feature map to obtain a first output feature map;

[0060] The expression of the channel attention map is specifically:

[0061] M c ( F Concat ) = σ { MLP [ AvgPool ( F Concat )] + MLP [ MaxPool ( F Concat )] } ;

[0062] The expression of the first output feature map is specifically: ;

[0063] in, is the channel attention map, is the first output feature map, is the Sigmoid activation function, , and They are multi-layer perceptron function, global average pooling operation function and global maximum pooling operation function respectively;

[0064] Inputting the first output feature map into the spatial attention submodule to generate a spatial attention map; fusing the spatial attention map with the first output feature map to obtain a second output feature map;

[0065] The expression of the spatial attention map is specifically:

[0066] M s ( F channel ) = σ { Conv 7 × 7 [ AvgPool ( F channel ); MaxPool ( F channel )] } ;

[0067] The expression of the second output feature map is specifically: ;

[0068] in, is the spatial attention map, is the second output feature map, is the 7×7 convolution operation function, [;] Represents the concatenation operation in the channel dimension;

[0069] The remaining convolutional layers of the ResNet residual network are used to perform feature extraction on the second output feature map to obtain the target feature map corresponding to the prescription to be identified.

[0070] Optionally, after performing feature extraction on the second output feature map, the method further includes:

[0071] Dividing the second output feature map after feature extraction into a plurality of one-dimensional sequences along the width direction;

[0072] The bidirectional LSTM model is used to perform forward and backward processing on all one-dimensional sequences to obtain the character probability distribution;

[0073] The character probability distribution is decoded using a connectivity temporal classification loss function.

[0074] Optionally, performing image distillation processing on the target feature map to obtain an initial text recognition result includes:

[0075] Get the initial student network and the pre-trained teacher network;

[0076] Inputting a preset noise-free prescription picture into the teacher network, adding noise into the noise-free prescription picture to obtain a noisy prescription picture, and inputting the noisy prescription picture into the initial student network;

[0077] Using a mean square error loss function between a first feature output by the teacher network and a second feature output by the initial student network as a distillation function, distilling the initial student network to obtain a target student network;

[0078] The expression of the mean square error loss function is specifically: ;

[0079] Among them, L distill is the mean square error loss function, L4 teacher and L4 student are the first feature output by the teacher network and the second feature output by the initial student network, respectively. is the norm;

[0080] The target feature map is recognized using the target student network to obtain the initial text recognition result.

[0081] Optionally, the step of sequentially performing named entity recognition and serialization on the initial text recognition result to obtain the target text recognition result of the prescription to be recognized includes:

[0082] Predefine the BERT model and obtain the fine-tuned BERT model;

[0083] Using a fine-tuned BERT model to perform named entity recognition on the initial text recognition result to obtain named entity information of the prescription to be recognized;

[0084] The named entity information is serialized according to the JSON format to obtain and store the target text recognition result.

[0085] In addition, the present invention also provides a prescription picture recognition system, which is applied to the above-mentioned prescription picture recognition method, and the system includes:

[0086] An image acquisition module, used to acquire a prescription image corresponding to the prescription to be identified;

[0087] A text detection module, used to perform text detection on the prescription image to obtain a text area image;

[0088] A feature processing module, used for performing feature extraction, feature denoising and feature fusion on the text area image to obtain a target feature map corresponding to the prescription to be identified;

[0089] A distillation processing module, used to perform image distillation processing on the target feature map to obtain an initial text recognition result;

[0090] The recognition processing module is used to perform named entity recognition and serialization on the initial text recognition results in sequence to obtain the target text recognition results of the prescription to be recognized.

[0091] In addition, the present invention also provides a prescription picture recognition device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the method steps in the aforementioned prescription picture recognition method when executed.

[0092] In addition, the present invention also provides a computer storage medium, which includes: at least one instruction, and when the instruction is executed by a computer, the method steps in the above-mentioned prescription image recognition method are implemented.

[0093] The beneficial effects of the present invention are as follows: by acquiring prescription pictures and detecting text, a text area image is obtained, which is convenient for subsequent feature processing and recognition of the text area, avoiding interference from irrelevant areas, and also reducing the amount of subsequent processing data to a certain extent; by performing feature extraction, feature denoising and feature fusion on the text area image, multi-level feature extraction and fusion can be achieved, thereby achieving high-precision recognition and strong robustness processing of the text area image; by image distillation processing, the robustness and generalization ability of the model can be further improved, a more accurate prediction of the target feature map can be achieved, and a more accurate initial text recognition result can be obtained; finally, by named entity recognition, the understanding of text semantics can be improved, key named entities can be accurately identified, and after serialization, the target text recognition result that is convenient to read and store can be obtained in a serialized data format;

[0094] The prescription image recognition method, system, device and storage medium of the present invention realize the effective processing of noise in prescription images, which not only improves the accuracy and robustness of recognition, but also enhances the understanding of text semantics and the recognition ability of named entities, and can better meet the needs of medical information management. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present invention in any way. In the accompanying drawings:

[0096] Figure 1 A flowchart of a prescription picture recognition method in Embodiment 1 of the present invention is shown;

[0097] Figure 2 A flow chart showing feature extraction, feature denoising and feature fusion of a text region image in the first embodiment of the present invention is shown;

[0098] Figure 3 A model diagram showing feature extraction, feature denoising and feature fusion of a text region image in the first embodiment of the present invention is shown;

[0099] Figure 4 A flowchart of image distillation processing in Embodiment 1 of the present invention is shown;

[0100] Figure 5 A model diagram of image distillation processing in Embodiment 1 of the present invention is shown;

[0101] Figure 6 A flowchart showing named entity recognition and serialization of initial text recognition results in sequence in Embodiment 1 of the present invention is shown;

[0102] Figure 7 The figure shows a structural diagram of a prescription picture recognition system in the second embodiment of the present invention. DETAILED DESCRIPTION

[0103] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0104] Embodiment 1

[0105] This embodiment provides a prescription image recognition method, such as Figure 1 As shown, the method includes:

[0106] S1: Obtain a prescription image corresponding to the prescription to be identified, perform text detection on the prescription image, and obtain a text area image;

[0107] S2: performing feature extraction, feature denoising and feature fusion on the text area image to obtain a target feature map corresponding to the prescription to be identified;

[0108] S3: performing image distillation processing on the target feature map to obtain an initial text recognition result;

[0109] S4: performing named entity recognition and serialization on the initial text recognition result in sequence to obtain a target text recognition result of the prescription to be recognized.

[0110] In this embodiment, a text area image is obtained by acquiring a prescription image and detecting text, which facilitates subsequent feature processing and recognition of the text area, avoids interference from irrelevant areas, and reduces the amount of subsequent processing data to a certain extent; by performing feature extraction, feature denoising and feature fusion on the text area image, multi-level feature extraction and fusion can be achieved, thereby achieving high-precision recognition and strong robustness processing of the text area image; through image distillation processing, the robustness and generalization ability of the model can be further improved, and a more accurate prediction of the target feature map can be achieved, thereby obtaining a more accurate initial text recognition result; finally, through named entity recognition, the understanding of text semantics can be improved, and key named entities can be accurately identified. After serialization, the target text recognition result that is easy to read and store is obtained in a serialized data format.

[0111] The prescription image recognition method of this embodiment realizes the effective processing of noise in prescription images, which not only improves the accuracy and robustness of recognition, but also enhances the understanding of text semantics and the recognition ability of named entities, and can better meet the needs of medical information management.

[0112] Each step of the prescription image recognition method of this embodiment is described in detail below.

[0113] Preferably, in this embodiment S1, an image acquisition device, such as a high-definition camera, is used to take a picture of the prescription to be identified to obtain a prescription picture; then a conventional text detection method is used, such as a text localization method based on deep learning, a text area extraction method based on the OpenCV visual library, etc., which is not limited here.

[0114] Preferably, if Figure 2 As shown, this embodiment S2 includes:

[0115] S21: Using a ResNet residual network, extracting features from the text area image to obtain a low-resolution feature map;

[0116] S22: using a super-resolution denoising module to perform feature denoising on the low-resolution feature map to obtain a target high-resolution feature map;

[0117] S23: Using the CBAM convolutional block attention module, perform feature fusion on the low-resolution feature map and the target high-resolution feature map to obtain the target feature map corresponding to the prescription to be identified.

[0118] Based on the ResNet residual network and the CBAM convolutional block attention module, multi-level feature extraction and feature fusion are formed, which can significantly improve the recognition accuracy; after using the ResNet residual network for feature extraction and before using the CBAM convolutional block attention module for feature fusion, the noise in the image can be effectively removed, thereby enhancing the model's anti-noise ability; through the above steps, it can be ensured that the accuracy and robustness of prescription image recognition are effectively improved.

[0119] Preferably, in this embodiment S21, the ResNet residual network includes multiple convolutional layers;

[0120] S21 specifically includes:

[0121] The first convolutional layer of the ResNet residual network is used to extract features of the text area image to obtain the low-resolution feature map.

[0122] ResNet (Residual Network) is a convolutional neural network (CNN) in deep learning. It solves the problem of gradient vanishing or gradient exploding when the traditional convolutional neural network is deepened by introducing residual blocks and residual connections. ResNet residual network can build a very deep network structure. For the text area image corresponding to the prescription picture, this deep structure helps to capture the subtle differences and key information in the image. At the same time, the residual connection in ResNet allows information to be directly transferred between layers, thereby retaining more original feature information. Therefore, in the feature extraction of the text area image corresponding to the prescription picture, this helps to retain key features such as text, symbols and layout on the prescription, providing more accurate and comprehensive feature representation for subsequent classification, recognition and other tasks. In addition, ResNet residual network has good adaptability and robustness to different types of prescription pictures.

[0123] Preferably, in this embodiment S22, the super-resolution denoising module includes an adaptive upsampler and a no-local processing layer;

[0124] S22 includes:

[0125] S221: using the adaptive upsampler, upsampling the low-resolution feature map to obtain an original high-resolution feature map;

[0126] S222: De-noising the original high-resolution feature map using the no-local processing layer to obtain the target high-resolution feature map.

[0127] The above-mentioned super-resolution denoising module including the adaptive upsampler and the no-local non-local processing layer combines the advantages of the two image processing technologies of super-resolution and denoising, and can simultaneously improve the resolution of the image and remove the noise therein. Among them, the adaptive upsampler can realize the efficient upsampling of the low-resolution feature map with lower computing resources, and convert the low-resolution input feature map (i.e., the low-resolution feature map) into a high-resolution output feature map (i.e., the original high-resolution feature map); and then use the no-local non-local processing layer for denoising, which can use the spatial redundant information of the image to remove the noise in the original high-resolution feature map, so that the extracted target high-resolution feature map is more robust to noise.

[0128] The adaptive upsampler can be selected according to the actual situation, and has the functions and advantages of dynamically adjusting the sampling rate, improving efficiency and accuracy, reducing noise and artifacts, adapting to different scenes, user-friendly interface and parameter setting, saving computing resources, and supporting parallel computing. The appropriate type can be selected according to the actual situation. This embodiment specifically adopts the DySample dynamic upsampler, which has the advantages of lightweight, high efficiency, optimized sampling process, wide applicability, simple implementation, excellent performance, and no need for high-resolution guided features. It can better process the prescription feature map and realize the conversion of the low-resolution feature map into the original high-resolution feature map.

[0129] Furthermore, S221 includes:

[0130] S2211: Select a two-dimensional feature position in the low-resolution feature map, and upsample the selected two-dimensional feature position using an adaptive upsampler based on a preset offset, to calculate a plurality of sampling positions at the selected two-dimensional feature position;

[0131] S2212: using a bilinear interpolation method to obtain a sampling feature value corresponding to each sampling position at the selected two-dimensional feature position from the low-resolution feature map;

[0132] S2213: performing weighted summation on the sampling feature values ​​at all sampling positions at the selected two-dimensional feature position to obtain the original high-resolution feature value at the selected two-dimensional feature position;

[0133] S2214: traverse each two-dimensional feature position in the low-resolution feature map, and obtain the original high-resolution feature value at each two-dimensional feature position according to the same method; and obtain the original high-resolution feature map based on the original high-resolution feature values ​​at all two-dimensional feature positions.

[0134] Specifically, in S2211 of this embodiment, assuming that the selected two-dimensional feature position is (h', w'), the formula for calculating the kth sampling position at the two-dimensional feature position (h', w') is specifically:

[0135] ;

[0136] Among them, p k is the kth sampling position at the two-dimensional feature position (h', w'), h' and w' are the height and width corresponding to the two-dimensional feature position (h', w'), respectively. and are the preset offsets of the kth sampling position at the two-dimensional feature position (h', w') in the height dimension and the width dimension respectively.

[0137] Specifically, in S2212 of this embodiment, for the selected two-dimensional feature position (h', w'), the kth sampling position p is calculated. k The corresponding sampling eigenvalue formula is:

[0138] ;

[0139] Among them, L1(p k ) is the kth sampling position p at the two-dimensional feature position (h', w') k The sampling eigenvalues ​​of is a bilinear interpolation function, and L1 is the low-resolution feature map.

[0140] Specifically, in S2213 of this embodiment, for the selected two-dimensional feature position (h', w'), the formula for calculating the corresponding original high-resolution feature value is specifically:

[0141] ;

[0142] Among them, UL1(h',w') is the original high-resolution eigenvalue at the two-dimensional feature position (h',w'), K 2 is the total number of channels of the low-resolution feature map, W k (h', w') is the preset weight corresponding to the kth sampling position at the two-dimensional feature position (h', w').

[0143] In the above-mentioned upsampling process of the present embodiment, a part with a large difference can be selected for sampling at each two-dimensional feature position based on a preset offset, and then a bilinear interpolation method is used to calculate the sampling feature value and the weighted sum of the sampling feature value based on the sampling position. By retaining only pixel features with large differences, the amount of data can be effectively reduced, the calculation complexity can be reduced, and the upsampling speed and efficiency can be improved.

[0144] Specifically, for the low-resolution feature map L1, its dimension is C×H×W, where C is the number of channels, and H and W are the height and width of the low-resolution feature map L1, respectively. Perform convolution operation on the low-resolution feature map L1 to obtain the offset , whose dimension is 2K 2 ×H'×W'. Through the convolutional layer Perform convolution operation on the low-resolution feature map L1 to obtain the weight , whose dimension is K 2 ×H'×W'.

[0145] The two-dimensional feature position in the low-resolution feature map L1 refers to the feature position that does not include the number of channels, and the offset is the three-dimensional offset, then the preset offset of the two-dimensional feature position in the height dimension and the width dimension is the offset The offset components in the height and width dimensions.

[0146] Further, S222 includes:

[0147] S2221: pre-define three learnable embedding transformation matrices;

[0148] S2222: using the three embedding transformation matrices, respectively perform linear embedding transformation on the original high-resolution feature value of each pixel in the original high-resolution feature map to obtain three feature vectors corresponding to each pixel;

[0149] S2223: Calculate the similarity weight between each pixel and the neighboring pixels in the original high-resolution feature map and the normalization factor corresponding to each pixel;

[0150] S2224: performing a non-local operation on each pixel in the original high-resolution feature map according to a similarity weight between each pixel and a neighboring pixel in the original high-resolution feature map and a normalization factor corresponding to each pixel, to obtain a non-local operation result corresponding to each pixel;

[0151] S2225: Obtain the target high-resolution feature map according to the non-local operation results of all pixels.

[0152] Specifically, in this embodiment S2221, the three learnable embedding transformation matrices are: , and ,in, , and , where m and l refer to the sizes of the embedding dimension and feature dimension in the embedding transformation matrix, respectively.

[0153] Specifically, in S2222 of this embodiment, for pixel i in the original high-resolution feature map, the formula for calculating the three feature vectors corresponding to pixel i is specifically:

[0154] ;

[0155] in, , and are the three eigenvectors corresponding to pixel i, , and They are the three embedding transformation matrices, UL1 i is the original high-resolution feature value corresponding to pixel i in the original high-resolution feature map.

[0156] Specifically, in S2223 of this embodiment, for pixel i in the original high-resolution feature map, the neighborhood of pixel i is set to S i , the neighborhood pixel j satisfies j∈S i , the formula for calculating the similarity weight between pixel i and neighboring pixel j is:

[0157] ;

[0158] in, Similarity weight between pixel i and neighboring pixel j, UL1 j is the original high-resolution feature value corresponding to the neighborhood pixel j in the original high-resolution feature map, is the feature vector The transposed vector of is an exponential function with the natural constant e as base;

[0159] For pixel i in the original high-resolution feature map, the formula for calculating the normalization factor corresponding to pixel i is specifically:

[0160] ;

[0161] in, is the normalization factor corresponding to pixel i, and UL1 is the original high-resolution feature map.

[0162] Specifically, in S2224 of this embodiment, for pixel i in the original high-resolution feature map, the formula for calculating the non-local operation result corresponding to pixel i is specifically:

[0163] ;

[0164] Among them, HL1 iis the non-local operation result corresponding to pixel i, g(UL1 j ) is a feature vector of neighborhood pixel j.

[0165] In the denoising process of this embodiment, three learnable embedding transformation matrices are first used to perform a linear embedding transformation on the original high-resolution feature value of each pixel in the original high-resolution feature map, which can reduce the dimension in the original high-resolution feature map while retaining the structure or relationship of the feature data. The obtained feature vector is convenient for subsequent further calculation and analysis, thereby improving efficiency. Then, by calculating the similarity weights between the pixel and the neighboring pixels, it can help capture the local structure and relationship between the pixels in the original high-resolution feature map; by calculating the normalization factor of the pixel, all the calculated similarity weights can be normalized, so that the normalized similarity weights can better reflect the relative relationship between the pixels and have better consistency and stability. Through the above denoising steps, not only the accuracy and robustness of image recognition can be improved, but also the recognition efficiency can be improved.

[0166] Non-local operations can consider the weighted sum of features of all pixel positions in the original high-resolution feature map, capture the global information of the feature image, enhance the expressiveness of image features, and retain and utilize the local structure and detail information of the feature image, ensuring the optimal denoising effect while retaining more complete information; at the same time, it can also improve the robustness and accuracy of the algorithm, reduce computational complexity, optimize performance, and improve computational efficiency.

[0167] In this embodiment S2225, the set of non-local operation results corresponding to all pixels {HL1 i}, which constitutes the target high-resolution feature map HL1.

[0168] Preferably, in this embodiment S23, the CBAM convolutional block attention module includes a channel attention submodule and a spatial attention submodule;

[0169] S23 includes:

[0170] S231: performing feature dimension reduction on the target high-resolution feature map according to a preset convolution block, and performing feature splicing on the low-resolution feature map and the target high-resolution feature map after feature dimension reduction to obtain a spliced ​​feature map;

[0171] The expression of the splicing feature map is specifically: F Concat = Concat [ Conv 3 × 3 ( HL 1 ), L 1 ] ;

[0172] Among them, F Concat is the splicing feature map, is the splicing operation function, is a 3×3 convolution operation function, HL1 is the target high-resolution feature map, and L1 is the low-resolution feature map;

[0173] S232: Input the concatenated feature map into the channel attention submodule to generate a channel attention map; fuse the channel attention map and the concatenated feature map to obtain a first output feature map;

[0174] The expression of the channel attention map is specifically:

[0175] M c ( F Concat ) = σ { MLP [ AvgPool ( F Concat )] + MLP [ MaxPool ( F Concat )] } ;

[0176] The expression of the first output feature map is specifically: ;

[0177] in, is the channel attention map, is the first output feature map, is the Sigmoid activation function, , and They are multi-layer perceptron function, global average pooling operation function and global maximum pooling operation function respectively;

[0178] S233: Inputting the first output feature map into the spatial attention submodule to generate a spatial attention map; fusing the spatial attention map with the first output feature map to obtain a second output feature map;

[0179] The expression of the spatial attention map is specifically:

[0180] M s ( F channel ) = σ { Conv 7 × 7 [ AvgPool ( F channel ); MaxPool ( F channel )] } ;

[0181] The expression of the second output feature map is specifically: ;

[0182] in, is the spatial attention map, is the second output feature map, is the 7×7 convolution operation function, [;] Represents the concatenation operation in the channel dimension;

[0183] S234: Utilize the remaining convolutional layers of the ResNet residual network to perform feature extraction on the second output feature map to obtain the target feature map corresponding to the prescription to be identified.

[0184] By performing feature dimensionality reduction on the target high-resolution feature map, it is convenient to splice the target high-resolution feature map with the low-resolution feature map in the same dimension and resolution; by splicing the features of the target high-resolution feature map with the low-resolution feature map, it is convenient to provide rich input data for the channel attention submodule and the spatial attention submodule in the subsequent CBAM convolution block attention module; the channel attention submodule and the spatial attention submodule in the CBAM convolution block attention module can respectively focus on the channel coding information and spatial coding information in the spliced ​​feature map. The application of these two attention submodules can increase the attention to important features, thereby improving the performance of feature extraction; finally, combined with the residual convolution layer of the ResNet residual network, the multi-level feature extraction and fusion technology is applied again, which can significantly improve the recognition accuracy of features in prescription images.

[0185] In this embodiment, after feature extraction in step S234, the feature map obtained is represented as L4. The model diagram of feature extraction, feature denoising and feature fusion described in steps S21 to S23 is as follows: Figure 3 shown.

[0186] Preferably, in S234, after extracting features from the second output feature map, the method further includes:

[0187] Dividing the second output feature map after feature extraction into a plurality of one-dimensional sequences along the width direction;

[0188] The bidirectional LSTM model is used to perform forward and backward processing on all one-dimensional sequences to obtain the character probability distribution;

[0189] The character probability distribution is decoded using a connectivity temporal classification loss function.

[0190] Dividing the second output feature map after feature extraction into a one-dimensional sequence can significantly reduce the complexity of subsequent processing and retain key information. On the other hand, it is more convenient for the bidirectional LSTM model to process and improve computational efficiency. The bidirectional LSTM model is also called the Bi-LSTM model, which is the Bidirectional Long Short-Term Memory bidirectional long short-term memory network model. It combines the forward LSTM and the backward LSTM. The forward LSTM performs forward sequence learning from the start position to the end position of the sequence, while the backward LSTM performs reverse sequence learning from the end position to the start position of the sequence. In this way, the bidirectional LSTM can capture information from the past and the future at each time point in the sequence, so as to more comprehensively understand the context of the sequence; based on the forward and backward processing of the bidirectional LSTM model, the bidirectional LSTM can capture more comprehensive context information, optimize the understanding and recognition of context information, and thus more accurately understand the intrinsic structure of the sequence data and obtain a more accurate character probability distribution. The Connectionist Temporal Classification Loss (CTC Loss) is a special loss function that is mainly used to process sequence data and can better identify characters. Therefore, the decoding process of the Connectivity Temporal Classification Loss function can decode the character probability distribution output by the bidirectional LSTM into an actual character sequence, which can effectively improve the prediction performance and ensure the recognition accuracy of the final prescription image.

[0191] Preferably, if Figure 4 As shown, this embodiment S3 includes:

[0192] S31: Obtain the initial student network and the pre-trained teacher network;

[0193] S32: inputting a preset noise-free prescription picture into the teacher network, adding noise into the noise-free prescription picture to obtain a noisy prescription picture, and inputting the noisy prescription picture into the initial student network;

[0194] S33: using a mean square error loss function between a first feature output by the teacher network and a second feature output by the initial student network as a distillation function, distilling the initial student network to obtain a target student network;

[0195] The expression of the mean square error loss function is specifically: ;

[0196] Among them, L distill is the mean square error loss function, L4 teacher and L4 studentare the first feature output by the teacher network and the second feature output by the initial student network, respectively. is the norm;

[0197] S34: Using the target student network to identify the target feature map, to obtain the initial text recognition result.

[0198] The above-mentioned image distillation process can obtain a prescription image encoder that can process perturbed images (i.e., noisy prescription images) and generate features close to noise-free images. By minimizing the difference between the second feature generated by the student network and the clean first feature generated by the teacher network, the robustness of the final target student network to noise in the input image is enhanced, which not only improves the feature extraction capability, but also enhances the generalization performance of the overall model, which helps to improve the robustness and accuracy of the prescription image recognition or classification system.

[0199] The model diagram of the image distillation process described in step S4 of this embodiment is as follows: Figure 5 shown.

[0200] Preferably, if Figure 6 As shown, this embodiment S4 includes:

[0201] S41: pre-define a BERT model and obtain a fine-tuned BERT model;

[0202] S42: using a fine-tuned BERT model to perform named entity recognition on the initial text recognition result to obtain named entity information of the prescription to be recognized;

[0203] S43: Serialize the named entity information according to the JSON format to obtain and store the target text recognition result.

[0204] By fine-tuning the pre-defined BERT model, the fine-tuned BERT model can understand the context of the medical field and accurately identify the named entities of the medical text in the initial text recognition results; when the initial recognition results are used as input and sent to the fine-tuned BERT model. The BERT model (Bidirectional Encoder Representationsfrom Transformers model) will use contextual information to have a deeper semantic understanding of the text and generate more accurate named entity information (such as NER tags corresponding to name, gender, age, clinic number, etc., NER tags refer to Named Entity Recognition named entity tags), thereby enriching the semantic understanding of the text; because JSON is a lightweight data exchange format, the named entity information is finally serialized in JSON format, which can obtain the target text recognition results that are easy for humans to read and write, and easy for machines to parse and generate.

[0205] In a specific implementation, for the same prescription image, the output result after recognition using traditional technology is:

[0206] Pediatrics ***** Prescription Fee: Self-paid medical certificate / medical insurance card number Prescription number: 120008 Name: Zhang San Gender: Male Age: 1 year and 7 months Outpatient / inpatient medical record number: EL1508120005 Department (ward / bed number): General Practice Clinical diagnosis: Hand, foot and mouth disease Date of issue: 2015-08-12 Address / telephone: Guangzhou, Guangdong / 07562222470 Rp.1. Cefixime granules 50mg*6 bags X1 box Usage: Oral 50mg twice a day 2 Ribavirin granules 50mg *18 bags X1 box Usage: Oral administration 50mg three times a day 3. Isatis root granules 10g * 20 bags / box X1 large bag Usage: Oral administration 10g three times a day 10ml: 30gX1 box Usage: Apply externally to the affected area 4. Stomatitis spray appropriate amount three times a day (blank below) Drug costs Medical expenses Medical materials expenses Examination and treatment expenses Other expenses Total 40.50 10.00 1.00 35.000.00 86.50 Doctor: Dr. Li Drug amount: 40.50 Review pharmacist Dispensing pharmacist / physician Checking and dispensing pharmacist.

[0207] The output result after recognition by the prescription image recognition method of this embodiment is:

[0208] {

[0209] "Prescribing Information": {

[0210] "Hospital Name": "*****Prescription",

[0211] "Fee Type": "Self-paid",

[0212] "Prescription Number": "120008",

[0213] "Patient Information": {

[0214] "Name": "Zhang San",

[0215] "Gender": "Male",

[0216] "Age": "1 year and 7 months",

[0217] "Outpatient / Inpatient Medical Record Number": "EL1508120005",

[0218] "Address / Phone": "Guangzhou, Guangdong / 07562222470"

[0219] },

[0220] "Clinical diagnosis": "Hand, foot and mouth disease",

[0221] "Issuing Date": "2015-08-12",

[0222] "Drug Prescription": [

[0223] {

[0224] "Drug Name": "Cefixime Granules",

[0225] "Specification": "50mg*6 bags",

[0226] "Quantity": "X1 box",

[0227] "Usage": "Oral administration 50 mg, twice a day"

[0228] },

[0229] {

[0230] "Drug Name": "Ribavirin Granules",

[0231] "Specification": "50mg*18 bags",

[0232] "Quantity": "X1 box",

[0233] "Usage": "Oral administration 50 mg three times a day"

[0234] },

[0235] {

[0236] "Drug name": "Isatis root granules",

[0237] "Specification": "10g*20 bags / box",

[0238] "Quantity": "X1 big bag",

[0239] "Usage": "Oral administration: 10g each time, three times a day"

[0240] },

[0241] {

[0242] "Drug Name": "Stomatitis Spray",

[0243] "Usage": "Appropriate amount each time, three times a day, apply to the affected area"

[0244] }

[0245] ],

[0246] "Expense Details": {

[0247] "Drug cost": "40.50",

[0248] "Medical Fee": "10.00",

[0249] "Medical Materials Fee": "1.00",

[0250] "Examination and treatment fee": "35.00",

[0251] "Other Fees": "0.00",

[0252] "Total": "86.50"

[0253] },

[0254] "Doctor": "Doctor Li",

[0255] "Auditing Pharmacist": "",

[0256] "Pharmacist / Pharmacist": "",

[0257] "Pharmacist who checks and dispenses medicine": ""

[0258] }

[0259] }

[0260] By comparison, it can be seen that the prescription image recognition method described in this embodiment can show significant advantages in accurately identifying entity information in images compared to traditional technologies. By introducing specific processing modules, various noise interferences in the synthetic data set are effectively handled, thereby improving the robustness and accuracy of the recognition process. The experimental results on the synthetic data set show that after adding these modules, not only the accuracy of text recognition is improved, but the recognition system can also accurately identify entity information in prescription images. This result shows that the present invention has achieved comprehensive improvements in recognition accuracy, noise resistance, text semantic understanding and named entity recognition, can significantly improve the recognition accuracy of entity information, can effectively meet the needs of medical information management, and provide reliable technical support for applications in fields such as medical information management, and can effectively promote the development of paperless office in hospitals.

[0261] Embodiment 2

[0262] A prescription image recognition system is applied to the prescription image recognition method of embodiment 1, such as Figure 7 As shown, the system includes:

[0263] An image acquisition module, used to acquire a prescription image corresponding to the prescription to be identified;

[0264] A text detection module, used to perform text detection on the prescription image to obtain a text area image;

[0265] A feature processing module, used for performing feature extraction, feature denoising and feature fusion on the text area image to obtain a target feature map corresponding to the prescription to be identified;

[0266] A distillation processing module, used to perform image distillation processing on the target feature map to obtain an initial text recognition result;

[0267] The recognition processing module is used to perform named entity recognition and serialization on the initial text recognition results in sequence to obtain the target text recognition results of the prescription to be recognized.

[0268] In this embodiment, the prescription image is acquired by the image acquisition module and the text detection module detects the text of the prescription image to obtain a text area image, which is convenient for subsequent feature processing and recognition of the text area, avoids interference from irrelevant areas, and reduces the amount of subsequent processing data to a certain extent; the text area image is subjected to feature extraction, feature denoising and feature fusion by the feature processing module, which can realize multi-level feature extraction and fusion, and then realize high-precision recognition and strong robustness processing of the text area image; the image distillation processing of the target feature map by the distillation processing module can further improve the robustness and generalization ability of the model, realize more accurate prediction of the target feature map, and obtain more accurate initial text recognition results; finally, the named entity recognition of the recognition processing module can improve the understanding of text semantics, accurately identify key named entities, and then obtain the target text recognition results that are easy to read and store in a serialized data format after serialization by the recognition processing module.

[0269] The prescription image recognition system of this embodiment realizes the effective processing of noise in prescription images, which not only improves the accuracy and robustness of recognition, but also enhances the understanding of text semantics and the recognition ability of named entities, and can better meet the needs of medical information management.

[0270] The functions of each module in the prescription image recognition system described in this embodiment are the same as the method steps of the prescription image recognition method described in Example 1. Therefore, for details not provided in this embodiment, please refer to Example 1 and Figures 1 to 6 The detailed description will not be repeated here.

[0271] Embodiment 3

[0272] A prescription picture recognition device comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the computer program implements the method steps in the prescription picture recognition method of the first embodiment when it is executed.

[0273] By storing a computer program in a memory and running it on a processor, effective processing of noise in prescription images is achieved, which not only improves recognition accuracy and robustness, but also enhances the understanding of text semantics and the ability to recognize named entities, and can better meet the needs of medical information management.

[0274] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of a computer device, and uses various interfaces and lines to connect various parts of the entire computer device.

[0275] The memory can be used to store computer programs and / or models. The processor realizes various functions of the computer device by running or executing the computer programs and / or models stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the mobile phone (such as audio data, video data, etc.). In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0276] It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by a computer program. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0277] These computer programs may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0278] These computer programs can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0279] This embodiment further provides a computer storage medium, which includes: at least one instruction, which implements the method steps in the prescription image recognition method of embodiment 1 when the instruction is executed by a computer.

[0280] By executing a computer storage medium containing at least one instruction, effective processing of noise in prescription images is achieved, which not only improves the accuracy and robustness of recognition, but also enhances the understanding of text semantics and the recognition ability of named entities, and can better meet the needs of medical information management.

[0281] Similarly, for details not yet included in this embodiment, see Embodiment 1, Embodiment 2 and Figures 1 to 7 The detailed description will not be repeated here.

[0282] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A prescription image recognition method, characterized in that: The method comprises: Obtain a prescription image corresponding to the prescription to be identified, perform text detection on the prescription image, and obtain a text area image; Performing feature extraction, feature denoising and feature fusion on the text area image to obtain a target feature map corresponding to the prescription to be identified; Performing image distillation processing on the target feature map to obtain an initial text recognition result; Performing named entity recognition and serialization on the initial text recognition result in sequence to obtain a target text recognition result of the prescription to be recognized; The step of performing feature extraction, feature denoising and feature fusion on the text region image to obtain a target feature map corresponding to the prescription to be identified includes: Using a ResNet residual network, extracting features from the text area image to obtain a low-resolution feature map; Using a super-resolution denoising module, the low-resolution feature map is subjected to feature denoising to obtain a target high-resolution feature map; The CBAM convolutional block attention module is used to perform feature fusion on the low-resolution feature map and the target high-resolution feature map to obtain the target feature map corresponding to the prescription to be identified.

2. The prescription image recognition method according to claim 1, characterized in that: The ResNet residual network includes multiple convolutional layers; The ResNet residual network is used to extract features from the text area image to obtain a low-resolution feature map, including: The first convolutional layer of the ResNet residual network is used to extract features of the text area image to obtain the low-resolution feature map.

3. The prescription image recognition method according to claim 1, characterized in that: The super-resolution denoising module includes an adaptive upsampler and a no-local processing layer; The method of using a super-resolution denoising module to perform feature denoising on the low-resolution feature map to obtain a target high-resolution feature map includes: Using the adaptive upsampler, upsampling the low-resolution feature map to obtain an original high-resolution feature map; The original high-resolution feature map is denoised by using the no-local non-local processing layer to obtain the target high-resolution feature map.

4. The prescription image recognition method according to claim 3, characterized in that: The step of upsampling the low-resolution feature map by using the adaptive upsampler comprises: Select a two-dimensional feature position in the low-resolution feature map, and upsample the selected two-dimensional feature position using an adaptive upsampler based on a preset offset, to calculate a plurality of sampling positions at the selected two-dimensional feature position; Assume that the selected two-dimensional feature position is (h', w'), and the formula for calculating the kth sampling position at the two-dimensional feature position (h', w') is as follows: ； Among them, p k is the kth sampling position at the two-dimensional feature position (h', w'), h' and w' are the height and width corresponding to the two-dimensional feature position (h', w'), respectively. and are the preset offsets of the kth sampling position at the two-dimensional feature position (h', w') in the height dimension and the width dimension respectively; Using a bilinear interpolation method, a sampling feature value corresponding to each sampling position at the selected two-dimensional feature position is obtained from the low-resolution feature map; For the selected two-dimensional feature position (h', w'), calculate the kth sampling position p k The corresponding sampling eigenvalue formula is: ； Among them, L1(p k ) is the kth sampling position p at the two-dimensional feature position (h', w') k The sampling eigenvalues ​​of is a bilinear interpolation function, L1 is the low-resolution feature map; Performing weighted summation on the sampling eigenvalues ​​at all sampling positions at the selected two-dimensional feature position to obtain the original high-resolution eigenvalue at the selected two-dimensional feature position; For the selected two-dimensional feature position (h', w'), the formula for calculating the corresponding original high-resolution feature value is: ； Among them, UL1(h',w') is the original high-resolution eigenvalue at the two-dimensional feature position (h',w'), K 2 is the total number of channels of the low-resolution feature map, W k (h', w') is the preset weight corresponding to the kth sampling position at the two-dimensional feature position (h', w'); Traverse each two-dimensional feature position in the low-resolution feature map, and obtain the original high-resolution feature value at each two-dimensional feature position in the same way; and obtain the original high-resolution feature map based on the original high-resolution feature values ​​at all two-dimensional feature positions.

5. The prescription image recognition method according to claim 4, characterized in that: The step of denoising the original high-resolution feature map by using the no-local non-local processing layer to obtain the target high-resolution feature map includes: Three learnable embedding transformation matrices are predefined; Using the three embedding transformation matrices, respectively perform linear embedding transformation on the original high-resolution feature value of each pixel in the original high-resolution feature map to obtain three feature vectors corresponding to each pixel; For pixel i in the original high-resolution feature map, the formula for calculating the three feature vectors corresponding to pixel i is specifically: ； in, , and are the three eigenvectors corresponding to pixel i, , and They are the three embedding transformation matrices, UL1 i is the original high-resolution feature value corresponding to pixel i in the original high-resolution feature map; Calculate the similarity weight between each pixel and the neighboring pixels in the original high-resolution feature map and the normalization factor corresponding to each pixel; For pixel i in the original high-resolution feature map, let the neighborhood of pixel i be S i , the neighborhood pixel j satisfies j∈S i , the formula for calculating the similarity weight between pixel i and neighboring pixel j is: ； in, Similarity weight between pixel i and neighboring pixel j, UL1 j is the original high-resolution feature value corresponding to the neighborhood pixel j in the original high-resolution feature map, is the feature vector The transposed vector of is an exponential function with the natural constant e as base; For pixel i in the original high-resolution feature map, the formula for calculating the normalization factor corresponding to pixel i is specifically: ； in, is the normalization factor corresponding to pixel i, and UL1 is the original high-resolution feature map; According to the similarity weight between each pixel and the neighboring pixels in the original high-resolution feature map and the normalization factor corresponding to each pixel, a non-local operation is performed on each pixel in the original high-resolution feature map to obtain a non-local operation result corresponding to each pixel; For pixel i in the original high-resolution feature map, the formula for calculating the non-local operation result corresponding to pixel i is specifically: ； Among them, HL1 i is the non-local operation result corresponding to pixel i, g(UL1 j ) is a feature vector of neighborhood pixel j; According to the non-local operation results of all pixels, the target high-resolution feature map is obtained.

6. The prescription image recognition method according to claim 2, characterized in that: The CBAM convolutional block attention module includes a channel attention submodule and a spatial attention submodule; The method of using the CBAM convolutional block attention module to perform feature fusion on the low-resolution feature map and the target high-resolution feature map to obtain the target feature map corresponding to the prescription to be identified includes: Performing feature dimension reduction on the target high-resolution feature map according to a preset convolution block, and performing feature splicing on the low-resolution feature map and the target high-resolution feature map after feature dimension reduction to obtain a spliced ​​feature map; The expression of the splicing feature map is specifically: ; Among them, F Concat is the splicing feature map, is the splicing operation function, is a 3×3 convolution operation function, HL1 is the target high-resolution feature map, and L1 is the low-resolution feature map; Inputting the concatenated feature map into the channel attention submodule to generate a channel attention map; fusing the channel attention map and the concatenated feature map to obtain a first output feature map; The expression of the channel attention map is specifically: ； The expression of the first output feature map is specifically: ; in, is the channel attention map, is the first output feature map, is the Sigmoid activation function, , and They are multi-layer perceptron function, global average pooling operation function and global maximum pooling operation function respectively; Inputting the first output feature map into the spatial attention submodule to generate a spatial attention map; fusing the spatial attention map with the first output feature map to obtain a second output feature map; The expression of the spatial attention map is specifically: ； The expression of the second output feature map is specifically: ; in, is the spatial attention map, is the second output feature map, is the 7×7 convolution operation function, Represents the concatenation operation in the channel dimension; The remaining convolutional layers of the ResNet residual network are used to perform feature extraction on the second output feature map to obtain the target feature map corresponding to the prescription to be identified.

7. The prescription image recognition method according to claim 6, characterized in that: After performing feature extraction on the second output feature map, the method further includes: Dividing the second output feature map after feature extraction into a plurality of one-dimensional sequences along the width direction; The bidirectional LSTM model is used to perform forward and backward processing on all one-dimensional sequences to obtain the character probability distribution; The character probability distribution is decoded using a connectivity temporal classification loss function.

8. The prescription image recognition method according to claim 1, characterized in that: The performing image distillation processing on the target feature map to obtain an initial text recognition result includes: Get the initial student network and the pre-trained teacher network; Inputting a preset noise-free prescription picture into the teacher network, adding noise into the noise-free prescription picture to obtain a noisy prescription picture, and inputting the noisy prescription picture into the initial student network; Using a mean square error loss function between a first feature output by the teacher network and a second feature output by the initial student network as a distillation function, distilling the initial student network to obtain a target student network; The expression of the mean square error loss function is specifically: ; Among them, L distill is the mean square error loss function, L4 teacher and L4 student are the first feature output by the teacher network and the second feature output by the initial student network, respectively. is the norm; The target feature map is recognized using the target student network to obtain the initial text recognition result.

9. The prescription image recognition method according to any one of claims 1 to 8, characterized in that: The step of sequentially performing named entity recognition and serialization on the initial text recognition result to obtain the target text recognition result of the prescription to be recognized includes: Predefine the BERT model and obtain the fine-tuned BERT model; Using a fine-tuned BERT model to perform named entity recognition on the initial text recognition result to obtain named entity information of the prescription to be recognized; The named entity information is serialized according to the JSON format to obtain and store the target text recognition result.

10. A prescription image recognition system, characterized in that: Applied to the prescription picture recognition method according to any one of claims 1 to 9, the system comprises: An image acquisition module, used to acquire a prescription image corresponding to the prescription to be identified; A text detection module, used to perform text detection on the prescription image to obtain a text area image; A feature processing module, used for performing feature extraction, feature denoising and feature fusion on the text area image to obtain a target feature map corresponding to the prescription to be identified; A distillation processing module, used to perform image distillation processing on the target feature map to obtain an initial text recognition result; A recognition processing module, used for sequentially performing named entity recognition and serialization on the initial text recognition result to obtain a target text recognition result of the prescription to be recognized; The feature processing module performs feature extraction, feature denoising and feature fusion on the text area image to obtain a target feature map corresponding to the prescription to be identified, specifically including: Using a ResNet residual network, extracting features from the text area image to obtain a low-resolution feature map; Using a super-resolution denoising module, the low-resolution feature map is subjected to feature denoising to obtain a target high-resolution feature map; The CBAM convolutional block attention module is used to perform feature fusion on the low-resolution feature map and the target high-resolution feature map to obtain the target feature map corresponding to the prescription to be identified.

11. A prescription image recognition device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the computer program implements the method steps in the prescription picture recognition method according to any one of claims 1 to 9 when executed.

12. A computer storage medium, characterized in that: The computer storage medium comprises: at least one instruction, and when the instruction is executed by a computer, the method steps in the prescription picture recognition method according to any one of claims 1 to 9 are implemented.

Images