Book and archive management method, system, equipment and medium

Abstract

The invention discloses a book and archive management method, system and device and a medium, and the method comprises the steps: obtaining the image data of the serial number of a photographed book and archive, and obtaining the photographing position information; performing text detection on the picture data to obtain a text region in the picture data; performing text recognition on the text area to obtain file number information of the book; and binding the file number information of the book with the position information, and sending the bound information to a cloud end or a database for storage. Compared with the prior art, the method has the following advantages that the machine vision mode is adopted, the book and archive numbers are automatically recognized, the book and archive numbers and the book position relation are bound, compared with manual recording, the working efficiency is improved, meanwhile, wrong recording caused by human factors can be avoided, and the user experience is improved. Therefore, the accuracy of the book and archive information is ensured.

Description

Technical Field

[0001] The present application relates to the technical field of book and archive management, and in particular to a book and archive management method, system, equipment and medium. Background Art

[0002] With the continuous advancement of information construction, library and archive management is gradually developing in the direction of intelligence. The traditional filing cabinet management model has many inconveniences, such as low efficiency of manual operation and poor security. In recent years, with the development of Internet of Things technology and artificial intelligence, it has become possible to improve the efficiency of archive management through remote control and automatic recognition technology.

[0003] However, most of the existing intelligent archive management methods and systems have single functions and cannot achieve full automated monitoring and management of archive access. Summary of the invention

[0004] The purpose of this application is to provide a book archive management method, system, device and medium to solve the problem that the existing book archive management method and system in the related technology are not intelligent enough and inefficient.

[0005] In order to achieve the above objectives, this application provides the following technical solutions:

[0006] In a first aspect, the present application provides a method for managing book archives, comprising:

[0007] Get the image data of the book file number and the location information of the shooting;

[0008] Performing text detection on the image data to obtain a text area in the image data;

[0009] Performing text recognition on the text area to obtain the file number information of the book;

[0010] The file number information of the book is bound to the location information, and the bound information is sent to the cloud or database for storage.

[0011] Furthermore, the method also includes: regularly obtaining the file number information and corresponding location of the book according to a preset time, and determining whether the location information has changed. If the location information has changed, sending an alarm message to relevant personnel for processing.

[0012] Furthermore, the performing text detection on the image data to obtain the text region in the image data includes: performing text feature extraction on the image data by using a convolution method to obtain the text region in the image data.

[0013] Furthermore, the extracting text features from the image data by convolution to obtain the text area in the image data includes:

[0014] The image data is subjected to multiple convolution processes, and during the convolution process, the image information is subjected to position information enhancement processing and context semantic information enhancement processing, so as to obtain a text detection result.

[0015] Furthermore, the position information enhancement processing specifically adopts the following formula:

[0016]

[0017] in, represents adaptive weight, ⊙ represents Hadamard product, AAP(·) represents adaptive average pooling layer adapted to feature layers of different semantic scales, represents adaptive weight, AMP(·) represents adaptive maximum pooling layer, Conv 1x1 Represents a convolution operation with a kernel size of 1.

[0018] Furthermore, the context semantic information enhancement process specifically adopts the following calculation formula:

[0019] A i (x)=Wx+x,

[0020]

[0021] Among them, A i represents the channel interaction operation, W represents the weight generated by the channel interaction module, Conv DW represents the depthwise convolution operation, Represents the dilated convolution operation, Conv 1x1 Represents a convolution operation with a kernel size of 1.

[0022] Furthermore, the performing text recognition on the text area to obtain the information of the book archive includes:

[0023] Performing visual feature extraction processing on the text area to obtain a character feature sequence;

[0024] Performing sequence feature extraction processing on the character feature sequence to obtain a prediction result of the character feature;

[0025] Performing character escape processing on the prediction result of the character feature to obtain a character label sequence;

[0026] The character label sequences are integrated to obtain text recognition results.

[0027] In a second aspect, the present application also provides a book archive management system, including:

[0028] The shooting module is used to obtain the image data of the book file number and the shooting location information;

[0029] A text detection module, used to perform text detection on the image data to obtain a text area in the image data;

[0030] A text recognition module, used to perform text recognition on the text area to obtain the file number information of the book;

[0031] The information saving module is used to bind the file number information of the book with the location information, and send the bound information to the cloud or database for storage.

[0032] In a third aspect, the present application also provides a computer electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above-mentioned methods for managing book archives when executing the computer program.

[0033] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the book archive management method described in any one of the above are implemented.

[0034] The present application provides a method, system, device and medium for managing book archives. The method obtains the image data of the book archive number and the location information of the shooting; performs text detection on the image data to obtain the text area in the image data; performs text recognition on the text area to obtain the book archive number information; binds the book archive number information with the location information, and sends the bound information to the cloud or database for storage. Compared with the related art, the present application has the following effects:

[0035] 1. Improve operational efficiency: realize remote control, automatic identification and remote inventory.

[0036] 2. Enhanced security: A secure communication method, such as RS485 communication protocol, is used between the filing cabinet terminal and the identification terminal, and the system can detect abnormal behavior in a timely manner by comparing the previous and subsequent identification results, thereby reducing the risk of file loss or misplacement.

[0037] 3. Improve accuracy: The automatic recognition of file codes by cameras avoids erroneous records caused by human factors and ensures the accuracy of file information. At the same time, the system saves opening and closing records, personnel entry and exit records, file access records, etc., which is convenient for subsequent auditing and tracing.

[0038] 4. Realize intelligent management: Through network communication protocols, such as HTTP protocol, the file access information is sent to the server in real time, supporting remote management and data analysis, and providing decision support for management.

[0039] 5. Optimize user experience: Simplify the file management process through automated processes, reduce the workload of staff, and improve overall work efficiency.

[0040] 6. Reduce operating costs: The automation system reduces dependence on manual operations and reduces labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of a method for managing book archives in an embodiment of the present application;

[0042] Figure 2 is a schematic diagram of a text detection algorithm framework in an embodiment of the present application;

[0043] Figure 3 is a schematic diagram of a location information enhancement module in an embodiment of the present application;

[0044] Figure 4 is a schematic diagram of a contextual semantic information enhancement module in an embodiment of the present application;

[0045] Figure 5 is a schematic diagram of channel interaction branches in an embodiment of the present application;

[0046] Figure 6 It is a flowchart of a text recognition algorithm in an embodiment of the present application;

[0047] Figure 7 is a schematic diagram of a visual feature extraction module in an embodiment of the present application;

[0048] Figure 8 is a schematic diagram of a character feature sequence in an embodiment of the present application;

[0049] Fig. 9 is a schematic diagram of a sequence feature extraction module in an embodiment of the present application;

[0050] Fig.10 is a schematic diagram of a character escape module in an embodiment of the present application;

[0051] Fig.11 is a schematic diagram of a character decoding enhancement module in an embodiment of the present application;

[0052] Fig.12 It is a structural diagram of a book archive management system according to an embodiment of the present application;

[0053] Fig.13 It is a structural schematic diagram of a computer electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. In contrast, when an element is referred to as being "directly on" another element, there is no intermediate element. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0056] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0057] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0058] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms "a", "said" and "the" used in one or more embodiments of the present application are also intended to include plural forms, unless the context clearly indicates other meanings.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of the template are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used herein includes any and all combinations of one or more of the associated listed items.

[0060] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when the time of".

[0061] The following is a brief introduction to the application scenarios of this application.

[0062] Generally speaking, in order to facilitate the management of books, a file number will be assigned to the books. This file number is a code used to identify items. It is numbered using letters and numbers according to prescribed rules to form a unique code for the file, called a file identification code.

[0063] After the books are numbered, they need to be put on the shelves, that is, put into the corresponding filing cabinets. Filing cabinets are physical storage devices for storing paper or other forms of files, and can control the closing, opening or moving of filing cabinets. There may be multiple faces in the filing cabinet, multiple layers in each face, and multiple node cabinets in each layer.

[0064] For places that need to store a large number of books, such as libraries, multiple filing cabinets are generally required. These filing cabinets will be placed in different places. Therefore, these filing cabinets need to be set with library numbers, area codes, and column numbers. This information is used to identify the overall filing cabinet coordinate information, the warehouse to which the filing cabinet belongs, the area, and the column number.

[0065] At present, the existing book and archive management methods generally require manual recording of the book and archive numbers in different filing cabinets, or use RFID to record, and then summarize and save the information. However, the above methods will have the following problems:

[0066] 1. Manual registration may result in errors and omissions, and is time-consuming and labor-intensive.

[0067] 2. RFID requires the purchase of a large number of RFID tags, which is costly.

[0068] 3. RFID cannot accurately locate the location of files.

[0069] 4. RFID cannot record pictures of the scene.

[0070] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes are not repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0071] Please refer to Figure 1 The present application provides a method for managing book archives, which comprises at least the following steps:

[0072] S10, obtaining the image data of the book file number and obtaining the location information of the shooting.

[0073] Specifically, in this embodiment, by installing a plurality of camera devices in the filing cabinet, the camera devices can capture the file number information of the books.

[0074] It should be noted that, while photographing the book archive information, it is also necessary to obtain the photographed location information, that is, the location information of the book in the archive cabinet.

[0075] S20: Perform text detection on the image data to obtain a text area in the image data.

[0076] Specifically, in this embodiment, after obtaining the image data with the book file number, it is necessary to detect the text data in the image to obtain the corresponding text area in the image data.

[0077] In one embodiment of the present application, the step S20 of performing text detection on the image data to obtain a text area in the image data includes:

[0078] The text features are extracted from the image data by using a convolution method to obtain the text area in the image data.

[0079] Specifically, the image data is subjected to multiple convolution processes, and during the convolution process, the image information is subjected to position information enhancement processing and context semantic information enhancement processing, so as to obtain the text detection result.

[0080] The following example uses 8 convolutions on the image data to illustrate the process:

[0081] Please refer to Figure 2, input a three-channel text image to be detected, the image height is recorded as H, the width is recorded as W, then the input image can be expressed as (H, W, 3), first through a downsampling operation consisting of 3x3 convolution to obtain the feature map F1 (H / 2, W / 2, 6), then F1 through the same downsampling operation to obtain the feature map F2 (H / 4, W / 4, 12), F2 through the same downsampling operation to obtain the feature map F4 (H / 8, W / 8, 24), F4 through the same downsampling operation to obtain the feature map F6 (H / 16, W / 16, 48), F6 through the downsampling operation to obtain the feature map F8 (H / 32, W / 32, 96), so far, the input image through the downsampling operation to extract the text area features of the process has been completed. Feature map F2 retains the position information in feature map F1 through the PIEM1 module, and integrates and enhances the position information in feature maps F1 and F2, and then undergoes two upsampling operations consisting of 3x3 deconvolution to obtain feature map F3. Feature map F4 retains the position information in feature map F2 through the PIEM2 module, and integrates and enhances the position information in feature maps F2 and F4, and then undergoes two upsampling operations consisting of 3x3 deconvolution to obtain feature map F5. Feature map F6 retains the position information in feature map F4 through the PIEM3 module, and integrates and enhances the position information in feature maps F4 and F6, and then undergoes two upsampling operations consisting of 3x3 deconvolution to obtain feature map F7. Then feature map F8 retains the position information in feature map F6 through the PIEM4 module, and integrates and enhances the position information in feature maps F6 and F8, and then undergoes two upsampling operations consisting of 3x3 deconvolution to add element by element with feature map F6, so that feature map F7 is supplemented with some feature information of F6. Feature map F7 is added element-by-element to feature map F4 after two upsampling operations consisting of 3x3 deconvolution, and part of the information of feature map F4 is retained for feature map F5. Feature map F5 is added element-by-element to feature map F2 after two upsampling operations consisting of 3x3 deconvolution, and part of the feature map information of F2 is retained for feature map F3.After a 1x1 convolution operation, the feature map F3 fully expands the network's receptive field in the channel dimension through the CSEM1 module to better integrate the spatial position information of the feature maps F3 and F5 learned by the lower network. After two upsampling operations consisting of 3x3 deconvolutions, the feature map F5 is integrated with the spatial position information of F5 and F7 learned by the lower network in the channel dimension through the CSEM2 module. After four upsampling operations consisting of 3x3 deconvolutions, the feature map F7 is integrated with the spatial position information of F7 and F8 learned by the lower network in the channel dimension through the CSEM3 module. After eight upsampling operations consisting of 3x3 deconvolutions, the feature map F8 is integrated with the spatial position information learned by the lower network in the channel dimension through the CSEM4 module. These feature maps with enhanced spatial position information interaction through the CSEM module are concatenated according to the channel dimension to obtain the final feature map F9. According to the feature map F9, the text region segmentation result and the text region thresholding result are obtained by reasoning. Since the standard thresholding operation sets a fixed threshold, any pixel value in the image that is greater than the threshold is judged as the text part, otherwise it is judged as the non-text part. However, this operation is not differentiable in the mathematical sense, and the deep learning model training process involves the operation of gradient descent to converge the loss function, so the standard thresholding operation needs to be modified to make it differentiable, so that the model training can proceed normally. Therefore, the obtained text region segmentation result and text region thresholding result are subjected to differentiable binarization operation to obtain the text region binarization result, and finally the final text detection result is obtained through the prediction box correction operation.

[0082] Standard thresholding operation:

[0083]

[0084] Among them, B i，j Represents the threshold value of the image at pixel (i, j), P i，j It represents the pixel value of the image at the pixel point (i, j), and t represents the threshold value.

[0085] Differentiable binarization operation:

[0086]

[0087] in, is the differentiable binary value of the image at pixel (i, j), T is the adaptive threshold learned from the text detection network, and k is the magnification factor, which can be set to 50 based on experience.

[0088] It should be noted that, in the downsampling process of the text detection network, convolution operations are performed using convolution kernels of different sizes to extract text region features of different resolutions, which allows the network to better extract the semantic information of the text image, but it is inevitable that the location information of the text image will be lost in the shallow network, causing the high-level network to lose its positioning function. In the file number detection scenario, since the file boxes are often densely placed, the detection network needs to have a strong ability to extract location information. In order to compensate for the location information lost by the shallow network, in this example, please refer to Figure 3 In the convolution process, the position information enhancement module is introduced to enhance the position information. The specific processing formula is as follows:

[0089]

[0090] in, represents the adaptive weight, ⊙ represents the Hadamard product, and AAP(·) represents the adaptive average pooling layer that adapts to different semantic scale feature layers {F2, F4, F6, F8}.

[0091]

[0092] in, represents adaptive weights, and AMP(·) represents an adaptive maximum pooling layer.

[0093]

[0094] Among them, Conv 1x1 Represents a convolution operation with a kernel size of 1.

[0095] In this embodiment, the position information enhancement module takes the semantic features F2, F4, F6, and F8 of different resolutions extracted from the text detection shallow network as input, and calculates the restrictive features and position features through the saliency pooling layer and the position pooling layer. Both features contain the position and edge information of the file number. After adding these two feature map information element by element, they are merged into the subsequent detection network through 1x1 convolution, retaining the detection network's ability to extract position information at the shallow layer.

[0096] It should be further explained that in order to further improve the situation of channel information loss caused by frequent channel processing operations during the upsampling process of the text detection network, and the loss of semantic information caused by aliasing of pixels at the file box number due to the direct fusion feature processing method after interpolation during the upsampling process, in this embodiment, please refer to Figure 4, a contextual semantic information enhancement module (CSEM) is designed to extract spatial information and channel information respectively, and then the two kinds of information are fused to improve the interaction of spatial and channel information during the network sampling process, avoiding the situation where the network cannot extract effective text features due to the loss of too much channel information and semantic information.

[0097] The contextual semantic information enhancement module processes spatial information and channel information separately. In the spatial interaction branch, the present invention adopts depthwise separable convolution and dilated convolution to replace the 1x1 convolution operation used in the conventional upsampling process. Depthwise separable convolution is used to interact with local spatial information in each channel, and dilated convolution is used to interact with global spatial information in each channel, so as to further improve the receptive field size of the network during the upsampling process and strengthen the extraction of global information in spatial information.

[0098] Please refer to Figure 5 In the channel interaction branch, this method gives priority to the interaction of information between channels. In order to transfer key feature information between channels to promote the overall performance of the network during upsampling, this method adds a weight-level priority processing method between channels, and adaptively transfers the feature information extracted by the network according to the weight size of different channels. The global average pooling layer and the global maximum pooling layer are used to save the global information and key features of each channel respectively. At the same time, the fully connected network FC1 is used to learn the shared weights of these two types of information, and the weights are prioritized by the ReLU activation function and the results are transferred to the fully connected network FC2 by channel splicing. In order to retain the complete channel information, the weight information of the fully connected network FC2 is superimposed with the original channel information after Sigmoid normalization through the residual connection. The specific calculation formula is as follows:

[0099] A i (x)=Wx+x,i=3,5,7,8,

[0100] Among them, A i represents the channel interaction operation, and W represents the weight generated by the channel interaction module.

[0101]

[0102] Among them, Conv DW represents the depthwise convolution operation, Represents a dilated convolution operation.

[0103]

[0104] Among them, Conv 1x1 Represents a convolution operation with a kernel size of 1.

[0105] S30, performing text recognition on the text area to obtain the file number information of the book.

[0106] Specifically, the text recognition of the file number information is implemented based on the file number text detection. First, the text detection algorithm proposed in this method is used to accurately locate the position coordinates of the label where the file number is located in each collected file image (upper left vertex, upper right vertex, lower left vertex, lower right vertex). According to the coordinates, the rectangular frame area where the file number is located can be drawn in the original collected image. The image of this area is cropped through conventional image processing methods to obtain the input of the text recognition algorithm, that is, the text image to be recognized.

[0107] In one embodiment of the present application, please refer to Figure 6 The step S30, performing text recognition on the text area to obtain the file number information of the book, includes:

[0108] S301: Perform visual feature extraction processing on the text area to obtain a character feature sequence.

[0109] S302: Perform sequence feature extraction processing on the character feature sequence to obtain a prediction result of the character feature.

[0110] S303: Perform character escape processing on the prediction result of the character feature to obtain a character label sequence.

[0111] S304: Integrate the character tag sequences to obtain text recognition results.

[0112] It should be noted that the above-mentioned visual feature extraction, sequence feature extraction and character escape are respectively completed by the corresponding visual feature extraction module, sequence feature extraction module and character escape module.

[0113] Specifically, a three-channel text image to be recognized is input, the image height is recorded as H, and the width is recorded as W, then the text image can be expressed as (H, W, 3). First, before the text image is input into the visual feature extraction module, the text image needs to be scaled first, because the text recognition algorithm of the present invention is to perform single-line recognition on text of any length, so it is necessary to ensure that the height of all input text images remains consistent, and the image width can be arbitrary. After the image preprocessing operation, the image is input into the visual feature extraction module, and the convolution layer, maximum pooling layer and average pooling layer contained in the visual feature extraction module extract the character features from low to high level in the text image layer by layer to obtain a character feature sequence, and then the character feature sequence is input into the sequence feature extraction module, and the deep bidirectional long short-term memory network predicts the character features of each frame in the feature sequence, and finally the character feature sequence predicted by the deep bidirectional long short-term memory network is translated by the character escape module to obtain a character label sequence, and finally the label sequence is integrated to obtain the final text recognition result.

[0114] See also Figure 7 For the visual extraction module, the image tensor size (64, 16, 3) obtained after the preprocessing operation of the text image to be recognized is first subjected to two layers of convolution operations with a convolution kernel size of 3x3 to preliminarily extract the shallow features of the image, and the obtained feature map size is (64, 16, 256), and then the maximum pooling layer with a convolution kernel size of 2x2 is used to filter unnecessary text features to obtain a feature map size of (65, 8, 256), and then the feature map size is (65, 8, 512) obtained after 5 layers of convolution operations with a convolution kernel size of 3x3, and then the feature map size is (65, 8, 512) obtained after 3 layers of convolution operations with a convolution kernel size of 2x2, and the feature map size is (65, 4, 512) obtained after one layer of convolution operation with a convolution kernel size of 2x2, and finally the feature map size is (65, 3, 512) obtained after the average pooling layer. The purpose of this module is to fully extract the character features in the text image. Therefore, in order to reduce the size of the model parameters, the fully connected layer in the convolutional neural network is removed. The convolutional layer, maximum pooling layer and average pooling layer in the convolutional neural network are mainly used to extract the character feature map. For a schematic diagram of the character features, please refer to Figure 8 , and then convert the character feature map from left to right into the corresponding character feature sequence by column, that is, each character feature sequence corresponds to a rectangular box area in the input image.

[0115] Since simply using a convolutional neural network to extract text character features to handle text recognition problems cannot solve the situation where the text length changes, in this embodiment, the deep character features extracted by the convolutional neural network are converted into a sequence representation so that it is not affected by changes in the length of sequence characters.

[0116] See also Fig. 9 , for the sequence feature extraction module, the module uses a bidirectional long short-term memory network to predict each frame in the feature sequence, and outputs the corresponding character feature sequence. The present invention uses the long short-term memory network to have three advantages: first, the long short-term memory network has a strong ability to capture contextual information in a sequence. For image-based sequence recognition problems, using contextual information is a more stable and effective method than processing each character individually, especially for processing some ambiguous characters, such as the number '1' and lowercase 'l', the number '0' and lowercase 'O'; secondly, the long short-term memory network can back-propagate the error differential loss to the input end, and the convolutional neural network and the long short-term memory network can be trained together; finally, the long short-term memory network can process sequences of any length from beginning to end. However, the traditional LSTM network is only unidirectional, that is, it can only use past context information. But in the image-based series, both past and future context information are very helpful for character recognition. Therefore, in this embodiment, a bidirectional LSTM network is adopted, and multiple bidirectional LSTM networks are stacked to form a deep bidirectional LSTM network to further enhance the capture of long-distance dependencies between characters, so as to facilitate the subsequent escape operation of the recognized character feature sequence.

[0117] See also Fig.10 For the character escape module, in the process of file management, the file management personnel will formulate different numbering rules according to the attributes of the file to uniquely identify the file. Although the rules are different, the numbers of the same type of files generally start from the number 0, and the same ones are accumulated. If the length defined by the numbering rule is too long, multiple digits will be the same. The existing text recognition algorithm has a poor effect on the escape of multiple consecutive identical character features in the text during the escape stage, which is specifically manifested in that a certain character feature is easily lost, resulting in the final recognized text length being less than the actual text length. To alleviate this situation, in this embodiment, a character decoding enhancement module is added to the character escape module. The module converts the character feature sequence predicted by the long short-term memory network for each frame into a set of character sequence features by exchanging dimensions as the input of the character decoding enhancement module. The enhanced character sequence features are obtained through the character decoding enhancement module, and the enhanced character sequence features are flattened. Through the fully connected layer, a prediction sequence result closest to the actual label sequence in the predicted label sequence is found. Generally speaking, there are two situations, with a dictionary and without a dictionary: if the test set has a dictionary, then the output result is to calculate the probabilities of all dictionaries and select the corresponding character with the largest probability as the final prediction result; if the test set does not have a dictionary, that is, the test set does not give the specific characters it contains, then the character corresponding to the prediction result with the largest probability is selected as the final prediction result.

[0118] See also Fig.11 For the character decoding enhancement module, considering that the semantic information of the file number is a combination of English and Arabic numerals, and the extraction of the file number requires the text detection algorithm of the present invention to have sufficient positioning accuracy, in order to avoid omissions in the text detection algorithm, this method adds a layer of character position information screening in the character decoding process to filter out the interference of the Chinese file name that may exist in the file number area. The character decoding enhancement module takes the character feature sequence as input, first enhances the position information feature of the character through the position decoder, outputs the position information Pi of the character, and then parses the semantics of each character according to the position information through the semantic decoder, and outputs the semantic value Ci. The position decoder only generates a position label that identifies the character feature. When the position decoder recognizes a new character, it triggers the semantic decoder, and uses the hidden state of the position decoder to calculate the attention information contained in the recognized character content. This ensures that the position label generated by the position decoder corresponds to the semantic sequence generated by the semantic decoder. The outputs of the two decoders are merged to obtain the final text recognition content.

[0119] It should be noted that the recurrent neural network units in the position decoder and the semantic decoder generate new semantic labels in the current time step based on the context vector, the hidden state of the previous unit, and the previously generated semantic labels in each time step t. That is, it is used to represent the generation process of new semantic labels. To represent the generation process of new semantic tags.

[0120]

[0121] c t =f t ⊙c t-1 +i t ⊙g t ,

[0122] h t =o t ⊙tanh(c t ),

[0123] Among them, i t , f t , o t , c t ,h t They are the input gate, forget gate, output gate, memory unit and hidden state in the recurrent neural network unit, which are used to control the flow, update and generation of information in the entire unit. is a context vector that captures the semantic information related to a specific input position. E represents an embedding matrix used to map the input semantic label to the embedding space. m and n represent the dimensions of the embedding matrix and the recurrent neural network unit, respectively. σ represents the logical sigmoid activation function, and ⊙ represents element-by-element multiplication.

[0124] The attention units in the position decoder and semantic decoder calculate the context vector based on the feature vectors extracted from different semantic positions in the character feature sequence. For each position, the attention unit generates a positive weight, and then evaluates the relevance and importance of the generated positive weight for generating the next semantic label based on the recurrent neural network unit at the previous moment. If the relevance and importance are high, this weight will then be used to calculate the context vector.

[0125]

[0126] Among them, φ represents a and its corresponding positive weight α i Calculate the context vector function.

[0127] S40, binding the file number information of the book with the location information, and sending the bound information to the cloud or database for storage.

[0128] Specifically, after obtaining the file number signal of the book, it is necessary to bind the file number signal and the location information of the book in the filing cabinet, and send the bound information to the cloud or data for storage to facilitate subsequent management and use.

[0129] The present application provides a method, system, device and medium for managing book archives. The method obtains the image data of the book archive number and the location information of the shooting; performs text detection on the image data to obtain the text area in the image data; performs text recognition on the text area to obtain the book archive number information; binds the book archive number information with the location information, and sends the bound information to the cloud or database for storage. Compared with the related art, the present application has the following effects:

[0130] 1. Improve operational efficiency: realize remote control, automatic identification and remote inventory.

[0131] 2. Enhanced security: The filing cabinet terminal and the identification terminal use secure communication methods, such as RS485 communication protocol, and the system can detect abnormal behavior in time by comparing the previous and subsequent identification results, reducing the risk of file loss or misplacement.

[0132] 3. Improve accuracy: The automatic recognition of file codes by cameras avoids erroneous records caused by human factors and ensures the accuracy of file information. At the same time, the system saves opening and closing records, personnel entry and exit records, file access records, etc., which is convenient for subsequent auditing and tracing.

[0133] 4. Realize intelligent management: Through network communication protocols, such as HTTP protocol, the file access information is sent to the server in real time, supporting remote management and data analysis, and providing decision support for management.

[0134] 5. Optimize user experience: Simplify the file management process through automated processes, reduce the workload of staff, and improve overall work efficiency.

[0135] 6. Reduce operating costs: The automation system reduces dependence on manual operations and reduces labor costs.

[0136] In one embodiment of the present application, the method further includes: S50, regularly obtaining the file number information and corresponding location of the book according to a preset time, and determining whether the location information has changed. If the location information has changed, sending an alarm message to relevant personnel for processing.

[0137] It is understandable that after a book is put on the shelf in a filing cabinet, its location is generally not easily changed, which is to facilitate the management of the book. In order to prevent the book from being misplaced in different places later, in this embodiment, the file number information and location information of the book are obtained regularly. If it is determined that the location information of the book has changed, an alarm message is sent to the relevant personnel for processing.

[0138] See also Fig.12 The present application also provides a book archive management system 200, including:

[0139] The shooting module 201 is used to obtain the picture data of the book file number and obtain the shooting location information;

[0140] A text detection module 202, used to perform text detection on the image data to obtain a text area in the image data;

[0141] The text recognition module 203 is used to perform text recognition on the text area to obtain the file number information of the book;

[0142] The information saving module 204 is used to bind the file number information of the book with the location information, and send the bound information to the cloud or database for storage.

[0143] See also Fig.13The embodiment of the present application also provides a computer electronic device 300, including a memory 303 and a processor 302, wherein the memory 303 stores a computer program, and when the processor executes the computer program, the steps of the book archive management method described in any one of the above items are implemented.

[0144] Specifically, the electronic device 300 includes: a transceiver 301, a bus interface and a processor 302, wherein the processor 302 is used to obtain the image data of the photographed book file number and obtain the shooting location information; perform text detection on the image data to obtain the text area in the image data; perform text recognition on the text area to obtain the file number information of the book; bind the file number information of the book with the location information, and send the bound information to the cloud or database for storage.

[0145] In the embodiment of the present application, the electronic device 300 further includes: a memory 303. Fig.13 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors represented by processor 302 and memory represented by memory 303. The bus architecture may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 301 may be a plurality of components, i.e., including a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. The processor 302 is responsible for managing the bus architecture and general processing, and the memory 303 may store data used by the processor 302 when performing operations.

[0146] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the book archive management method described in any one of the above are implemented.

[0147] In this embodiment, the computer readable storage medium may be a non-volatile storage medium or a volatile storage medium. For example, the computer storage medium may include but is not limited to: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0148] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.

[0149] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0150] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow diagram, and the combination of boxes in the structure diagram and / or the flow diagram, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0151] In addition, the functional modules or units in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0152] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a terminal device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application.

[0153] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A method for managing book archives, characterized in that: include: Get the image data of the book file number and the location information of the shooting; Performing text detection on the image data to obtain a text area in the image data; Performing text recognition on the text area to obtain the file number information of the book; The file number information of the book is bound to the location information, and the bound information is sent to the cloud or database for storage.

2. The method for managing book archives according to claim 1, characterized in that: The method further comprises: According to the preset time, the file number information and the corresponding location of the book are obtained regularly, and it is determined whether the location information has changed. If the location information has changed, an alarm message is sent to relevant personnel for processing.

3. The method for managing book archives according to claim 1, characterized in that: The performing text detection on the image data to obtain a text area in the image data includes: The text features are extracted from the image data by using a convolution method to obtain the text area in the image data.

4. The method for managing book archives according to claim 3, characterized in that: The extracting text features from the image data by using a convolution method to obtain a text area in the image data includes: The image data is subjected to multiple convolution processes, and during the convolution process, the image information is subjected to position information enhancement processing and context semantic information enhancement processing, so as to obtain a text detection result.

5. The method for managing book archives according to claim 4, characterized in that: The position information enhancement processing specifically adopts the following formula: in, represents adaptive weight, ⊙ represents Hadamard product, AAP(·) represents adaptive average pooling layer adapted to feature layers of different semantic scales, represents adaptive weight, AMP(·) represents adaptive maximum pooling layer, Conv 1x1 Represents a convolution operation with a kernel size of 1.

6. The method for managing book archives according to claim 4, characterized in that: The contextual semantic information enhancement process specifically adopts the following calculation formula: A i (x)=Wx+x, Among them, A i represents the channel interaction operation, W represents the weight generated by the channel interaction module, Conv DW represents the depthwise convolution operation, Represents the dilated convolution operation, Conv 1x1 Represents a convolution operation with a kernel size of 1.

7. The method for managing book archives according to claim 1, characterized in that: The step of performing text recognition on the text area to obtain information of the book archive includes: Performing visual feature extraction processing on the text area to obtain a character feature sequence; Performing sequence feature extraction processing on the character feature sequence to obtain a prediction result of the character feature; Performing character escape processing on the prediction result of the character feature to obtain a character label sequence; The character label sequences are integrated to obtain text recognition results.

8. A book archive management system, characterized in that: include: The shooting module is used to obtain the image data of the book file number and the shooting location information; A text detection module, used to perform text detection on the image data to obtain a text area in the image data; A text recognition module, used to perform text recognition on the text area to obtain the file number information of the book; The information saving module is used to bind the file number information of the book with the location information, and send the bound information to the cloud or database for storage.

9. A computer electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method for managing book archives described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the book archive management method described in any one of claims 1 to 7 are implemented.

Images