Intelligent library book checking system based on deep learning
By combining deep learning and OCR technology, real-time location and orderly cropping of library books have been achieved, solving the problems of low accuracy and high cost of existing RFID inventory methods, and providing an efficient solution for book information retrieval and inventory.
Patent Information
- Application Number
- CN202411061534.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-05
- Publication Date
- 2024-11-29
AI Technical Summary
Existing RFID-based library book inventory methods suffer from high maintenance costs, low recognition accuracy, complex robot design, and the impact of ambient lighting and book arrangement on recognition accuracy. Furthermore, they lack multi-technology integration and optimization.
A deep learning-based intelligent inventory system for library books is adopted. It uses the YOLOv9 network for object detection and combines it with the PaddleOCR framework for OCR recognition. Through data augmentation, cropping and sorting, and multiple recognition optimizations, a cloud platform is built to achieve real-time spine detection and segmentation, and image processing is performed under the Flask framework.
It enables real-time location detection and orderly cropping of library books, improves the accuracy and efficiency of book title recognition, supports real-time access to and inventory of book information, and reduces maintenance costs.
Smart Images

Figure CN119049062A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of library management, and in particular relates to a library book intelligent inventory system based on deep learning. Background Art
[0002] With the continuous advancement of artificial intelligence technology, traditional libraries are gradually transforming into smart libraries. In this transformation process, although RFID (radio frequency identification) technology is one of the mainstream technologies in the traditional book management model, it has some limitations in practical applications, such as time-consuming operation, high cost, and relatively high recognition error rate. In order to overcome these challenges, the use of robotic technology for book photography, spine positioning, and identification matching has gradually become a trend in the construction of smart libraries. Intelligent book access and inventory technology is the key to promoting the intelligent development of libraries.
[0003] At present, the inventory method based on RFID technology is still the mainstream, which obtains book information by scanning RFID tags manually or by robots. However, this method is highly dependent on RFID tags, which not only increases maintenance costs, but also in practical applications, the inventory accuracy of RFID robots is often lower than that of vision-based inventory methods. In addition, RFID-based scanning requires the robot to be close to the book during operation, which brings additional challenges in obstacle avoidance and adapting to small spaces, making the robot's size design and path planning more complicated.
[0004] In order to solve the challenges faced by traditional RFID inventory counting methods, intelligent inventory counting technology based on computer vision is being actively studied and optimized. Deep learning technology plays an important role in book spine detection and segmentation algorithms, while OCR (optical character recognition) technology shows its potential in book title matching algorithms. These technologies are constantly being proposed, tested, and improved in order to achieve higher automation and accuracy. However, the intelligent inventory counting algorithm that combines these two technologies is still in the exploratory stage. At present, although object detection and OCR recognition technologies have made certain progress, they still have certain limitations in practical applications. The following are several major factors that affect the further improvement of the accuracy of these technologies:
[0005] Complexity of book arrangement: Books in libraries are densely arranged and may be tilted and repeated, which will interfere with the effectiveness of the target detection algorithm.
[0006] Impact of ambient light and image characteristics: The lighting conditions in the library and the slender characteristics of the spine image may reduce the OCR recognition rate, resulting in low accuracy of book title matching.
[0007] Insufficient technology integration: Existing algorithms often rely on a single technology and lack attempts to combine multiple technologies for optimization. Summary of the invention
[0008] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a library book intelligent inventory system based on deep learning.
[0009] In order to achieve the purpose of the present invention, the present invention will be implemented by adopting the technical solution described below.
[0010] A deep learning-based intelligent inventory method for library books includes the following steps:
[0011] S100, create a data set and divide it into a training set, a validation set, and a test set in a ratio of 7:1:2;
[0012] S200, build a target detection network model through the YOLOv9 network model. The YOLOv9 network model is composed of input end, Backbone, Neck and Prediction, among which:
[0013] At the input end, the input image is preprocessed, wherein the preprocessing is to use an adaptive image scaling method to scale the input image to the same size as the input required by the network, and then perform a normalization operation, and use adaptive anchor frame calculation to generate a corresponding prediction frame based on the initial anchor frame when starting training;
[0014] Backbone, used to extract image features;
[0015] Neck, used to combine and mix image features and transmit image features to the prediction layer;
[0016] Prediction: predict image features, generate bounding boxes and predict the category of the target;
[0017] S300, using SGD as the optimizer for target detection network training, the batch size is 4, the initial learning rate is 0.01, the momentum is set to 0.9, the training round is set to 1000, and the loss function is CIoU Loss or DFL Loss. The target detection network model is trained, verified and tested through the training set, validation set and test set, wherein the training set is used for training the target detection network model; the validation set tests and saves the optimal weight of the target detection network model; the test set identifies the generalization effect of the target detection network model;
[0018] It is characterized in that the library book intelligent inventory method also includes the following steps:
[0019] S400, using the prediction frame generated in step S200, the book spines in each image are framed one by one, and the book spines framed in each image are cropped by a cropping and sorting module, and are sorted in order from top to bottom and from left to right to obtain an ordered book spine image;
[0020] S500, OCR recognition and matching method, comprising the following steps:
[0021] S501, performing data enhancement on the ordered spine image obtained in step S400;
[0022] S502, performing affine transformation on the book spine image to correct the tilted book spine image;
[0023] S503, performing pixel filling on the spine image so that the relative distance between characters becomes smaller;
[0024] S504, using the ultra-lightweight model and OCR recognition model provided by the PaddleOCR framework to perform text recognition on the pixel-filled book spine image, and perform data cleaning to obtain the detection result of the OCR recognition model, match the detection result with the titles of all books in the library, and return the book title information with the highest similarity to the detection result as the book title corresponding to the book spine image;
[0025] S600, an interface program developed based on the Flask framework, writes the target detection method and OCR recognition and matching method into the business logic to build a cloud platform. The cloud platform can detect and segment the book spine after the image is uploaded in real time, and recognize and match it one by one, and finally return the recognition results to the UI interface for users or staff to check and count;
[0026] in:
[0027] The Backbone includes GELAN and PGI, wherein the GELAN is constructed by introducing the CSPNet structure on the basis of ELAN, and the CSPNet structure enables GELAN to use any computing block as a basic module; the PGI generates reliable gradients and updates network parameters by assisting reversible branches;
[0028] The Neck adopts a PaFPN structure and uses an SPP module and a PAN module;
[0029] The Prediction includes the Bounding box loss function and NMS non-maximum suppression, wherein the Bounding box loss function uses CIoU_Loss and DFL Loss as loss functions;
[0030] As a preferred embodiment of the present invention, the method for preparing the data set comprises the following steps:
[0031] S101. Collect images of bookshelves in different areas of the library according to unified standard requirements;
[0032] S102, using labelimg software to label a single book spine and the entirety of adjacent book spines in the acquired bookshelf image.
[0033] As a preferred solution of the present invention, the specific steps of step S502 are as follows:
[0034] S5021, converting the initial image into a grayscale image, and detecting the edge of the image using the Canny edge detection algorithm;
[0035] S5022, perform Hough straight line transformation on the edge, find the straight line in the image, and save the relevant information;
[0036] S5023, cutting the detected edge of the book spine;
[0037] S5024, filling and stretching the cut image to a vertical spine state.
[0038] As a preferred solution of the present invention, the specific steps of step S504 are as follows:
[0039] S5041. Use the ultra-lightweight model to perform a first recognition on the pixel-filled book spine image, and perform data cleaning on the recognition result;
[0040] S5042: Calculate the similarity between the recognition result after data cleaning and the titles of all books in the library, wherein the similarity calculation formula is as follows:
[0041]
[0042] Among them, str1 represents the recognition result after cleaning, str2 represents the book title in the library, length1 represents the character length of str1, length2 represents the character length of str2, num represents the number of characters in the field that are the same as those in the book title, and LevenshteinDistance represents the edit distance algorithm, which is used to measure the similarity between two characters;
[0043] S5043, taking the book title with the highest similarity as the matching result, if the highest similarity is greater than 0.3, returning the matching result, otherwise proceeding to step S5044 for a second recognition;
[0044] S5044, using the OCR recognition model to perform a second recognition on the pixel-filled book spine image, and using the similarity calculation formula to calculate the similarity between the recognition result of the second recognition and the titles of all books in the library, and taking the title with the highest similarity as the matching result, if the highest similarity is greater than 0.3, returning the matching result, otherwise proceeding to step S5045 for the third recognition;
[0045] S5045. After performing affine transformation on the pixel-filled book spine image, use the OCR recognition model to perform a third recognition. Use the similarity calculation formula to calculate the similarity between the recognition result of the third recognition and the titles of all books in the library. Use the title with the highest similarity as the matching result. If the highest similarity is greater than 0.3, return the matching result. Otherwise, return "0", indicating that the title recognition of the book spine photo failed.
[0046] As a preferred solution of the present invention, the specific logic of the business logic is as follows:
[0047] S601, receiving a request sent by the front end, the request including the storage address and database number of the image to be processed in the server;
[0048] S602, obtaining the image to be searched from the request;
[0049] S603, calling the target detection method, performing book spine detection on the image in step S602, and saving the coordinate information [x, y, w, h] of the detection box corresponding to each book spine after detection, where x and y represent the horizontal coordinate and vertical coordinate of the center point of the book spine, respectively, and w and h represent the width and height of the book spine, respectively;
[0050] S604, calling a cutting and sorting module to cut out the book spines in the image according to the saved coordinate information and save them in order;
[0051] S605, calling the OCR recognition and matching method, performing OCR recognition on the saved book spine images in sequence, and performing similarity calculation on the book titles in the library to obtain recognition results;
[0052] S606: Return the recognition result to the cloud platform front end for result display.
[0053] As a preferred embodiment of the present invention, the target detection method is to input a given input bookshelf image into a trained yolov9 target detection network model, generate a detection frame for the spine target, and save the coordinates of the detection frame, so as to crop the framed spine through a cropping and sorting module, segment the corresponding spine target, and obtain an orderly arranged spine image.
[0054] As a preferred embodiment of the present invention, the UI interface includes three user-selectable functions: a selection library, a task name, and an attachment upload.
[0055] As a preferred embodiment of the present invention, the data cleaning is to mask special characters in the field, and the special characters include but are not limited to "著", "(", ")", ",", ":".
[0056] As a preferred embodiment of the present invention, the pixel filling is to fill the left and right sides of the spine image with the RGB color of (240, 240, 240) respectively, and the filling width is 5 times the width of the spine image.
[0057] As a preferred embodiment of the present invention, the cropping and sorting module retains the book small boxes located in the books large box among all the detection results, sorts the coordinate information of all the small boxes in the order from top to bottom and from left to right, and performs cropping processing to obtain an orderly arranged spine image.
[0058] The present invention has the following advantages:
[0059] The present invention supports uploading shelf images in a library environment, performing real-time positioning detection on book spines, performing title text recognition after orderly cropping, and matching with a library to return a matching result for readers or staff to check all book information on the shelf.
[0060] (1) In the target detection part, YOLOv9 is used as the target detection algorithm. Based on its characteristics of fast real-time output and high accuracy, it can process images quickly and accurately and locate the target. In addition, a cropping and sorting module is designed, which can output the spines in the input image in an orderly manner;
[0061] (2) In the OCR recognition and matching part, the algorithm model provided by the PaddleOCR framework is used to recognize the cropped spine image and match it with the library to return the actual book title. Among the post-recognition processing steps, operations such as sharpening, affine transformation, equal ratio filling, and data cleaning will be added, which helps to enhance the recognition accuracy of the OCR model;
[0062] (3) In the construction part of the cloud platform, an interface program is developed based on the Flask framework, and the target detection algorithm and the OCR recognition and matching algorithm are written in the business logic to build a cloud platform application. The application can realize real-time image upload, spine detection and segmentation, individual recognition and matching, and return the final result to the UI interface for users or staff to check and inventory. Description of the Drawings
[0063] The present invention will be further described below with reference to the accompanying drawings.
[0064] Figure 1The flowchart of the algorithm of the present invention, wherein: Figure (a) is a target detection flowchart, Figure (b) is an OCR recognition flowchart, and Figure (c) is a cloud platform application flowchart;
[0065] Figure 2 This is an example diagram of the target detection result of the present invention;
[0066] Figure 3 This is an example diagram of a segmented book spine image of the present invention;
[0067] Figure 4 is the affine change graph of the present invention;
[0068] Figure 5 is a pixel filling map of the present invention;
[0069] Figure 6 The cloud platform UI interface of the present invention;
[0070] Figure 7 An example diagram of a reference image of the present invention;
[0071] Figure 8 This is an example diagram of the search results of the present invention. DETAILED DESCRIPTION
[0072] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0073] Implementation Example 1: The deep learning-based intelligent book-viewing algorithm for libraries provided by the present invention. For the training of the YOLOv9 network, the input image is first preprocessed, especially Mosaic data enhancement is performed to enrich the detection data set and improve the generalization of the network. Then the pre-trained weights trained on the coco data set are imported. After a certain number of training iterations, the network accuracy reaches the fitting rate and the model weights are saved. Based on the ultra-lightweight model and high-precision model provided by PaddleOCR, the spine image is recognized up to three times to obtain the best recognition effect. Finally, the interface is developed using the Flask framework and the program is deployed on the server and the cloud platform is built. The spine target detection and segmentation of the uploaded bookshelf image are realized, and the segmented spine images are identified and matched one by one, and the results are displayed in the UI interface in sequence. The structure reference of the intelligent book-viewing algorithm for libraries based on deep learning provided by the present invention Figure 1 , the detailed steps are described below:
[0074] Step 1: Dataset creation. The library of Nanjing University of Aeronautics and Astronautics was used as the data collection scene. The image acquisition device was Xiaomi 14Pro. The camera parameters were: 64 megapixels for the main camera, 12 megapixels for the wide-angle lens, 8 megapixels for the telephoto lens, and 5 megapixels for the macro lens. The image size was 4096×3072 pixels. The dataset contained 245 bookshelf images, of which 125 were complete pictures of 2 rows of bookshelves, and the other 120 were pictures of a row of complete bookshelves and other incomplete book spines. Then, Labelimg software was used to annotate the images. The spine of each book was annotated as book, and all the books in a row of bookshelves were annotated as books. The dataset was randomly divided into 171 training sets, 24 validation sets, and 50 test sets in a ratio of 7:1:2.
[0075] Step 2: Build the target detection network. The present invention adopts the YOLOv9 algorithm. The construction of the YOLOv9 network consists of an input end, Backbone, Neck, and Prediction. The input end first stretches the image size to 640×640, and then uses Mosaic data enhancement, that is, four images are spliced by random scaling, random cropping, and random arrangement to form a new image. In addition, the input end will also adaptively calculate the anchor frame. When training starts, the corresponding starting prediction frame is generated according to the initial anchor frame. Backbone uses GELAN. The original ELAN only uses the stacking of convolutional layers as the basic module. GELAN introduces the CSPNet structure based on ELAN. This structure can use any calculation block, so that GELAN can use any calculation block as the basic module. In addition, PGI is also used to generate reliable gradients and update network parameters through auxiliary reversible branches. Neck adopts the PaFPN (Paddle Feature Pyramid Network) structure and uses the SPP (Spatial Pyramid Pooling) module and the PAN (Path Aggregation Network) module. Prediction includes Bounding box loss function and NMS non-maximum suppression, where the Bounding box loss function uses CIoU_Loss and DFL Loss as loss functions.
[0076] Step 3: Target detection network training. The present invention uses the PyTorch deep learning framework to build all networks, the programming language used is python3.9, the CPU is Intel Core i7-10750H@2.60GHz CPU, and the GPU is NVIDIAGeForce RTX 20708G video memory. First, the data set is divided into training set, validation set and test set in a ratio of 7:1:2. The training set is used for network training, validation set testing and saving the best weights, and the test set identifies the generalization effect of the network. YOLOv9 training uses the official YOLOv9-e pre-training weights, SGD is used as the optimizer for model training, the batch size is 4, the initial learning rate is 0.01, and the momentum is set to 0.9. The training round is set to 1000, and the best model appears in epoch792.
[0077] Step 4: Object detection result processing. Each image input into the object detection network will generate a corresponding prediction box, which will frame the book spines in the image one by one, such as Figure 2 In order to extract the book spines corresponding to all the prediction boxes in order, a cropping and sorting module is designed. This module will retain the small book boxes in the books box in all the detection results, sort the coordinate information of all the small boxes from top to bottom and from left to right, and perform cropping to obtain the orderly arranged book spine images, as shown in Figure 1. Figure 3 As shown, it is convenient for subsequent identification and matching output results to be in orderly correspondence.
[0078] Step 5: OCR recognition and matching algorithm. First, the Laplacian operator is used to sharpen the ordered spine image obtained in the previous step to enhance the edges and details in the image. Secondly, the spine image is affine transformed to correct the tilted spine image, such as Figure 4 As shown, the specific steps are as follows:
[0079] (1) Convert the initial image into a grayscale image and use the Canny edge detection algorithm to detect the edge of the image;
[0080] (2) Perform Hough straight line transform on the edge, find the straight line in the image and save the relevant information;
[0081] (3) cutting the detected edge of the book spine;
[0082] (4) Fill and stretch the cut image to a vertical spine state.
[0083] The affine transformation is used to avoid the problem of low recognition accuracy of tilted images by the OCR detection model. At the same time, the adjacent spine information contained in the tilted spine image is excluded, thereby achieving the effect of removing redundant information interference.
[0084] Then, pixel filling is performed on the spine image to reduce the relative distance between characters. As shown in Figure 5 . That is, on the left and right sides of the spine image, RGB color (240, 240, 240) is used for filling respectively, and the filling width is 5 times the width of the spine image, to avoid the situation where the character intervals in the book title of the spine image are large and are recognized by the OCR detection model in different fields, resulting in the final matching failure.
[0085] Finally, the ultra-lightweight model and high-precision OCR recognition model provided by the PaddleOCR framework are used to perform text recognition on the pixel-filled spine image. First, the filled spine image is recognized using the ultra-lightweight model for the first time, and data cleaning is performed on it, that is, special characters including but not limited to "著", "(", ")", ",", ":", etc. are masked in the field. Then, the similarity between the cleaned recognition result and all the book titles in the library is calculated. The similarity calculation formula is as follows:
[0086]
[0087] Among them, str1 represents the cleaned recognition result, str2 represents the book title in the library, length1 represents the character length of str1, length2 represents the character length of str2, num represents the number of characters that are the same in the field and the book title, and LevenshteinDistance represents the edit distance algorithm, which is used to measure the similarity between two characters.
[0088] Secondly, the book title with the highest similarity is used as the matching result. If the highest similarity is greater than 0.3, the matching result is returned; otherwise, a second recognition is performed; for the second recognition, the high-precision model is used, and the book title with the highest similarity is used as the matching result. If the highest similarity is greater than 0.3, the matching result is returned; otherwise, a third recognition is performed; for the third recognition, the image is first subjected to affine transformation and then the high-precision model is used. If the similarity is greater than 0.3, the matching result is returned; otherwise, "0" is returned, indicating that the book title recognition of this spine photo fails. Through OCR recognition of 2819 spine images segmented from 60 pictures collected from the library of Nanjing University of Aeronautics and Astronautics, the recognition accuracy of the present invention is 94.36%, meeting the accuracy requirements for book retrieval.
[0089] Step 6: Construction and application of the cloud platform. The present invention develops an interface program based on the Flask framework and writes the object detection algorithm and OCR recognition and matching algorithm in the business logic, thereby constructing a cloud platform application. This application can realize real-time image uploading, spine detection and segmentation, and one-by-one recognition and matching, and finally return the results to the UI interface for users or staff to check and inventory. The specific business logic is as follows:
[0090] (1) Receive the request sent by the front end, which contains the storage address and database number of the image to be processed in the server;
[0091] (2) Obtain the image to be viewed from the request;
[0092] (3) Call the target detection program and use the trained model to detect the book spine of the image described in step 2, and save the coordinate information [x, y, w, h] of the target box corresponding to each book spine after detection, where x and y represent the horizontal and vertical coordinates of the center point of the book spine, respectively, and w and h represent the width and height of the book spine, respectively;
[0093] (4) Call the cutting and sorting module to cut out the book spines in the image and save them in order according to the saved coordinate information;
[0094] (5) Call the OCR recognition and matching algorithm to perform OCR recognition on the saved book spine images in turn, and calculate the similarity with the book titles in the library to obtain the recognition results;
[0095] (6) Return the recognition results to the cloud platform front end for result display.
[0096] in, Figure 6 The cloud platform UI interface described in the present invention includes three user-selectable functions: library selection, task name, and attachment upload. Library selection indicates the library that needs to be matched, task name indicates the task to be performed (0 indicates only spine cropping, 1 indicates spine cropping and returning the book title), and attachment upload indicates the book photos that the user wants to inventory. Figure 7 As shown in the figure, it is the attachment uploaded by the user. Figure 8 As shown, the returned result is displayed after the attachment is uploaded and the task name is 1.
[0097] The technical solution of the present invention is described in detail above in conjunction with the embodiments / drawings, but the present invention is not limited to the above technical solution. For ordinary technicians in this technical field, after knowing the contents recorded in the present invention, they can make several equivalent changes and substitutions without departing from the principle of the present invention. These equivalent changes and substitutions should also be regarded as belonging to the protection scope of the present invention.
Claims
1. A deep learning-based intelligent inventory method for library books, comprising the following steps: S100, create a data set and divide it into a training set, a validation set, and a test set in a ratio of 7:1:2; S200, build a target detection network model through the YOLOv9 network model. The YOLOv9 network model is composed of input, Backbone, Neck and Prediction. in: At the input end, the input image is preprocessed, wherein the preprocessing is to use an adaptive image scaling method to scale the input image to the same size as the input required by the network, and then perform a normalization operation, and use adaptive anchor frame calculation to generate a corresponding prediction frame based on the initial anchor frame when starting training; Backbone, used to extract image features; Neck, used to combine and mix image features and transmit image features to the prediction layer; Prediction: predict image features, generate bounding boxes and predict the category of the target; S300, using SGD as the optimizer for target detection network training, the batch size is 4, the initial learning rate is 0.01, the momentum is set to 0.9, the training round is set to 1000, and the loss function is CIoU Loss or DFL Loss. The target detection network model is trained, verified and tested through the training set, validation set and test set, wherein the training set is used for training the target detection network model; the validation set tests and saves the optimal weight of the target detection network model; the test set identifies the generalization effect of the target detection network model; It is characterized in that the library book intelligent inventory method also includes the following steps: S400, using the prediction frame generated in step S200, the book spines in each image are framed one by one, and the book spines framed in each image are cropped by a cropping and sorting module, and are sorted in order from top to bottom and from left to right to obtain an ordered book spine image; S500, OCR recognition and matching method, comprising the following steps: S501, performing data enhancement on the ordered spine image obtained in step S400; S502, performing affine transformation on the book spine image to correct the tilted book spine image; S503, performing pixel filling on the spine image so that the relative distance between characters becomes smaller; S504, using the ultra-lightweight model and OCR recognition model provided by the PaddleOCR framework to perform text recognition on the pixel-filled book spine image, and perform data cleaning to obtain the detection result of the OCR recognition model, match the detection result with the titles of all books in the library, and return the book title information with the highest similarity to the detection result as the book title corresponding to the book spine image; S600, an interface program developed based on the Flask framework, writes the target detection method and OCR recognition and matching method into the business logic to build a cloud platform. The cloud platform can detect and segment the book spine after the image is uploaded in real time, and recognize and match it one by one, and finally return the recognition results to the UI interface for users or staff to check and count; in: The Backbone includes GELAN and PGI, wherein the GELAN is constructed by introducing the CSPNet structure on the basis of ELAN, and the CSPNet structure enables GELAN to use any computing block as a basic module; the PGI generates reliable gradients and updates network parameters by assisting reversible branches; The Neck adopts a PaFPN structure and uses an SPP module and a PAN module; The Prediction includes a Bounding box loss function and NMS non-maximum suppression, wherein the Bounding box loss function uses CIoU_Loss or DFL Loss as the loss function.
2. According to claim 1, a deep learning-based intelligent inventory method for library books, It is characterized in that The method for preparing the data set comprises the following steps: S101. Collect images of bookshelves in different areas of the library according to unified standard requirements; S102, using labelimg software to label a single book spine and the entirety of adjacent book spines in the acquired bookshelf image.
3. According to claim 1, a deep learning-based intelligent inventory method for library books, It is characterized in that The specific steps of step S502 are as follows: S5021, converting the initial image into a grayscale image, and detecting the edge of the image using the Canny edge detection algorithm; S5022, perform Hough straight line transformation on the edge, find the straight line in the image, and save the relevant information; S5023, cutting the detected edge of the book spine; S5024, filling and stretching the cut image to a vertical spine state.
4. According to claim 1, a deep learning-based intelligent inventory method for library books, It is characterized in that The specific steps of step S504 are as follows: S5041. Use the ultra-lightweight model to perform a first recognition on the pixel-filled book spine image, and perform data cleaning on the recognition result; S5042: Calculate the similarity between the recognition result after data cleaning and the titles of all books in the library, wherein the similarity calculation formula is as follows: Among them, str1 represents the recognition result after cleaning, str2 represents the book title in the library, length1 represents the character length of str1, length2 represents the character length of str2, num represents the number of characters in the field that are the same as those in the book title, and LevenshteinDistance represents the edit distance algorithm, which is used to measure the similarity between two characters; S5043, taking the book title with the highest similarity as the matching result, if the highest similarity is greater than 0.3, returning the matching result, otherwise proceeding to step S5044 for a second recognition; S5044. Use the OCR recognition model to perform a second recognition on the spine image after pixel filling. Calculate the similarity between the recognition result of the second recognition and all the book titles in the library using the similarity calculation formula. Take the book title with the highest similarity as the matching result. If the highest similarity is greater than 0.3, return the matching result; otherwise, proceed to step S5045 for the third recognition. S5045. After performing an affine transformation on the spine image after pixel filling, use the OCR recognition model for the third recognition. Calculate the similarity between the recognition result of the third recognition and all the book titles in the library using the similarity calculation formula. Take the book title with the highest similarity as the matching result. If the highest similarity is greater than 0.3, return the matching result; otherwise, return "0", indicating that the book title recognition of this spine photo fails.
5. A method for intelligent inventory of library books based on deep learning according to claim 1, characterized in that, the specific logic of the business logic is as follows: S601. Receive the request sent by the front end. The request contains the save address of the picture to be processed in the server and the database number. S602. Obtain the picture to be consulted from the request. S603. Call the object detection method to perform spine detection on the picture in step S602, and save the coordinate information [x, y, w, h] of the detection box corresponding to each detected spine. Here, x and y respectively represent the abscissa and ordinate of the center point of the spine, and w and h respectively represent the width and height of the spine. S604. Call the cropping and sorting module to crop and save the spines in the picture in order according to the saved coordinate information. S605. Call the OCR recognition and matching method to perform OCR recognition on the saved spine pictures in sequence, and calculate the similarity with the book titles in the library to obtain the recognition result. S606. Return the recognition result to the front end of the cloud platform for result display.
6. A method for intelligent inventory of library books based on deep learning according to claim 1 or 5, characterized in that, the object detection method is to input the given input bookshelf image into the trained yolov9 object detection network model to generate the detection box of the spine target, and save the coordinates of the detection box, so as to crop the framed spine through the cropping and sorting module, segment the corresponding spine target, and obtain the spine images arranged in order.
7. A method for intelligent inventory of library books based on deep learning according to claim 1, characterized in that, the UI interface includes three functions that can be independently selected by users: select library, task name, and upload attachment.
8. A method for intelligent inventory of library books based on deep learning according to claim 1, characterized in that, the data cleaning is to mask the special characters in the field. The special characters include but are not limited to "著", "(", ")", ",", ":".
9. A method for intelligent inventory of library books based on deep learning according to claim 1, characterized in that, The pixel filling is performed on the left and right sides of the spine image, using RGB colors of (240, 240, 240) respectively, and the filling width is 5 times the width of the spine image.
10. According to claim 1, a deep learning-based intelligent inventory method for library books, It is characterized in that The cropping and sorting module retains the book small frames in the books large frame in all detection results, sorts the coordinate information of all small frames from top to bottom and from left to right, and performs cropping processing to obtain orderly arranged book spine images.
Citation Information
Cited By
Mobile robot vision book checking method and system
CN121505621A
