Automatic book checking identification method and system for smart library

Abstract

The invention relates to the technical field of automatic inventory, and particularly discloses an automatic book inventory identification method and system for a smart library. Comprising the following steps: S1, obtaining an original image of a book through a mobile device; s2, transmitting the original image to an edge computing device, and preprocessing the original image through the edge computing device to obtain a preprocessed image; s3, performing analysis processing and probability normalization processing on the preprocessed image to obtain book category probability distribution; s4, comparing the book category probability distribution with a book information database to obtain book detailed information; s5, the book detailed information is sent to a mobile device, and the mobile device classifies and arranges the book detailed information and generates a visual inventory report; the technical problems that in the prior art, cost is high, efficiency is low, and accuracy is difficult to guarantee are solved.

Description

Technical Field

[0001] This invention relates to the field of automatic inventory technology, and specifically to an automatic book inventory and identification method and system for smart libraries. Background Technology

[0002] In the development of smart libraries, book inventory is a fundamental and crucial task, and its efficiency and accuracy directly affect the library's management level and service quality. Traditional book inventory methods mainly rely on manual barcode scanning, which has many obvious drawbacks. On the one hand, manual barcode scanning requires a large amount of manpower, as staff need to scan each book individually, resulting in extremely high workload. On the other hand, inventory efficiency is low; for libraries with large collections, completing a comprehensive inventory often takes several days or even longer, seriously affecting the library's normal operation.

[0003] To address the shortcomings of traditional manual barcode scanning for inventory, RFID (Radio Frequency Identification) technology has been increasingly applied to book inventory. RFID identifies target objects using radio frequency signals, offering a speed improvement over manual scanning. However, practical applications of RFID technology require not only equipping each book with an RFID tag but also purchasing a large number of RFID readers and writers. This represents a significant financial burden for many libraries, especially small and medium-sized ones, severely limiting the widespread adoption of this technology.

[0004] Meanwhile, the continuous development of computer vision and edge computing technologies has provided new ideas for innovation in book inventory technology. While existing visual recognition-based book inventory solutions have improved efficiency to some extent, they still face numerous challenges in practical applications. For example, some solutions rely on cloud servers for processing large amounts of image data, which not only incurs high data transmission costs but may also affect the real-time performance of inventory operations due to network latency. Furthermore, some solutions employ overly complex deep learning models that demand high computing power from hardware, making them difficult to deploy and run on low-cost edge devices, thus significantly reducing the overall practicality and cost-effectiveness of the solutions.

[0005] In summary, the current technology of book inventory management faces the critical challenge of achieving low-cost, high-efficiency inventory management while ensuring accuracy. Therefore, there is an urgent need for an automated book inventory method to address the problems of high cost, low efficiency, and difficulty in guaranteeing accuracy in existing technologies, thus meeting the practical needs of smart libraries for efficient and low-cost book inventory management. Summary of the Invention

[0006] In order to overcome the above-mentioned technical problems in the prior art, the present invention provides a method and system for automatic book inventory and identification in smart libraries. By integrating mobile devices, edge computing and lightweight visual recognition technology, the method and system can perform automatic book inventory to solve the technical problems of high cost, low efficiency and difficulty in guaranteeing accuracy in the prior art.

[0007] To achieve the above objectives, embodiments of the present invention provide an automatic book inventory and identification method for smart libraries, comprising the following steps: S1: Obtain the original image of the book via a mobile device; S2: The original image is transmitted to an edge computing device, and a preprocessed image is obtained by preprocessing the original image through the edge computing device; S3: Perform analysis and probability normalization on the preprocessed image to obtain the probability distribution of book categories; S4: Compare the probability distribution of the book categories with the book information database to obtain detailed book information; S5: Send the detailed book information to the mobile device, which then categorizes and organizes the detailed book information and generates a visual inventory report.

[0008] Preferably, step S1 specifically includes: taking pictures of the books on the bookshelf using the camera of a mobile device, and collecting images containing visual information such as book covers and spines as original images.

[0009] Preferably, step S2 specifically includes: the edge computing device includes a low-power processor, memory, storage, a communication module, and a power module; the edge computing device performs image decoding, histogram equalization-based image enhancement, image cropping, and normalization preprocessing on the original image sequentially to generate a preprocessed image; the histogram equalization calculation formula is: Where r_k is the gray level of the original image, s_k is the gray level after transformation, n_j is the frequency of gray level j, n is the total number of pixels in the image, and L is the number of gray levels in the image; the image cropping includes removing irrelevant data from the original image and retaining the main data of the book in the original image, the irrelevant data including bookshelves and environmental clutter; the normalization process includes adjusting the size of the original image to a preset size and mapping the pixel values ​​of the original image to a preset range.

[0010] Preferably, step S3 specifically includes: the edge computing device has a built-in lightweight CNN model, which analyzes the preprocessed image; the lightweight CNN model extracts image features by performing convolution operations on the preprocessed image through multiple convolution kernels, outputs the probability values ​​of the book image belonging to different categories through a fully connected layer, and then performs probability normalization processing through the Softmax function to generate a book category probability distribution.

[0011] Preferably, the lightweight CNN model extracts image features by performing convolution operations on the preprocessed image using multiple convolutional kernels, outputs the probability values ​​of book images belonging to different categories through fully connected layers, and then performs probability normalization processing using the Softmax function to generate a book category probability distribution. Specifically, this includes: performing sliding window calculations on the pixel matrix of the preprocessed image using multiple convolutional kernels of different sizes to extract image features such as edges, textures, and text outlines, generating multi-layer feature maps. The sliding window calculation formula is as follows:

[0012] Where F is the output feature map, K is the convolution kernel, I is the input image / feature map, b is the bias, and l is the number of network layers; the multi-layer feature map is flattened into a one-dimensional vector, and corresponding operations are performed on the features and book categories through a fully connected layer to obtain the matching degree score between the current image and each book category, and generate the original category score; the original category score is probability normalized by the Softmax function to output the book category probability distribution, and the probability normalization calculation formula is:

[0013] Where S_i is the probability of the i-th class, V_i is the original class score of the model output, and C is the total number of classes.

[0014] Preferably, step S4 specifically includes: collecting a book information database through the edge computing device, the book information database including identification information such as ISBN, book title, author, and library location; obtaining the category with the highest probability according to the probability distribution of book categories, using the corresponding identification information to search and match in the book information database, and outputting the detailed book information.

[0015] Preferably, step S5 specifically includes: the edge computing device sending detailed book information to the mobile device via the communication module; the mobile device classifying and organizing the detailed book information according to dimensions such as bookshelf and category, and generating a visual inventory report; the mobile device uploading the visual inventory report to the library management system via the network interface, and synchronously updating the book data in the library management system.

[0016] Accordingly, the present invention also provides an automatic book inventory and identification system for smart libraries, the system comprising: Image acquisition module: Used to acquire image data of books using the camera of a mobile device; Data transmission module: used to transmit the image data to the edge computing device and send the recognition results and book details back to the mobile device; Edge computing module: used for preprocessing image data, analyzing it using a lightweight CNN model, and comparing it with a book information database; Data processing and management module: Used to organize and display data on mobile devices, generate inventory reports, and synchronize them to the library management system.

[0017] The present invention has at least the following technical effects through the technical solution provided by the present invention: By using edge computing devices to preprocess raw images and perform model inference, the original image data can be uploaded to the cloud without uploading it to the cloud, reducing network transmission latency and achieving real-time "instant recognition." Furthermore, the hardware cost of edge computing devices is far lower than that of high-performance servers, and their energy consumption is lower, resulting in significant long-term cost advantages. By embedding lightweight CNN models within edge computing devices, there is no need to deploy expensive GPU servers. Local computing also reduces the bandwidth consumption of data uploads to the cloud, avoiding high data transmission costs over time. Preprocessing of raw images improves feature quality, laying the foundation for accurate model classification. The lightweight model and probabilistic calculations significantly improve recognition accuracy, and database comparison ensures accurate information matching. This effectively solves the problems of "inefficient manual work, expensive RFID, and high energy consumption of cloud solutions" in traditional book inventory management, providing a practical, economical, and reliable technical solution for smart libraries, and promoting the automation and intelligent upgrading of book management. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of an automatic book inventory and identification method for smart libraries provided by an embodiment of the present invention; Figure 2 This is a structural diagram of an automatic book inventory and identification system for smart libraries provided in an embodiment of the present invention. Detailed Implementation

[0019] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0020] In this invention, the terms "system" and "network" are used interchangeably. "Multiple" refers to two or more; therefore, in this invention, "multiple" can also be understood as "at least two." "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / ", unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, it should be understood that in the description of this invention, terms such as "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or order.

[0021] Please see Figure 1 This invention provides an automatic book inventory and identification method for smart libraries, comprising the following steps: S1: Obtain the original image of the book via a mobile device; S2: The original image is transmitted to an edge computing device, and a preprocessed image is obtained by preprocessing the original image through the edge computing device; S3: Perform analysis and probability normalization on the preprocessed image to obtain the probability distribution of book categories; S4: Compare the probability distribution of the book categories with the book information database to obtain detailed book information; S5: Send the detailed book information to the mobile device, which then categorizes and organizes the detailed book information and generates a visual inventory report.

[0022] In this embodiment of the invention, during the preliminary preparation, QR code labels containing the bookshelf number (e.g., 1A-01) and the level (L1, L2) are affixed to the edge of each shelf. Auxiliary devices can be used, such as installing a smart reflector at the bottom of the bookshelf for reflective lighting, and attaching a strong light filter film to the bookshelf near the window. For step S1, the mobile device used is a smartphone or tablet that supports high-resolution shooting and has stable wireless communication capabilities, such as a smartphone or tablet with a camera of 12 megapixels or higher. The mobile device is used to photograph the books on the bookshelf to obtain original images. During shooting, the moving speed is controlled at 0.2-0.5 meters per second, and the camera tilt angle is selected between 15° and 25° to ensure the spine area is fully visible. The tilt angle is calculated using the following formula:

[0023] Where h_spine is the height of the spine, and d_shooting is the shooting distance; during shooting, images of visual information such as the book cover and spine are captured as the original images.

[0024] Furthermore, in step S2, various types of library information are stored in a structured manner in a lightweight embedded relational database (hereinafter referred to as: SQLite database). This SQLite database information is imported into the edge computing device. The edge computing device has a built-in low-power processor, memory, storage, communication module, and power module. The communication module is selected as an integrated Wi-Fi 6 and Bluetooth 5.0 communication module, and the power module is a 5000mAh lithium battery. The edge computing device uses a structural similarity algorithm (SSIM algorithm). When the similarity between adjacent frames is <92%, the current frame is retained. Approximately 3-5 valid images are selected every 10 seconds of video. Then, the edge computing device is used to preprocess the original images according to the following process: (1) First, the original image is decoded. The libjpeg library is used to restore the original image to an RGB three-channel pixel matrix. For example, a 1920×1080 image is converted into a 1920×1080×3 matrix data. (2) Then, image enhancement is performed based on histogram equalization, and the gray values ​​of the original image are redistributed using the following formula:

[0025] Where r_k is the gray level of the original image, s_k is the gray level after transformation, n_j is the frequency of gray level j, n is the total number of pixels in the image, and L is the number of gray levels in the image. (3) Next, image cropping is performed. Irrelevant data is identified by edge detection algorithm. Irrelevant data includes bookshelf boundaries and environmental clutter. Irrelevant data such as bookshelf frames and adjacent clutter are removed in order to retain the main data of the book. (4) Finally, normalize the image size to a preset size, for example, if the preset size is 224×224 pixels, then adjust the image size to 224×224 pixels and map the pixel values ​​of the image to a preset range, for example, if the preset range is [0,1], then map the pixel values ​​to [0,1]. The preprocessed image is obtained by processing the original image through the above process.

[0026] Furthermore, in step S3, a lightweight CNN model is built into the edge computing device. The lightweight CNN model can be either MobileNet or ShuffleNet. The preprocessed image is analyzed and processed using this lightweight CNN model according to the following procedure: (1) The lightweight CNN model extracts image features by performing convolution operations on the preprocessed image using multiple convolution kernels to obtain multi-layer feature maps. Specifically, it uses multiple convolution kernels of different sizes to perform sliding window calculations on the pixel matrix of the preprocessed image. For example, it uses convolution kernels of different sizes such as 3×3 and 5×5 to perform sliding window calculations. The calculation formula is as follows:

[0027] Where F is the output feature map, K is the convolution kernel, I is the input image / feature map, b is the bias, and l is the number of network layers; Taking the first convolutional layer as an example, 32 3×3 convolutional kernels are used to generate 32 feature maps to extract low-level features such as spine edges and text outlines; (2) The probability values ​​of the book images belonging to different categories are output through the fully connected layer. Specifically, the multi-layer feature map is flattened into a one-dimensional vector (e.g., 1×1024). The corresponding operation is performed on the features and book categories through the fully connected layer to obtain the matching degree score between the current image and each book category, and the original category score is generated. That is, the full connected layer is mapped to the library collection category dimension (e.g., 100,000 books correspond to 100,000 dimensions) to generate the original category score. For example, after calculation, the category score of "Journey to the West" is 9.5 and the score of "Dream of the Red Chamber" is 7.2. (3) Then, the probability is normalized using the Softmax function to generate the book category probability distribution; specifically, the original category scores are normalized using the Softmax function to generate the book category probability distribution, which is calculated using the following formula:

[0028] Where S_i is the probability of the i-th class, V_i is the original class score output by the model, and C is the total number of classes; in the example above, the probability of "Journey to the West" is calculated using the following formula:

[0029] Therefore, the probability of matching "Journey to the West" is calculated to be 0.92, which means there is a 92% probability that the book is a match.

[0030] In this embodiment of the invention, for step S4, in the preliminary preparation, various types of library information are stored in a structured manner in a lightweight embedded relational database (hereinafter referred to as: SQLite database). The SQLite database information is imported into the edge computing device. The SQLite database information includes identification information such as International Standard Book Number (ISBN code), book title, author, and collection location (e.g., row 2, shelf 3 in section A). Based on the highest probability category in the output book category probability distribution, the SQLite database is retrieved and matched through the corresponding identification information (e.g., ISBN code), and detailed book information is output. The detailed book information is structured book information, which includes ISBN code, book title, author, and collection location.

[0031] Furthermore, in step S5, the edge computing device transmits book information back to the mobile device via Wi-Fi 6 at a transmission rate of approximately 867 Mbps, with the transmission time for 100 book records being less than 1 second. Next, the mobile device organizes the data by bookshelf (e.g., section A, section B) and category (e.g., literature, science), and marks missing books (e.g., books that exist in the database but are not identified). Based on the organized data and the data of marked missing books, a visual inventory report is generated. This visual inventory report is in PDF format and includes a list of inventoried books, location anomaly alerts, etc. Finally, the mobile device uploads this visual inventory report to the library management system via a network interface, synchronously updating the book data in the library management system.

[0032] In one implementation, if the inventory method provided by this invention is used to inventory an old library, the hardware requires a high-sensitivity camera equipped with a laser fill light. The algorithm can employ a noise reduction algorithm using multi-frame fusion, i.e., continuously capturing multiple frames of images and reducing noise through weighted averaging. For example, continuously capturing 5 frames of images and reducing noise through weighted averaging, the formula is as follows: ; For the restoration of worn text, a GAN (Generative Adversarial Network) can be used to reconstruct the blurred text. For example, the worn "08" in "ISBN" 978-7-100-08 can be reconstructed into a complete number.

[0033] Please see Figure 2 Based on the same inventive concept, embodiments of the present invention provide an automatic book inventory and identification system for smart libraries, the system comprising: Image acquisition module: Used to acquire book image data using the camera of a mobile device. It consists of a mobile terminal and an adjustable tilt bracket. The bracket supports tilt adjustment from 15° to 45°. It is equipped with casters at the bottom for easy movement between bookshelves. The camera supports autofocus and optical image stabilization and can produce clear images within a distance of 0.5 meters to 1.5 meters. Data transmission module: used to transmit the image data to the edge computing device and send the recognition results and book details back to the mobile device. It adopts dual-link backup of Wi-Fi 6 and Bluetooth. Wi-Fi is used for high-bandwidth image transmission (peak rate 1.2Gbps) and Bluetooth is used for low-power control signal transmission (such as device pairing and shooting commands). The transmission protocol adopts TCP / IP and a retransmission mechanism is set to ensure data integrity.

[0034] Edge computing module: used for preprocessing image data, analyzing it using a lightweight CNN model, and comparing it with a book information database. The hardware adopts a portable box design (15cm×10cm×5cm) and integrates a processor, memory, storage, communication module, and power module.

[0035] Data Processing and Management Module: This module is used to organize, display, generate inventory reports, and synchronize data to the library management system on mobile devices. The mobile devices support Android and iOS systems. Functions include: real-time image preview and shooting control, inventory task creation and progress display (e.g., "30% of A-section completed"), data visualization (bookshelf heat map, category distribution pie chart), abnormal book marking and manual review entry, report export and system synchronization functions.

[0036] Furthermore, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the present invention.

[0037] The optional embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention.

[0038] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not describe the various possible combinations separately.

[0039] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a microcontroller, chip, or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0040] Furthermore, various different implementations of the present invention can be combined arbitrarily, as long as they do not violate the spirit of the present invention, they should also be regarded as the content disclosed in the present invention.

Claims

1. A book automatic inventory identification method for a smart library, characterized in that, The method comprises the following steps: S1: obtaining an original image of a book through a mobile device; S2: transmitting the original image to an edge computing device to obtain a preprocessed image after preprocessing the original image by the edge computing device; S3: performing analysis processing and probability normalization processing on the preprocessed image to obtain a book category probability distribution; S4: comparing the book category probability distribution with a book information database to obtain detailed information of the book; S5: sending the detailed information of the book to the mobile device, and performing classification and arrangement on the detailed information of the book by the mobile device to generate a visual inventory report.

2. The automatic book inventory recognition method for a smart library according to claim 1, wherein, The step S1 specifically comprises: capturing the books on the bookshelf through the camera of the mobile device to collect an image containing visual information of the book cover and the book spine as the original image.

3. The automatic book inventory recognition method for a smart library according to claim 1, wherein, The step S2 specifically comprises: The edge computing device comprises a low-power processor, a memory, a storage, a communication module and a power module; The edge computing device generates the preprocessed image after sequentially performing image decoding, image enhancement based on histogram equalization, image cropping and normalization preprocessing on the original image; The calculation formula of the histogram equalization is: where r_k is the gray level of the original image, s_k is the transformed gray level, n_j is the frequency of the gray level j, n is the total number of pixels in the image, and L is the number of gray levels of the image. The image cropping comprises removing irrelevant data in the original image and retaining the main data of the book in the original image, wherein the irrelevant data comprises bookshelves and environmental sundries; The normalization processing comprises adjusting the size of the original image to a preset size and mapping the pixel value of the original image to a preset interval.

4. The automatic book inventory recognition method for a smart library according to claim 3, wherein, The step S3 specifically comprises: The edge computing device is built-in with a lightweight model, and the preprocessed image is analyzed by the lightweight CNN model; The lightweight CNN model extracts image features by performing convolution operation on the preprocessed image through multiple convolution kernels, outputs probability values of the book image belonging to different categories through a full connection layer, and performs probability normalization processing through a Softmax function to generate a book category probability distribution.

5. The automatic book inventory recognition method for a smart library according to claim 4, wherein, The lightweight CNN model extracts image features by performing convolution operation on the preprocessed image through multiple convolution kernels, outputs probability values of the book image belonging to different categories through a full connection layer, and performs probability normalization processing through a Softmax function to generate a book category probability distribution, specifically comprising: A sliding window calculation is performed on the preprocessed image pixel matrix using multiple convolution kernels of different sizes to extract image features such as edges, textures and text outlines, generate multiple feature maps, and the sliding window calculation formula is: wherein F is an output feature map, K is a convolution kernel, I is an input image / feature map, b is a bias, and l is a network layer number; The multiple feature maps are flattened into a one-dimensional vector, and a full connection layer is used to perform corresponding operations on the features and the book categories to obtain a matching score of the current image and each book category, and generate an original category score; The original category score is calculated by a Softmax function to output a book category probability distribution, and the probability normalization calculation formula is: where S_i is the i-th class probability, V_i is the model output raw class score, and C is the total number of classes.

6. The automatic book inventory recognition method for a smart library according to claim 5, wherein, The step S4 specifically comprises: The edge computing device collects a book information database, and the book information database comprises identification information such as an international standard book number, a book name, an author and a library location; According to the book category probability distribution, the category with the highest probability is obtained, the corresponding identification information is used to retrieve the match in the book information database, and the book detailed information is output.

7. The automatic book inventory recognition method for a smart library according to claim 6, wherein, The step S5 specifically includes: The edge computing device sends the book detailed information to the mobile device through the communication module; The mobile device classifies and arranges the book detailed information according to the dimensions such as bookshelf and category, and generates a visual inventory report; The mobile device uploads the visual inventory report to the library management system through the network interface, and synchronously updates the book data of the library management system.

8. An automatic book inventory recognition system for a smart library, characterized by, The system includes: An image acquisition module is configured to acquire book image data by using a camera of the mobile device; A data transmission module is configured to transmit the image data to the edge computing device, and return the recognition result and the book detailed information to the mobile device; An edge computing module is configured to pre-process the image data, analyze the image data by using a lightweight CNN model, and compare the image data with a book information database; A data processing and management module is configured to arrange, display, generate an inventory report, and synchronize the data to a library management system on the mobile device.

Images