Automatic book checking and abnormity early warning system and method

Abstract

The invention discloses an automatic book checking and abnormity early warning system and method, and the method comprises the steps: carrying out the marking of books through a CODE128 bar code, precisely positioning each bookshelf through a cloud platform spherical camera which is disposed at a position opposite to the front surface of the bookshelf, capturing an image, correcting the image through a perspective transformation algorithm, segmenting and extracting the bar code through a YOLOv8 target detection algorithm, and carrying out the early warning of the abnormity of the books. Meanwhile, the abnormal state of book placement is detected, and if abnormality is detected, the system records abnormal information and gives an alarm; the bar code is identified through a ZXing library, and when identification fails, the image is automatically rotated for retry; the system records and informs an administrator of a bar code which cannot be recognized; the infrared sensor arranged in the bookshelf monitors the state change of the books in real time, and if the state change occurs, the system automatically triggers the re-checking of the area so as to update the inventory information in real time. According to the invention, the problems of low efficiency, long period and incapability of updating book position information in real time of traditional library manual checking are effectively solved.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to an automatic book inventory and abnormality early warning system and method. Background Art

[0002] Traditional book inventory work mainly relies on manual operation, which is time-consuming, labor-intensive, and prone to errors. Especially in large libraries or bookstores, faced with a large number of books of various types, manual inventory is inefficient and difficult to meet the requirements of modern library management for efficiency and accuracy. Although the existing book management system has introduced barcode technology to assist management, in actual application, the rapid positioning and identification of barcodes still face great challenges. Especially in an environment with dense and complex bookshelves, it is extremely inconvenient to manually scan the barcode of each book, which can easily lead to omissions or repeated counting. In addition, irregular placement of books (such as books inverted, reversed, or barcodes blocked) frequently occurs, further increasing the difficulty of inventory. For the monitoring of abnormal status of books, traditional systems generally lack effective automated monitoring and early warning mechanisms, resulting in the inability to detect problems in a timely manner, thus affecting the effective management of book resources and the reader's experience.

[0003] In the context of the prior art, automatic book inventory systems are mainly divided into two categories. One is a system based on RFID technology, which uses the non-contact reading and writing characteristics of RFID to achieve precise positioning and real-time management of books, allowing books to be placed in disorder and managed in order. Patent CN202210378539.5 discloses a book inventory method based on a book inventory robot. This method combines RFID technology with visual sensing technology, significantly improves inventory efficiency and accuracy, can accurately locate the specific location of books, and reduce miscounting, and effectively identify empty bookshelves. However, the cost of such solutions is relatively high, and signal interference is prone to occur when multiple RFID tags are close, affecting recognition accuracy.

[0004] The second is a book inventory system based on computer vision technology. Patent CN202310377032.2 discloses a book inventory method, device and terminal equipment based on visual recognition, which processes images through horizontal windows and vertical windows, identifies barcodes in bookshelves, and realizes automatic inventory. However, this method has certain limitations in complex lighting conditions, different book arrangements, and barcode recognition accuracy, especially when dealing with barcodes at long distances or poor angles. Therefore, achieving fast and accurate inventory in book management, and automatically identifying and processing various anomalies in the inventory process, are still technical problems that need to be solved in the current field.

[0005] Based on the above technical background, it is crucial to develop a system that can automatically and efficiently inventory books, accurately identify barcodes, and promptly warn of abnormal situations.

[0006] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgment or any form of suggestion that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0007] The purpose of the present invention is to provide an automated book inventory and abnormal warning system and method, which utilizes computer vision and image processing technology to overcome the defects in the above-mentioned prior art.

[0008] To achieve the above-mentioned purpose, the present invention provides an automated book inventory and abnormal warning system, comprising a barcode, a pan-tilt spherical camera, an image correction module, a barcode detection module, an abnormality detection module, a barcode recognition module, an infrared sensing module, and a book management system; the barcode is attached to the book for book identification and management; the pan-tilt spherical camera is respectively connected to the image correction module and the book management system for acquiring local images of each grid of the bookshelf and transmitting the image data to the image correction module; the image correction module is respectively connected to the barcode detection module and the abnormality detection module for perspective transformation of the local image, correcting image distortion, and transmitting the corrected image to the barcode detection module and the abnormality detection module respectively; the barcode detection module is connected to the barcode recognition module for detecting and segmenting the barcode in the corrected image, and the segmented barcode image is sent to the barcode recognition module; the abnormality detection module is connected to the book management system for detecting and segmenting the barcode in the corrected image, and the segmented barcode image is sent to the barcode recognition module; The barcode detection module is connected to the library management system for communication, and is used to detect the book placement status in the corrected image, and the detected abnormal book placement information is transmitted to the library management system; the barcode detection module is connected to the library management system for communication, and is used to identify the barcode in the barcode image, and send the identification result to the library management system; the infrared sensing module is connected to the library management system for communication, and is used to detect the state change of the books in each grid of the bookshelf, and send the book state change information to the library management system; the library management system is used to receive the abnormal book placement information, the barcode identification information and the book state change information, on the one hand, determine the location of the abnormal book according to the abnormal book placement information and the barcode identification information, quickly generate alarm information to notify the administrator, and on the other hand, control the pan-tilt ball camera to collect the grid local image of the book state change according to the book state change information, re-inventory the books in the grid according to the barcode identification information of the grid local image, and synchronize and update the inventory result to the inventory data in real time.

[0009] A method for using an automated book inventory and abnormal warning system, the steps of which are: S01. Attach a CODE128 barcode to each book. The CODE128 barcode contains 8 digits and is matched one by one with the corresponding book in the book database to ensure the accuracy of computer vision recognition; S02, the pan / tilt ball camera facing the bookshelf locates each grid of the bookshelf and captures a partial image of each grid; S03, using a perspective transformation algorithm to correct the local image captured in S02, selecting four key points of the target area in the local image, converting the parallelogram formed by the four key points into a rectangle, and eliminating image distortion; S04. Adjust the head structure of the YOLOv8 target detection model, remove the feature map level of large-size detection from the feature map level of the original YOLOv8 target detection model, and retain the feature map level of small-size P3 / 8 and medium-size P4 / 16 detection; use the book abnormal state training set to train the YOLOv8 target detection model, use the trained YOLOv8 target detection model to segment the corrected local image and extract the barcode image, and detect whether the placement of the book is inverted or reversed. If an abnormality is detected, send the alarm information to the library management system and notify the administrator to check and handle it; S05. Recognize the barcode image extracted in S04 through the ZXing library. If the initial recognition fails, the barcode image will be automatically rotated 5 degrees each time and re-recognized. If the barcode cannot be recognized due to occlusion, wear and tear or blur, the barcode image will be reported to the library management system to notify the administrator in time for processing; S06. When a book is taken out or put into a grid, the infrared sensor in the grid can immediately send the location information of the changed grid to the library management system. After receiving the location information, the library management system controls the pan-tilt dome camera to capture a local image of the grid. After the local image is processed through steps S03-S05, the library management system re-inventories the books in the grid according to the barcode decoding data, and updates the inventory results to the inventory data synchronously and in real time.

[0010] Preferably, in the technical solution, the perspective transformation algorithm process of step S03 is: S31, selecting four key points of the target area in the local image, where these points correspond to four corners of the target area; due to the deflection of the camera angle, these four points form a parallelogram in the local image; S32, setting a target rectangle in the corrected partial image, the rectangle representing the corrected target area, the four corners of which respectively correspond to the four key points in the target area before correction; S33. Calculate a perspective transformation matrix according to the four vertices of the parallelogram in the local image and the four corners of the target rectangle, and apply the matrix to the local image to perform a perspective transformation on it, so that the distorted parallelogram area is corrected into a regular rectangle.

[0011] Preferably, in the technical solution, the selection of key points is only required during the first configuration, and after the coordinate information of the corresponding points is obtained, the perspective transformation of the target area of ​​the local image will be automatically performed based on the coordinate information.

[0012] Preferably, in the technical solution, the book abnormal state data set includes book barcode images and book spine images; the YOLOv8 target detection model includes a YOLOv8 barcode detection model and a YOLOv8 book abnormal placement detection model.

[0013] Preferably, in the technical solution, the book abnormal state data set is manually annotated, and the book barcode image and the book spine image are respectively classified into two different categories; the book abnormal state data set is trained using the YOLOv8 target detection model, so that the YOLOv8 barcode detection model learns and extracts the feature information of the barcode, and the YOLOv8 book abnormal placement detection model learns and extracts the feature information of the spine; after training, the YOLOv8 barcode detection model can accurately identify and distinguish the barcode of the book, and the YOLOv8 book abnormal placement detection model can accurately identify and distinguish the spine of the book.

[0014] Preferably, in the technical solution, the YOLOv8 target detection model detection process in step S04 is: S41, preprocessing the corrected local image, the preprocessing including compressing the image size and normalizing it, so that the size and pixel value of the local image meet the input requirements of the YOLOv8 target detection model through the preprocessing; S42, inputting the preprocessed local image into the YOLOv8 barcode detection model, the YOLOv8 barcode detection model extracts image features through multiple convolution and pooling operations and generates feature maps of different scales; on the feature map, the YOLOv8 convolutional neural network covers the target that may be the barcode with predefined rectangular boxes of different sizes and proportions, and predicts multiple rectangular boxes and target confidence for each possible target; S43, filtering different rectangular frames of the same target according to the confidence level, obtaining the position information of the barcode corresponding to the best rectangular frame, and then cutting out the barcode from the original image according to the ratio between the preprocessed image and the original image; S44, inputting the preprocessed local image into the YOLOv8 abnormal book placement detection model, the YOLOv8 abnormal book placement detection model extracts image features through multiple convolution and pooling operations and generates feature maps of different scales; on the feature map, the YOLOv8 abnormal book placement detection model uses predefined rectangular boxes of different sizes and proportions to cover targets that may be in abnormal placement states, and predicts multiple rectangular boxes and target confidence and category probability for each possible target; S45. Filter different rectangular frames of the same target according to confidence and category probability to obtain the best rectangular frame corresponding to the target, and then mark the target with abnormal placement status with a box on the original image according to the ratio relationship between the preprocessed image and the original image, and note the abnormal placement of the book on the box.

[0015] Preferably, in the technical solution, when the target barcode area in the local image is tilted at a certain angle, the optimal rectangular frame is rotated until the optimal rectangular frame completely covers the target barcode, and then the barcode is cropped from the original image.

[0016] Preferably, in the technical solution, in step S05, the process of ZXing library recognizing the barcode image is: S51, after receiving the barcode image segmented by the target detection model, the barcode recognition module pre-processes the input barcode image, converts it into a grayscale image, and then converts the grayscale image into a black and white image by binarization, so that the black and white stripes of the barcode are more obvious; S52, ZXing library uses edge detection algorithm to detect the edge in the black and white image. The black and white stripes of the barcode in the black and white image produce brightness changes. ZXing library analyzes the brightness changes of the black and white image during the detection process and determines the possible position of the barcode in the black and white image. S53. After determining the barcode area, the ZXing library analyzes the stripe features in detail and extracts the width information of each stripe. According to the combination of stripes of different widths, the ZXing library maps these combinations to corresponding characters, decodes them according to the barcode encoding rules, and outputs the decoded data to the library management system.

[0017] Compared with the prior art, the present invention has the following beneficial effects: (1) By integrating an adjustable-focus pan-tilt spherical camera with high-performance image processing technology, the system realizes rapid and automatic recognition of book barcodes, greatly reduces manual intervention, significantly improves the speed and accuracy of book inventory, and solves the problem of low efficiency and proneness to errors in traditional manual inventory.

[0018] (2) The system integrates an intelligent anomaly detection and early warning module, which can automatically identify abnormal book placement (such as inverted or reversed) and promptly notify management personnel to intervene, ensuring the accuracy of book data and the orderliness of bookshelves, and avoiding inventory errors caused by abnormal book status.

[0019] (3) By adopting the perspective transformation algorithm and the advanced YOLOv8 target detection model, the system can accurately segment and recognize barcodes in complex environments, significantly improving the stability and adaptability of image recognition, and ensuring that the system maintains efficient recognition performance under different lighting conditions and angles.

[0020] (4) The bookshelf is equipped with built-in infrared sensors to monitor the dynamics of books in real time and automatically trigger local inventory and inventory information updates in the changed areas, ensuring the real-time and accuracy of book location information, effectively improving the efficiency of book resource management.

[0021] (5) The system is seamlessly integrated with the book lending system, realizing the automatic update and synchronization of book location information, simplifying the book borrowing and returning process, improving the reader experience and library operational efficiency, and providing data support for library resource management and decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a module framework diagram of an automated book inventory and abnormal warning system of the present invention; Figure 2 This is a workflow diagram of an automated book inventory and abnormal warning system of the present invention; Figure 3 This is a structural diagram of the improved YOLOv8 target detection model for barcode segmentation in the present invention. DETAILED DESCRIPTION

[0023] The specific embodiments of the present invention are described in detail below, but it should be understood that the protection scope of the present invention is not limited by the specific embodiments.

[0024] Unless explicitly stated otherwise, throughout the specification and claims, the term “comprise” or variations such as “include” or “comprising”, etc., will be understood to include the stated elements or components but not to exclude other elements or components.

[0025] like Figure 1As shown, an automated book inventory and abnormality warning system includes a barcode, a PTZ spherical camera, an image correction module, a barcode detection module, an abnormality detection module, a barcode recognition module, an infrared sensing module, and a book management system; the barcode is attached to the book for book identification and management; the PTZ spherical camera is respectively connected in communication with the image correction module and the book management system to obtain local images of each grid of the bookshelf and transmit the image data to the image correction module; the image correction module is respectively connected in communication with the barcode detection module and the abnormality detection module to perform perspective transformation on the local image, correct image distortion, and transmit the corrected image to the barcode detection module and the abnormality detection module respectively; the barcode detection module is connected in communication with the barcode recognition module to detect and segment the barcode in the corrected image, and the segmented barcode image is sent to the barcode recognition module; the abnormality detection module is connected in communication with the book management system to The book placement status in the corrected image is detected, and the detected abnormal book placement information is transmitted to the library management system; the barcode detection module is connected to the library management system for identifying the barcode in the barcode image and sending the identification result to the library management system; the infrared sensing module is connected to the library management system for detecting the state change of the books in each grid of the bookshelf and sending the book state change information to the library management system; the library management system is used to receive the abnormal book placement information, barcode identification information and book state change information, on the one hand, determine the location of the abnormal book according to the abnormal book placement information and barcode identification information, quickly generate alarm information to notify the administrator, and on the other hand, control the pan-tilt spherical camera to collect the grid local image of the book state change according to the book state change information, re-inventory the books in the grid according to the barcode identification information of the grid local image, and synchronize and update the inventory result to the inventory data in real time.

[0026] like Figure 2 As shown, an automated book inventory and abnormal warning system and method, the steps are: S01. Attach a CODE128 barcode to each book. The CODE128 barcode contains 8 digits and is matched one by one with the corresponding book in the book database to ensure the accuracy of computer vision recognition; S02, the pan / tilt ball camera facing the bookshelf locates each grid of the bookshelf and captures a partial image of each grid; S03, using a perspective transformation algorithm to correct the local image captured in S02, selecting four key points of a target area such as a book spine in the local image, these points correspond to the four corners of the book spine; due to the deflection of the camera angle, these four points form a parallelogram in the local image; A target rectangle is set in the corrected local image, the rectangle represents the corrected target area, and the four corners of the rectangle correspond to the four key points in the target area before correction; According to the four vertices of the parallelogram in the local image and the four corners of the target rectangle, a perspective transformation matrix is ​​calculated, and the matrix is ​​applied to the local image to perform a perspective transformation on it, so that the distorted parallelogram area is corrected into a regular rectangle; The selection of key points is only required during the first configuration. After obtaining the coordinate information of the corresponding points, the spine of the local image will be automatically transformed according to the coordinate information. S04. Adjust the head structure of the YOLOv8 target detection model, remove the feature map level of large-size detection in the original YOLOv8 target detection model, and retain the feature map level of small-size P3 / 8 and medium-size P4 / 16 detection, such as Figure 3 As shown; The YOLOv8 target detection model is trained using the book abnormal state training set, which includes book barcode images and book spine images. The YOLOv8 target detection model includes the YOLOv8 barcode detection model and the YOLOv8 book abnormal placement detection model. The book abnormal state data set is manually annotated, and the book barcode images and book spine images are classified into two different categories respectively. The YOLOv8 target detection model is used to train this book abnormal state data set, so that the YOLOv8 barcode detection model learns and extracts the feature information of the barcode, and the YOLOv8 book abnormal placement detection model learns and extracts the feature information of the spine. After training, the YOLOv8 barcode detection model can accurately identify and distinguish the barcode of the book, and the YOLOv8 book abnormal placement detection model can accurately identify and distinguish the spine of the book. Preprocess the corrected local image, including compressing the image size and normalizing it. The preprocessing makes the size and pixel value of the local image meet the input requirements of the YOLOv8 target detection model. The preprocessed local image is input into the YOLOv8 barcode detection model. The YOLOv8 barcode detection model extracts image features through multiple convolution and pooling operations and generates feature maps of different scales. On the feature map, the YOLOv8 convolutional neural network covers the target that may be a barcode with predefined rectangular boxes of different sizes and proportions, and predicts multiple rectangular boxes and target confidence for each possible target. The confidence indicates the probability that the rectangular box actually contains the target object. Different rectangular frames of the same target are filtered according to the confidence level to obtain the position information of the barcode corresponding to the best rectangular frame, and then the barcode is cut out from the original image according to the ratio between the preprocessed image and the original image; when the target barcode area in the local image is tilted at a certain angle, the best rectangular frame is rotated until the best rectangular frame completely covers the target barcode, and then the barcode is cut out from the original image; The preprocessed local image is input into the YOLOv8 abnormal book placement detection model. The YOLOv8 abnormal book placement detection model extracts image features through multiple convolution and pooling operations and generates feature maps of different scales. On the feature map, the YOLOv8 abnormal book placement detection model uses pre-defined rectangular boxes of different sizes and proportions to cover the targets that may be in abnormal placement states, and predicts multiple rectangular boxes and target confidence and category probability for each possible target. The confidence indicates the possibility that the rectangular box actually contains the target object, and the category probability indicates the possibility that the target object in the rectangular box belongs to different categories. Different rectangular boxes of the same target are filtered according to confidence and category probability to obtain the best rectangular box corresponding to the target. Then, according to the ratio between the pre-processed image and the original image, the target with abnormal placement is marked with a box on the original image, and the abnormal placement of the book is noted on the box. The alarm information is sent to the library management system and the administrator is notified to check and handle it. S05, the barcode image extracted in S04 is recognized through the ZXing library. After receiving the barcode image segmented by the target detection model, the barcode recognition module will first pre-process the input barcode image and convert it into a grayscale image, and then convert the grayscale image into a black and white image through binarization, so that the black and white stripes of the barcode are more obvious; The ZXing library uses edge detection algorithms to detect edges in black-and-white images. The black-and-white stripes of the barcode in the black-and-white image produce brightness changes. The ZXing library analyzes the brightness changes of the black-and-white image during the detection process and determines the possible position of the barcode in the black-and-white image. After determining the barcode area, the ZXing library analyzes the stripe features in detail and extracts the width information of each stripe. Based on the combination of stripes of different widths, the ZXing library maps these combinations to corresponding characters, decodes them according to the barcode encoding rules, and outputs the decoded data to the library management system. If the initial recognition fails, the barcode image will be automatically rotated 5 degrees each time and re-recognized; if the barcode cannot be recognized due to occlusion, wear and tear or blur, the system will report the barcode image to the library management system and notify the administrator in time for processing; S06. When a book is taken out or put into a grid, the infrared sensor in the grid can immediately send the location information of the changed grid to the library management system. After receiving the location information, the library management system controls the pan-tilt dome camera to capture a local image of the grid. After the local image is processed through steps S03-S05, the library management system re-inventories the books in the grid according to the barcode decoding data, and updates the inventory results to the inventory data synchronously and in real time.

[0027] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.

Claims

1. An automated book inventory and abnormal warning system, comprising a barcode, a PTZ spherical camera, an image correction module, a barcode detection module, an abnormality detection module, a barcode recognition module, an infrared sensing module, and a book management system; the barcode is attached to the book for book identification and management; the PTZ spherical camera is respectively connected to the image correction module and the book management system for obtaining local images of each grid of the bookshelf and transmitting the image data to the image correction module; the image correction module is respectively connected to the barcode detection module and the abnormality detection module for perspective transformation of the local image, correction of image distortion, and transmission of the corrected image to the barcode detection module and the abnormality detection module; the barcode detection module is connected to the barcode recognition module for detecting and segmenting the barcode in the corrected image, and the segmented barcode image is sent to the barcode recognition module; the abnormality detection module is connected to the book management system for detecting the book placement status in the corrected image, and the detected abnormal placement information of the book is transmitted to the book management system; The barcode detection module is connected to the library management system for identifying the barcode in the barcode image and sending the identification result to the library management system; The infrared sensing module is communicatively connected with the library management system, and is used for detecting the state change of the books in each grid of the bookshelf, and sending the book state change information to the library management system; the library management system is used for receiving the abnormal placement information of the books, the barcode identification information and the book state change information, and on the one hand, determines the location of the abnormal book according to the abnormal placement information of the books and the barcode identification information, and quickly generates an alarm message to notify the administrator; on the other hand, controls the pan / tilt spherical camera to collect the local image of the grid where the book state changes according to the book state change information, and re-inventories the books in the grid according to the barcode identification information of the local image of the grid, and synchronously updates the inventory result to the inventory data in real time.

2. A method for using the automated book inventory and abnormal warning system according to claim 1, comprising the following steps: S01. Attach a CODE128 barcode to each book. The CODE128 barcode contains 8 digits and is matched one by one with the corresponding book in the library management system; S02, the pan / tilt ball camera facing the bookshelf locates each grid of the bookshelf and captures a partial image of each grid; S03, using a perspective transformation algorithm to correct the local image captured in S02, selecting four key points of the target area in the local image, converting the parallelogram formed by the four key points into a rectangle, and eliminating image distortion; S04. Adjust the head structure of the YOLOv8 target detection model, remove the feature map level of large-size detection from the feature map level of the original YOLOv8 target detection model, and retain the feature map level of small-size P3 / 8 and medium-size P4 / 16 detection; use the book abnormal state training set to train the YOLOv8 target detection model, use the trained YOLOv8 target detection model to segment the corrected local image and extract the barcode image, and detect whether the placement of the book is inverted or reversed. If an abnormality is detected, send the alarm information to the library management system and notify the administrator to check and handle it; S05. Recognize the barcode image extracted in S04 through the ZXing library. If the initial recognition fails, the barcode image will be automatically rotated 5 degrees each time and re-recognized. If the barcode cannot be recognized due to occlusion, wear and tear or blur, the barcode image will be reported to the library management system to notify the administrator in time for processing; S06. When a book is taken out or put into a grid, the infrared sensor in the grid can immediately send the location information of the changed grid to the library management system. After receiving the location information, the library management system controls the pan-tilt dome camera to capture a local image of the grid. After the local image is processed through steps S03-S05, the library management system re-inventories the books in the grid according to the barcode decoding data, and updates the inventory results to the inventory data synchronously and in real time.

3. The method for using the automated book inventory and abnormal warning system according to claim 2 is characterized by: The perspective transformation algorithm process of step S03 is: S31, selecting four key points of the target area in the local image, where these points correspond to four corners of the target area; due to the deflection of the camera angle, these four points form a parallelogram in the local image; S32, setting a target rectangle in the corrected partial image, the rectangle representing the corrected target area, the four corners of which respectively correspond to the four key points in the target area before correction; S33. Calculate a perspective transformation matrix according to the four vertices of the parallelogram in the local image and the four corners of the target rectangle, and apply the matrix to the local image to perform a perspective transformation on it, so that the distorted parallelogram area is corrected into a regular rectangle.

4. The method for using the automated book inventory and abnormal warning system according to claim 3 is characterized by: The selection of key points is only required during the first configuration. After obtaining the coordinate information of the corresponding points, the perspective transformation of the target area of ​​the local image will be automatically performed based on the coordinate information.

5. The method for using the automated book inventory and abnormal warning system according to claim 2 is characterized by: The book abnormal status dataset includes book barcode images and book spine images; the YOLOv8 target detection model includes the YOLOv8 barcode detection model and the YOLOv8 book abnormal placement detection model.

6. The method for using the automated book inventory and abnormal warning system according to claim 5 is characterized by: Manually annotate the abnormal book status dataset, and classify the book barcode images and book spine images into two different categories. Use the YOLOv8 target detection model to train this abnormal book status dataset, so that the YOLOv8 barcode detection model can learn and extract the feature information of the barcode, and the YOLOv8 abnormal book placement detection model can learn and extract the feature information of the book spine. After training, the YOLOv8 barcode detection model can accurately identify and distinguish the barcodes of books, and the YOLOv8 abnormal book placement detection model can accurately identify and distinguish the spines of books.

7. The method for using the automated book inventory and abnormal warning system according to claim 6 is characterized by: The YOLOv8 target detection model detection process in step S04 is: S41, preprocessing the corrected local image, the preprocessing including compressing the image size and normalizing it, so that the size and pixel value of the local image meet the input requirements of the YOLOv8 target detection model through the preprocessing; S42, inputting the preprocessed local image into the YOLOv8 barcode detection model, the YOLOv8 barcode detection model extracts image features through multiple convolution and pooling operations and generates feature maps of different scales; on the feature map, the YOLOv8 convolutional neural network covers the target that may be the barcode with predefined rectangular boxes of different sizes and proportions, and predicts multiple rectangular boxes and target confidence for each possible target; S43, filtering different rectangular frames of the same target according to the confidence level, obtaining the position information of the barcode corresponding to the best rectangular frame, and then cutting out the barcode from the original image according to the ratio between the preprocessed image and the original image; S44, inputting the preprocessed local image into the YOLOv8 abnormal book placement detection model, the YOLOv8 abnormal book placement detection model extracts image features through multiple convolution and pooling operations and generates feature maps of different scales; on the feature map, the YOLOv8 abnormal book placement detection model uses predefined rectangular boxes of different sizes and proportions to cover targets that may be in abnormal placement states, and predicts multiple rectangular boxes and target confidence and category probability for each possible target; S45. Filter different rectangular frames of the same target according to confidence and category probability to obtain the best rectangular frame corresponding to the target, and then mark the target with abnormal placement status with a box on the original image according to the ratio relationship between the preprocessed image and the original image, and note the abnormal placement of the book on the box.

8. The method for using the automated book inventory and abnormal warning system according to claim 7 is characterized by: When the target barcode area in the local image is tilted at a certain angle, the optimal rectangular frame is rotated until the optimal rectangular frame completely covers the target barcode, and then the barcode is cropped from the original image.

9. The method for using the automated book inventory and abnormal warning system according to claim 2 is characterized by: Step S05 The recognition process of the barcode image by the ZXing library is as follows: S51, after receiving the barcode image segmented by the target detection model, the barcode recognition module pre-processes the input barcode image, converts it into a grayscale image, and then converts the grayscale image into a black and white image by binarization, so that the black and white stripes of the barcode are more obvious; S52, ZXing library uses edge detection algorithm to detect the edge in the black and white image. The black and white stripes of the barcode in the black and white image produce brightness changes. ZXing library analyzes the brightness changes of the black and white image during the detection process and determines the possible position of the barcode in the black and white image. S53. After determining the barcode area, the ZXing library analyzes the stripe features in detail and extracts the width information of each stripe. According to the combination of stripes of different widths, the ZXing library maps these combinations to corresponding characters, decodes them according to the barcode encoding rules, and outputs the decoded data to the library management system.

Images