Book self-service returning and sorting control system and method based on vision

Abstract

The invention discloses a self-service book returning and sorting control system and method based on vision, and relates to the technical field of self-service book returning and sorting control, and the system comprises a book returning tray device, a book front visual recognition device, a visual recognition control module, a book sorting and warehousing device and a book management server. Self-service book returning is achieved through the induction, recognition and sorting linkage closed-loop design, the book processing time consumption is greatly shortened, the queuing pressure of a library in the peak period is effectively relieved, and the workload of manual sorting is reduced. Through combination of book front and back image cross validation, the recognition accuracy is greatly improved; and the flexible mechanical clamping jaw is matched with the delta manipulator, so that zero damage during book grabbing and transferring is ensured, and the sorting error rate is greatly reduced. And due to the arrangement of the temporary book returning labels, workers can quickly obtain the returning information of the books and quickly carry out shelving operation in batches, the workload of shelving the books is greatly reduced, the accuracy rate of shelving the books is improved, and the equipment requirement for fully-automatic shelving of the books is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of book self-return sorting control, in particular to a book self-return sorting control system and method based on vision. BACKGROUND

[0002] With the continuous increase of library collection, the traditional manual book sorting method is inefficient, labor-intensive and prone to errors, which has been difficult to meet the needs of efficient management of modern libraries. Book self-return sorting control technology emerges as the times require, aiming to realize automatic sorting after book return and improve the circulation management efficiency of libraries. However, the existing book self-return sorting control technology still has some deficiencies in practical application, which limits its further development.

[0003] At present, book self-return sorting control technology mainly adopts bar code recognition, RFID (Radio Frequency Identification) technology, etc. Bar code recognition technology obtains book information by scanning the bar code on the book, and then realizes sorting. RFID technology uses radio frequency signals to realize non-contact information transmission and achieve identification through space coupling (alternating magnetic field or electromagnetic field). Many libraries have introduced self-return book systems based on RFID technology. When readers return books, the system can automatically read RFID tag information, classify books into corresponding categories, and automatically transport them to the vicinity of corresponding bookshelves.

[0004] Defects of bar code recognition technology: high requirements for the placement position of books and the angle of bar codes. In actual operation, it is difficult for readers to ensure that the bar code is presented to the scanning device at the right angle and position every time when returning books, which leads to identification failure or the need to adjust the book position multiple times, affecting the return efficiency. In addition, bar codes are prone to wear and tear, and once such a situation occurs, the scanning device may not be able to accurately identify the book information.

[0005] Defects of RFID technology: relatively high technology cost, each book needs to be pasted with an RFID tag, increasing the operating cost of the library, which is a heavy economic burden for some small or limited libraries. At the same time, RFID technology may have signal interference problems in complex environments, leading to tag reading errors. For example, when multiple books are placed closely together, tag signal conflict may occur, affecting the accuracy of identification. In addition, RFID technology involves readers' personal information and behavior data, and privacy protection issues have also attracted widespread attention. SUMMARY

[0006] In order to solve the above technical problems, the present application provides a book self-return sorting control system and method based on vision. The following technical solutions are adopted:

[0007] The visual-based book self-return sorting control system comprises a book return tray device, a book front visual identification device, a visual identification control module, a book sorting and storage device and a book management server, the book return tray device is used for sensing a book to be returned and taking a visual picture of the back of the book to be returned, the book front visual identification device is located above the book return tray device, and takes a visual picture of the front of the book to be returned after the book return tray device senses the book to be returned, the visual identification control module is in communication connection with the book return tray device and the book front visual identification device to collect back and front images of the book to be returned and perform visual analysis to obtain book information, the book sorting and storage device is installed on one side of the book return tray device, the visual identification control module generates book return information and book sorting instruction data based on the book information, the visual identification control module controls the book sorting and storage device to transfer the book to be returned to an instruction position based on the book sorting instruction data, and the visual identification control module interacts with the book management server to exchange the book return information.

[0008] Optionally, the book return tray device comprises a tray, a book back visual collection assembly and a pressure-sensitive sensor, the top of the tray is provided with a book collection piece installation slot, the pressure-sensitive sensor is installed at the book collection piece installation slot, the bottom of the book back visual collection assembly is installed at the book collection piece installation slot, and the pressure-sensitive sensor detects the weight change of the book back visual collection assembly after installation, the pressure-sensitive sensor is in communication connection with the visual identification control module, and when it is judged that the weight change value detected by the pressure-sensitive sensor is greater than a set weight threshold value, the visual identification control module controls the book back visual collection assembly and the book front visual identification device to take back and front images of the book to be returned respectively.

[0009] Optionally, the book back visual collection assembly comprises a box, a load-bearing transparent plate and a back visual module, the bottom of the box is installed at the book collection piece installation slot, the bottom surface of the load-bearing transparent plate is detachably installed at the top of the box, the load-bearing transparent plate is made of transparent material, and the top surface is used for bearing the book to be returned, the back visual module is installed in the box, a lens of the back visual module faces upwards to take a back visual picture of the book to be returned, and the back visual module is in communication interaction with the visual identification control module to exchange back visual picture data.

[0010] By adopting the above technical scheme, the user places the book to be returned on the book return tray device, and the book does not need to be intentionally aligned. When the pressure-sensitive sensor or the infrared sensor built in the book return tray device senses the weight or the blocking signal of the book, the system is triggered to start the work flow, and the book back visual collection assembly of the system is activated synchronously to take a picture of the back of the book (usually printed with an ISBN bar code, a two-dimensional code or a press mark, etc.) to obtain back image data.

[0011] The transparent carrier plate is made of transparent material (such as high-transparency acrylic and tempered glass), which not only serves as a carrier platform for books but also does not block the shooting path of the back visual module, ensuring that the camera module can clearly shoot the back of the book (especially the key information such as the barcode and QR code printed on the back) through the transparent plate. The box forms a relatively closed internal space, which can reduce the interference of external light on the back visual module; at the same time, the inside of the box can integrate a light supplement module (such as a LED light strip surrounding the camera module), which, combined with the design of the lens of the camera module facing upwards, can provide uniform and stable illumination for the back of the book, avoiding image blurring and information loss caused by insufficient or reflected light, and improving the accuracy of subsequent image recognition.

[0012] Integration of carrying and collecting: The transparent carrier plate directly serves as a book placement surface, and users only need to place the book on the plate to complete positioning, without the need for additional adjustment of the book angle, simplifying user operation; at the same time, the flat surface of the transparent plate can reduce the inclination or wrinkles of the book when placed, indirectly reducing the difficulty of image preprocessing.

[0013] The bottom surface of the transparent carrier plate can be detachably installed on the top of the box (such as fixed by buckles, magnetic attraction or screws), which can be quickly disassembled, replaced or cleaned when the surface of the transparent plate is scratched or stained due to long-term use, avoiding the decline of shooting quality caused by the pollution of the transparent plate, and prolonging the service life of the component.

[0014] The book front visual recognition device (such as a high-definition industrial camera above the tray, equipped with a light supplement lamp to ensure image clarity) is linked with the book return tray device, which starts simultaneously after the tray senses the book to shoot the front image of the book from above. The front image contains features such as cover patterns and book title, which can assist in confirming the identity of the book based on the back information (especially when the back barcode is blurred, the cover features can be used as supplementary identification basis).

[0015] The book front visual recognition device and the book return tray device transmit the front and back image data shot in real time to the visual recognition control module. The module is built-in with image preprocessing algorithms (such as deblurring, noise reduction and distortion correction) and feature recognition models (such as OCR text recognition, barcode / QR code decoding and image feature matching algorithm).

[0016] Prior analysis of back image: The ISBN number (core identifier) is extracted through barcode / QR code decoding technology, if the back information is blurred or missing, the front image is analyzed, the cover title and author information are recognized through OCR, and the unique identity information of the book (such as book title, ISBN, category and storage location) is finally determined by pattern matching with the book cover feature library (pre-stored in the local module or book management server).

[0017] The visual recognition control module generates two types of data based on the parsed book information:

[0018] Book return information: including book ISBN, return time, user ID (if associated with a borrowing card), etc., for synchronization to the book management server;

[0019] Book sorting instruction data: according to the book category (such as literature, science and technology) or the predefined storage location (such as a specific shelf number), a specific sorting path instruction is generated.

[0020] Subsequently, the visual recognition control module sends the sorting instruction data to the book sorting and storage device to control its action.

[0021] The visual recognition control module uploads the book return information to the book management server in real time, the server updates the book inventory status from borrowed to in stock, records the return time, and synchronizes to the library management system, ensuring that users and administrators can query the book status in real time. At the same time, the server can feedback abnormal information (such as the book has overdue, there is damage record, etc.) to the control module, and the control module can prompt the user through the display screen of the tray device.

[0022] Compared with traditional manual return or semi-automatic system, the system has the following significant advantages through the deep integration of visual recognition and automatic control:

[0023] Users can complete the book return by themselves without the intervention of administrators, solving the problem of queuing at the manual window during the peak period of the library and reducing the workload.

[0024] Combined with the cross verification of the front cover (cover features) and the back cover (barcode / QR code) dual visual information, even if the back cover barcode is worn or blurred, the identity can still be confirmed through the cover feature matching, the recognition accuracy is greatly improved; the algorithm of the visual recognition control module can adapt to complex scenes such as book tilt, wrinkles, and partial occlusion, without the need for users to deliberately align the books, reducing the user operation threshold.

[0025] The operation process is extremely simple, users do not need to manually scan the code or input information, and it is linked with the book management server in real time, which can prompt the user about the book status in time, reducing the user's subsequent query troubles.

[0026] Optionally, the side surface of the bearing type transparent plate and the box body is provided with a mechanical hand grabbing opening.

[0027] By adopting the above technical scheme, in order to cooperate with the book sorting and storage device to facilitate the removal of the book, the mechanical hand grabbing opening is provided, and the mechanical hand grabbing opening does not affect the structure and function of the bearing type transparent plate and the box body.

[0028] Optionally, the book sorting and storage device comprises a delta robot, a mechanical bionic gripper, and a plurality of book flow storage boxes, the delta robot is installed above the load-bearing transparent plate, the mechanical bionic gripper is installed at the movable end of the delta robot, and the plurality of book flow storage boxes are installed on the side of the book return tray device, and the visual recognition control module controls the delta robot and the mechanical bionic gripper to perform the book return action based on the book sorting instruction data.

[0029] Optionally, the book sorting and storage device further comprises an automatic labeler, which communicates and interacts with the visual recognition control module to print a temporary book return label based on the book return information.

[0030] By adopting the above technical scheme, the library scene is not suitable for using the conveyor belt robot to realize fully automatic book shelving operation. If it is necessary to completely cover each bookshelf, a large conveyor belt device is required, and each bookshelf is provided with a large mechanical hand, which is an unreasonable design. The book sorting and storage device only forms a temporary book return label on the surface of the returned book according to the book return information, and then classifies the returned book into the corresponding book flow storage box according to the book return information. For example, each book flow storage box corresponds to a bookshelf. The subsequent staff can pull the book flow storage box to the corresponding bookshelf for centralized shelving operation, and the temporary book return label can be removed after the shelving operation is completed.

[0031] The delta robot is a parallel robot, which has the characteristics of fast response speed and high repeat positioning accuracy, and is suitable for high-speed grabbing and transfer within a short distance. It is installed above the load-bearing transparent plate, can quickly cover the book placement area, and cooperates with the mechanical bionic gripper to complete accurate grabbing of the book.

[0032] The mechanical bionic gripper adopts a design similar to a human hand (such as flexible silicone fingertips and adjustable clamping force), which can stably grab books of different thicknesses and avoid excessive clamping force that causes book cover wrinkles and corner wear, thereby protecting the integrity of the book.

[0033] The plurality of book flow storage boxes are distributed on the side of the book return tray device, and different flow directions can be set according to book information (such as category, borrowing state, and storage area).

[0034] The automatic labeler prints a temporary book return label (which can include a two-dimensional code and a serial number) based on the book return information (such as book title, return time, and identification state) transmitted by the visual recognition control module, and completes the label taking and labeling action before transfer by the delta robot, so that the book can be scanned and tracked in the whole process of sorting, temporary storage, and final storage.

[0035] Optionally, the book front visual recognition device comprises a support and a front visual module, the front visual module is installed above the book return tray device through the support and avoids the moving range of the active end of the delta robot, and the front visual module is in communication connection with the visual recognition control module.

[0036] Optionally, the visual recognition control module comprises a control memory, a visual analysis chip, a data analysis chip, an instruction chip and a control chip, the memory is in communication connection with the pressure-sensitive sensor, the back visual module and the front visual module respectively, the visual analysis chip, the data analysis chip and the instruction chip are in communication connection with the memory respectively, the visual analysis chip analyzes the back image and the front image of the book to be returned based on a visual feature recognition algorithm to obtain book information, the data analysis chip is in communication connection with the visual analysis chip, generates book return information based on the book information, the instruction chip is in communication connection with the data analysis chip, generates a shooting action instruction of the back visual module and the front visual module based on the signal of the pressure-sensitive sensor, generates book sorting instruction data and label generation instruction based on the book return information, and the control chip is in communication connection with the instruction chip, controls the execution action of the back visual module and the front visual module based on the shooting action instruction respectively, and controls the execution action of the automatic label machine based on the label generation instruction.

[0037] By adopting the technical scheme, the visual analysis chip focuses on image feature extraction (such as OCR text recognition and barcode decoding), the data analysis chip is responsible for data correlation and decision (such as matching a book database and generating return information), and the instruction chip generates specific device control instructions. This division of labor avoids the serial processing bottleneck of a single processor, so that image analysis (such as cover title recognition), data verification (such as ISBN matching) and instruction issuing can be performed in parallel, and the system response speed is improved by more than 50% (compared with a traditional single-chip scheme).

[0038] The control memory serves as a data hub, buffers the trigger signal of the pressure-sensitive sensor and the back / front image data, and ensures the real-time data transmission between the chips. For example, the visual analysis chip can directly read the preprocessed image from the memory, reducing data transmission delay.

[0039] The instruction chip dynamically controls the shooting time of the back visual module and the front visual module based on the signal of the pressure-sensitive sensor (such as detecting that a book is placed), avoids invalid shooting, and reduces system power consumption.

[0040] The communication data between the chips can be stored in the memory to form a complete book return operation record chain, facilitating subsequent auditing or system optimization.

[0041] The visual-based book self-service return and sorting control method is used for controlling a visual-based book self-service return and sorting control system and comprises the following steps:

[0042] Step 1, place the book to be returned above the load-bearing transparent plate;

[0043] Step 2, the pressure-sensitive sensor interacts with the visual recognition control module to change the weight of the visual acquisition assembly on the back of the book, the visual recognition control module judges that the weight change value detected by the pressure-sensitive sensor is greater than the set weight threshold, and the visual recognition control module controls the visual acquisition assembly on the back of the book and the visual recognition device on the front of the book to shoot the back image and the front image of the book to be returned respectively;

[0044] Step 3, the visual analysis chip analyzes the back image and the front image of the book to be returned based on the visual feature recognition algorithm to obtain the book information, the data analysis chip generates the book return information based on the book information, and the instruction chip generates the book sorting instruction data and the label generation instruction based on the book return information;

[0045] Step 4, the control chip controls the automatic label machine to print a temporary return label and output based on the label generation instruction;

[0046] Step 5, the control chip controls the delta robot and the mechanical bionic gripper to perform the label picking and labeling action, and then performs the book returning action based on the book sorting instruction data;

[0047] Step 6, the visual recognition control module communicates with the book management server to interact the book return information.

[0048] Optionally, the label picking and labeling action is that the delta robot drives the mechanical bionic gripper to move to the label outlet of the automatic label machine to grab the temporary return label, and then drives the mechanical bionic gripper to move above the book to be returned to paste the temporary return label on the surface of the book to be returned;

[0049] The book returning action is that the delta robot drives the mechanical bionic gripper to move to one side of the mechanical hand grabbing opening and inserts into the mechanical hand grabbing opening to grab the book to be returned and put it into the corresponding book flow storage box.

[0050] In summary, the present application has the following at least one beneficial technical effect:

[0051] The present application can provide a visual-based book self-return sorting control system and method, which realizes book self-return through inductive recognition and sorting linkage closed loop design, greatly shortens the processing time of a single book, effectively relieves the queuing pressure during the peak period of the library, and reduces the manual sorting workload.

[0052] Combined with the cross verification of the front and back images of the book, even if the barcode is worn, the identity can still be confirmed through OCR and image matching, the recognition accuracy is greatly improved, the flexible mechanical gripper cooperates with the delta robot to ensure zero damage during book grabbing and transfer, and the sorting error rate is greatly reduced.

[0053] Readers do not need to deliberately align books or manually scan codes, and can complete the return by placing them randomly.

[0054] The setting of the temporary return label can enable the staff to quickly obtain the return information of the books, quickly and in batches perform shelving operations, greatly reduce the workload of book shelving, improve the accuracy of book shelving, and reduce the device demand of completely automatic book shelving. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 is the electrical device connection principle diagram of the visual-based book self-service return sorting control system of the present application;

[0056] Figure 2 is the structural principle diagram of the visual-based book self-service return sorting control system of the present application.

[0057] Figure 3 is the state structure diagram of the book sorting and warehousing device grabbing the books to be returned of the visual-based book self-service return sorting control system of the present application;

[0058] Figure 4 is the state structure diagram of the temporary return label pasted on the surface of the book to be returned of the present application;

[0059] Figure 5 is the electrical device connection principle diagram of the visual recognition control module of the visual-based book self-service return sorting control system of the present application;

[0060] BRIEF DESCRIPTION OF DRAWINGS: 11, tray piece;121, box body;122, load type transparent plate;123, back vision module;13, pressure sensitive sensor;21, support;22, front vision module;3, visual recognition control module;31, control memory;32, vision analysis chip;33, data analysis chip;34, instruction chip;35, control chip;41, delta robot;42, mechanical bionic clamping jaw;43, book flow storage box;44, automatic label machine;5, book management server;100, book to be returned. DETAILED DESCRIPTION

[0061] The present application will be further described in detail below in combination with the drawings.

[0062] The embodiment of the present application discloses a visual-based book self-service return sorting control system and method.

[0063] REFERENCE Figures 1-5, the embodiment 1, the visual-based book self-return sorting control system includes a book return tray device, a book front visual recognition device, a visual recognition control module 3, a book sorting and warehousing device, and a book management server 5. The book return tray device is used to sense a book to be returned 100 and take a visual picture of the back of the book to be returned 100. The book front visual recognition device is located above the book return tray device. When the book return tray device senses the book to be returned 100, the book front visual recognition device takes a visual picture of the front of the book to be returned 100. The visual recognition control module 3 is in communication connection with the book return tray device and the book front visual recognition device to collect the back image and the front image of the book to be returned 100 and perform visual analysis to obtain book information. The book sorting and warehousing device is installed on one side of the book return tray device. The visual recognition control module 3 generates book return information and book sorting instruction data based on the book information. The visual recognition control module 3 controls the book sorting and warehousing device to transfer the book to be returned 100 to an instruction position based on the book sorting instruction data. The visual recognition control module 3 interacts with the book management server 5 to obtain the book return information.

[0064] Optionally, the book return tray device includes a tray piece 11, a book back visual collection assembly, and a pressure-sensitive sensor 13. The top of the tray piece 11 is provided with a book collection piece installation slot. The pressure-sensitive sensor 13 is installed at the book collection piece installation slot. The bottom of the book back visual collection assembly is installed at the book collection piece installation slot. After installation, the pressure-sensitive sensor 13 detects the weight change of the book back visual collection assembly. The pressure-sensitive sensor 13 is in communication connection with the visual recognition control module 3. When it is judged that the weight change value detected by the pressure-sensitive sensor 13 is greater than a set weight threshold value, the visual recognition control module 3 controls the book back visual collection assembly and the book front visual recognition device to take the back image and the front image of the book to be returned 100, respectively.

[0065] Optionally, the book back visual collection assembly includes a box body 121, a load-bearing transparent plate 122, and a back visual module 123. The bottom of the box body 121 is installed at the book collection piece installation slot. The bottom surface of the load-bearing transparent plate 122 is detachably installed at the top of the box body 121. The load-bearing transparent plate 122 is made of transparent material. The top surface is used to carry the book to be returned 100. The back visual module 123 is installed in the box body 121. The lens takes a back visual picture of the book to be returned 100 upward. The back visual module 123 is in communication connection with the visual recognition control module 3 to interact with the back visual picture data.

[0066] The user places the book to be returned 100 on the book return tray device, and the book does not need to be intentionally aligned. The book return tray device is built-in with a pressure-sensitive sensor 13 or an infrared sensor. When the book weight or shielding signal is sensed, the system triggers the workflow to start and synchronously activates the book back visual acquisition component to take a photo of the back of the book (usually printed with the ISBN barcode, QR code or publisher logo, etc. Key information) to obtain the back image data.

[0067] The load-bearing transparent plate 122 is made of transparent material (such as high-transparency acrylic, tempered glass), which serves as a book carrying platform and does not block the shooting path of the back visual module 123, ensuring that the camera module can clearly shoot the back of the book (especially for the barcode, QR code, and other key information printed on the back) through the transparent plate. The box 121 forms a relatively closed internal space, which can reduce the interference of external light on the back visual module 123; at the same time, the inside of the box can integrate a light supplement module (such as a LED light strip surrounding the camera module), which, together with the design of the camera module lens facing upwards, can provide uniform and stable illumination for the back of the book, avoiding image blur and information loss caused by insufficient or reflected light, and improving the accuracy of subsequent image recognition.

[0068] Integration of carrying and collecting: The load-bearing transparent plate 122 directly serves as a book placement surface, and the user only needs to place the book on the plate to complete the positioning, without the need for additional adjustment of the book angle, simplifying the user operation; at the same time, the flat surface of the transparent plate can reduce the inclination or wrinkles of the book when placed, indirectly reducing the difficulty of image preprocessing.

[0069] The bottom surface of the load-bearing transparent plate 122 can be detachably installed on the top of the box 121 (such as fixed by buckles, magnetic attraction or screws), which can be quickly disassembled, replaced or cleaned when the transparent plate surface is scratched or stained due to long-term use, affecting the light transmission, avoiding the decline of shooting quality caused by the pollution of the transparent plate, and prolonging the service life of the component.

[0070] The book front visual recognition device (such as a high-definition industrial camera above the tray, equipped with a light supplement lamp to ensure image clarity) is linked with the book return tray device, and starts synchronously after the tray senses the book to shoot the front image of the book from above. The front image contains cover patterns, book title characters, etc. Features, which can assist in book identity confirmation with back information (especially when the back barcode is blurred, the cover features can be used as supplementary identification basis).

[0071] The book front visual recognition device and the book return tray device transmit the front and back image data shot in real time to the visual recognition control module 3. The module is built-in with image preprocessing algorithms (such as deblurring, noise reduction, distortion correction) and feature recognition models (such as OCR character recognition, barcode / QR code decoding, image feature matching algorithm);

[0072] Back image priority analysis: Extract the ISBN number (core identification) through barcode / QR code decoding technology. If the back information is blurred or missing, call the front image analysis, identify the cover title and author information through OCR, and match the pattern with the book cover feature library (pre-stored locally or in the book management server 5) to finally determine the unique identity information of the book (such as book title, ISBN, category, and storage location, etc.).

[0073] The visual recognition control module 3 generates two types of data based on the analyzed book information:

[0074] Book return information: including book ISBN, return time, user ID (if associated with a library card), etc., for synchronization to the book management server 5;

[0075] Book sorting instruction data: According to the book category (such as literature, science and technology) or the predefined storage location (such as a specific bookshelf number), generate specific sorting path instructions.

[0076] Subsequently, the visual recognition control module 3 sends the sorting instruction data to the book sorting and storage device to control its action.

[0077] The visual recognition control module 3 uploads the book return information to the book management server 5 in real time, and the server updates the book inventory status from borrowed to in stock, records the return time, and synchronizes to the library management system, ensuring that users and administrators can query the book status in real time. At the same time, the server can feedback abnormal information (such as the book has been overdue, has damage records, etc.) to the control module, which can prompt the user through the display screen of the tray device.

[0078] Compared with traditional manual return or semi-automatic system, the system has the following significant advantages through the deep integration of visual recognition and automatic control:

[0079] Users can complete the book return by themselves without the intervention of administrators, solving the problem of queuing at the manual window during the peak period of the library and reducing the workload.

[0080] Combined with the cross verification of the front cover (cover features) and the back cover (barcode / QR code) dual visual information, even if the back barcode is worn or blurred, the identity can still be confirmed through cover feature matching, and the recognition accuracy is greatly improved; The algorithm of the visual recognition control module can adapt to complex scenes such as book tilt, wrinkles, and partial obstruction, without the need for users to deliberately align the books, reducing the user operation threshold.

[0081] The operation process is extremely simple, without the need for users to manually scan the code or input information, and it can be linked in real time with the book management server, which can prompt the user about the book status in real time, reducing the user's subsequent query troubles.

[0082] Optionally, the side of the carrying transparent plate 122 and the box body 121 is provided with a mechanical hand grabbing opening.

[0083] In order to cooperate with the book sorting and storage device to conveniently take out the books, the mechanical hand grabbing opening is opened, and the mechanical hand grabbing opening does not affect the structure and function of the carrying transparent plate 122 and the box body 121.

[0084] Optionally, the book sorting and storage device comprises a delta mechanical hand 41, a mechanical bionic gripper 42, and a plurality of book flow storage boxes 43, the delta mechanical hand 41 is installed above the carrying transparent plate 122, the mechanical bionic gripper 42 is installed at the movable end of the delta mechanical hand 41, the plurality of book flow storage boxes 43 are respectively installed at the side of the book return tray device, and the visual identification control module 3 controls the delta mechanical hand 41 and the mechanical bionic gripper 42 to perform the book returning action based on the book sorting instruction data.

[0085] Optionally, the book sorting and storage device further comprises an automatic labeler 44, the automatic labeler 44 communicates and interacts with the visual identification control module 3 to print a temporary book returning label, and the delta mechanical hand 41 performs the label taking and labeling action before executing the instruction to the side of the mechanical hand grabbing opening.

[0086] In fact, the scene of the library is not suitable for adopting the conveying belt mechanical hand scheme to realize the fully automatic book shelving operation. If it is necessary to completely cover each bookshelf, a huge conveying belt device is needed, and each bookshelf is provided with a very large mechanical hand, which is an unreasonable design. The book sorting and storage device herein only forms a temporary book returning label on the surface of the book to be returned 100 according to the book returning information, and then puts the book to be returned 100 into the corresponding book flow storage box 43 according to the book returning information, for example, each book flow storage box 43 corresponds to a bookshelf. Subsequent workers can concentrate on the shelving operation by pulling the book flow storage box 43 to the corresponding bookshelf, and can take down the temporary book returning label after the shelving operation is completed.

[0087] The delta mechanical hand 41 belongs to a parallel robot, has the characteristics of fast response speed and high repeat positioning accuracy, and is suitable for high-speed grabbing and transfer within a short distance. It is installed above the carrying transparent plate 122, can quickly cover the book placing area, and cooperates with the mechanical bionic gripper 42 to complete the accurate grabbing of the books.

[0088] The mechanical bionic gripper 42 adopts a design similar to a human hand (such as flexible silicone fingertips and adjustable clamping force), which can stably grab books of different thicknesses and avoid excessive clamping force to cause wrinkles on the book cover and wear on the corners, thereby protecting the integrity of the books.

[0089] A plurality of book flow storage boxes 43 are distributed on the side of the book return tray device, and different flow directions can be set according to book information (such as category, borrowing state, and storage area).

[0090] The automatic label machine 44 prints a temporary book return label (which can include a two-dimensional code and a serial number) based on the book return information (such as book title, return time, and recognition state) transmitted by the visual recognition control module 3, and the delta robot 41 completes the actions of taking and labeling the label before transfer, so that the book can be scanned and tracked in the whole process of sorting, temporary storage, and final storage.

[0091] Optionally, the book front visual recognition device includes a support 21 and a front visual module 22, the front visual module 22 is installed above the book return tray device through the support, and avoids the activity range of the moving end of the delta robot 41, and the front visual module 22 is in communication connection with the visual recognition control module 3.

[0092] Optionally, the visual recognition control module 3 includes a control memory 31, a visual analysis chip 32, a data analysis chip 33, an instruction chip 34, and a control chip 35, the memory 31 is in communication connection with the pressure-sensitive sensor 13, the back visual module 123, and the front visual module 22 respectively, the visual analysis chip 32, the data analysis chip 33, and the instruction chip 34 are in communication connection with the memory 31 respectively, the visual analysis chip 32 analyzes the back image and the front image of the book to be returned 100 based on a visual feature recognition algorithm to obtain book information, the data analysis chip 33 is in communication connection with the visual analysis chip 32, generates book return information based on the book information, the instruction chip 34 is in communication connection with the data analysis chip 33, generates shooting action instructions for controlling the back visual module 123 and the front visual module 22 based on the signal of the pressure-sensitive sensor 13, generates book sorting instruction data and label generation instructions based on the book return information, and the control chip 35 is in communication connection with the instruction chip 34, the control chip 35 controls the execution actions of the back visual module 123 and the front visual module 22 based on the shooting action instructions respectively, and controls the execution action of the automatic label machine 44 based on the label generation instructions.

[0093] The visual analysis chip 32 focuses on image feature extraction (such as OCR text recognition and barcode decoding), the data analysis chip 33 is responsible for data correlation and decision (such as matching a book database and generating return information), and the instruction chip 34 generates specific device control instructions. This division of labor avoids the serial processing bottleneck of a single processor, so that image analysis (such as cover title recognition), data verification (such as ISBN matching), and instruction issuing can be performed in parallel, and the system response speed is improved by more than 50% (compared with a traditional single-chip solution).

[0094] The control memory 31 serves as a data hub, buffering the trigger signals of the pressure sensor 13, the back / front image data, and ensuring the real-time data transmission between the chips. For example, the vision analysis chip 32 can directly read the pre-processed image from the memory, reducing the data transmission delay.

[0095] The instruction chip 34 dynamically controls the shooting timing of the back vision module 123 and the front vision module 22 based on the signals of the pressure sensor 13 (such as detecting the placement of a book), avoiding invalid shooting and reducing system power consumption.

[0096] The communication data between the chips can be stored in the memory 31, forming a complete book return operation record chain, facilitating subsequent auditing or system optimization.

[0097] The vision-based book self-service return sorting control method is used to control the vision-based book self-service return sorting control system, comprising the following steps:

[0098] Step 1, placing the book to be returned 100 above the load-bearing transparent plate 122;

[0099] Step 2, the pressure sensor 13 interacts with the vision recognition control module 3 to change the weight of the book back vision acquisition assembly, and the vision recognition control module 3 judges that the weight change value detected by the pressure sensor 13 is greater than the set weight threshold, and the vision recognition control module 3 controls the book back vision acquisition assembly and the book front vision recognition device to shoot the back image and front image of the book to be returned 100 respectively;

[0100] Step 3, the vision analysis chip 32 analyzes the back image and front image of the book to be returned 100 based on the vision feature recognition algorithm to obtain the book information, the data analysis chip 33 generates the book return information based on the book information, and the instruction chip 34 generates the book sorting instruction data and the label generation instruction based on the book return information;

[0101] Step 4, the control chip 35 controls the automatic label machine 44 to print and output the temporary book return label based on the label generation instruction;

[0102] Step 5, the control chip 35 controls the delta robot 41 and the mechanical bionic gripper 42 to perform the label taking and labeling action, and then performs the book return action based on the book sorting instruction data;

[0103] Step 6, the vision recognition control module 3 communicates with the book management server 5 to interact the book return information.

[0104] Optionally, the taking and labeling action is that the delta robot 41 drives the mechanical bionic gripper 42 to move to the label outlet of the automatic labeler 44 to grab the temporary return label, and then drives the mechanical bionic gripper 42 to move above the to-be-returned book 100 to paste the temporary return label on the surface of the to-be-returned book 100.

[0105] The returning action is that the delta robot 41 drives the mechanical bionic gripper 42 to move to one side of the mechanical hand grabbing opening and inserts the mechanical bionic gripper 42 into the mechanical hand grabbing opening to grab the to-be-returned book 100 and put it into the corresponding book flow storage box 43.

[0106] The implementation principle of the present application is described below through specific examples:

[0107] Zhang Mou places three books on the load-bearing transparent plate 122 of the book return tray device in turn. The pressure-sensitive sensor 13 below the load-bearing transparent plate 122 detects the weight change (the weight of a single book is about 300g, which exceeds the set threshold of 100g) and sends a trigger signal to the visual recognition control module 3.

[0108] The book back visual acquisition assembly is started: the back visual module 123 (20 million pixel industrial camera) in the box 121 takes pictures of the backs of the three books (focuses on capturing the ISBN barcode area) through the transparent plate.

[0109] The book front visual recognition device is started: the front visual module 22 (16 million pixel camera with ring-shaped fill light) fixed on the bracket 21 takes pictures of the covers of the three books from above (including book title text and cover pattern).

[0110] The back image (containing blurred barcode) and the front image (containing book title text) are transmitted in real time to the control memory 31 of the visual recognition control module 3 for caching.

[0111] Book information analysis:

[0112] The visual analysis chip 32 calls the barcode decoding algorithm to analyze the back image:

[0113] The back barcodes of “YYY Collection” and “ZZZ Principle” are clear, and the ISBN numbers (format example: 978-7-XXXX-XXXX-X, 978-7-YYYY-YYYY-Y) are successfully decoded;

[0114] The back barcode of “XXX Introduction” is worn and the decoding fails, triggering front image analysis: the cover text “XXX Introduction” is recognized by OCR, and the book identity is confirmed by matching with the local cover feature library (pre-storing the color distribution and font features of the cover of this book).

[0115] Data analysis chip 33 generates book return information based on the recognition result: contains book category (A, B, C), return time (2025-07-2115:30:22), reader ID (associated with campus card automatic identification).

[0116] Temporary label printing: instruction chip 34 generates label generation instructions, and control chip 35 drives automatic label machine 44 to print 3 temporary return labels, which contain:

[0117] Serial number (such as 20250721001, 20250721002, 20250721003);

[0118] Book category (A, B, C);

[0119] Storage shelf position data;

[0120] Return time (synchronized with book return information).

[0121] Labeling and grabbing action:

[0122] Delta robot 41 receives sorting instructions and drives mechanical bionic gripper 42 to complete the operation:

[0123] Move to the outlet of automatic label machine 44 to grab the label, adjust the angle through flexible fingertips, and paste the label flat on the upper right corner of the cover of each book;

[0124] From the mechanical hand grab opening on the side of the bearing type transparent plate 122, clamp the books (clamping force is set to 5N to avoid damaging the pages), and respectively transport to the corresponding book flow direction storage box 43.

[0125] The visual recognition control module uploads the book return information to the library management server 5, and the server updates the inventory status and synchronizes it to the library OPAC system.

[0126] The library staff regularly push the book flow direction storage box 43 to the corresponding shelf area, read the temporary label serial number through the code gun, confirm that the book matches the shelf position, complete the shelving, and tear off the temporary label (the label can be recycled).

[0127] The above are preferred embodiments of the present application, not limited to the protection scope of the present application, therefore: any equivalent changes made according to the structure, shape, principle of the present application should be covered within the protection scope of the present application.

Claims

1. A vision-based self-service book return sorting control system, characterized by: The invention comprises a book return tray device, a book front visual recognition device, a visual recognition control module (3), a book sorting and warehousing device and a book management server (5), wherein the book return tray device is used to sense the book to be returned (100) and visually photograph the back of the book to be returned (100), the book front visual recognition device is located above the book return tray device, and when the book return tray device senses the book to be returned (100), it photographs the front of the book to be returned (100) for visual photography, and the visual recognition control module (3) is respectively connected to the book return tray device and the book front visual recognition device. The device and the book front visual recognition device are connected in communication to collect back images and front images of books to be returned (100), and perform visual analysis to obtain book information. The book sorting and warehousing device is installed on one side of the book return tray device. The visual recognition control module (3) generates book return information and book sorting instruction data based on the book information. The visual recognition control module (3) controls the book sorting and warehousing device to transfer the books to be returned (100) to the instruction location based on the book sorting instruction data. The visual recognition control module (3) exchanges book return information with a book management server (5).

2. The vision-based self-service book return sorting control system according to claim 1 is characterized by: The book return tray device comprises a tray member (11), a book back visual collection component and a pressure-sensitive sensor (13). The top of the tray member (11) is provided with a book collection component installation slot. The pressure-sensitive sensor (13) is installed at the book collection component installation slot. The bottom of the book back visual collection component is installed at the book collection component installation slot. After installation, the pressure-sensitive sensor (13) detects the weight change of the book back visual collection component. The pressure-sensitive sensor (13) is connected to the visual recognition control module (3) for communication. When it is judged that the weight change value detected by the pressure-sensitive sensor (13) is greater than a set weight threshold, the visual recognition control module (3) controls the book back visual collection component and the book front visual recognition device to respectively capture the back image and the front image of the book to be returned (100).

3. The vision-based self-service book return sorting control system according to claim 2 is characterized by: The book back visual acquisition component comprises a box (121), a load-bearing transparent plate (122) and a back visual module (123). The bottom of the box (121) is mounted on a book acquisition component mounting slot. The bottom surface of the load-bearing transparent plate (122) is detachably mounted on the top of the box (121). The load-bearing transparent plate (122) is made of a transparent material. The top surface is used to carry the to-be-returned book (100). The back visual module (123) is mounted in the box (121). The lens faces upward to capture the back visual image of the to-be-returned book (100). The back visual module (123) communicates with the visual recognition control module (3) to exchange back visual image data.

4. The vision-based self-service book return sorting control system according to claim 3 is characterized by: Manipulator grabbing openings are provided on the sides of the load-bearing transparent plate (122) and the box body (121).

5. The vision-based self-service book return sorting control system according to claim 4 is characterized in that: The book sorting and warehousing device comprises a delta manipulator (41), a mechanical bionic gripper (42), and a plurality of book flow storage boxes (43). The delta manipulator (41) is installed above a load-bearing transparent plate (122), the mechanical bionic gripper (42) is installed at the movable end of the delta manipulator (41), and the plurality of book flow storage boxes (43) are respectively installed on the sides of a book return tray device. A visual recognition control module (3) controls the delta manipulator (41) and the mechanical bionic gripper (42) to perform a book return action based on book sorting instruction data.

6. The vision-based self-service book return sorting control system according to claim 5 is characterized by: The book sorting and warehousing device also includes an automatic labeling machine (44), which communicates with the visual recognition control module (3) to exchange book return information and prints a temporary book return label. The delta manipulator (41) first performs a label removal and labeling action before executing an instruction to the manipulator to grasp the opening side.

7. The vision-based self-service book return sorting control system according to claim 6 is characterized by: The book front visual recognition device comprises a bracket (21) and a front visual module (22). The front visual module (22) is installed above the book return tray device through the bracket and avoids the range of movement of the active end of the delta manipulator (41). The front visual module (22) is communicatively connected with the visual recognition control module (3).

8. The vision-based self-service book return sorting control system according to claim 7 is characterized by: The visual recognition control module (3) includes a control memory (31), a visual analysis chip (32), a data analysis chip (33), an instruction chip (34) and a control chip (35), wherein the memory (31) is respectively connected to the pressure sensitive sensor (13), the back visual module (123) and the front visual module (22), and the visual analysis chip (32), the data analysis chip (33) and the instruction chip (34) are respectively connected to the memory (31), and the visual analysis chip (32) analyzes the back image and the front image of the to-be-returned book (100) based on the visual feature recognition algorithm to obtain book information, and the data analysis chip (33) is connected to the visual sensor (13), the back visual module (123) and the front visual module (22). The analysis chip (32) is in communication connection with the data analysis chip (33), and generates book return information based on the book information. The instruction chip (34) is in communication connection with the data analysis chip (33), and generates shooting action instructions for controlling the back visual module (123) and the front visual module (22) based on the signal of the pressure sensitive sensor (13), and generates book sorting instruction data and label generation instructions based on the book return information. The control chip (35) is in communication connection with the instruction chip (34), and the control chip (35) controls the execution actions of the back visual module (123) and the front visual module (22) based on the shooting action instructions, and controls the execution action of the automatic labeling machine (44) based on the label generation instructions.

9. A vision-based self-service book return sorting control method, characterized by: The method for controlling the vision-based self-service book return sorting control system according to claim 8 comprises the following steps: Step 1: placing the book to be returned (100) on the load-bearing transparent plate (122); Step 2: The pressure-sensitive sensor (13) communicates the weight change of the visual collection component of the back of the book to the visual recognition control module (3); the visual recognition control module (3) determines that the weight change value detected by the pressure-sensitive sensor (13) is greater than a set weight threshold, and the visual recognition control module (3) controls the visual collection component of the back of the book and the visual recognition device of the front of the book to be returned to take a back image and a front image respectively; Step 3: The visual analysis chip (32) analyzes the back image and the front image of the to-be-returned book (100) based on a visual feature recognition algorithm to obtain book information, the data analysis chip (33) generates book return information based on the book information, and the instruction chip (34) generates book sorting instruction data and label generation instructions based on the book return information; Step 4: The control chip (35) controls the automatic labeling machine (44) to print and output a temporary book return label based on the label generation instruction; Step 5, the control chip (35) first controls the delta manipulator (41) and the mechanical bionic gripper (42) to perform the label taking and labeling action based on the book sorting instruction data, and then performs the book returning action; Step 6: The visual recognition control module (3) communicates with the book management server (5) to exchange book return information.

10. The vision-based self-service book return sorting control method according to claim 9, characterized in that: The label taking and labeling action is that the delta manipulator (41) drives the mechanical bionic gripper (42) to move to the label outlet of the automatic labeling machine (44) to grab the temporary book return label, and then drives the mechanical bionic gripper (42) to move above the to-be-returned book (100) to attach the temporary book return label to the surface of the to-be-returned book (100); The book returning action is that the Delta manipulator (41) drives the mechanical bionic gripper (42) to move to the side of the manipulator grasping opening, and inserts the manipulator grasping opening to grasp the book (100) to be returned and puts it into the corresponding book flow storage box (43).

Images