Book management system and book management method

Abstract

To provide a book collection management system capable of efficiently inspecting book collections. [Solution] This is a book management system that includes a mobile camera that moves around the library and photographs books on the bookshelves, an analysis device that analyzes the photographed images obtained from the mobile camera, including the spines of the books, and a management device that manages information about the library's book collection.The analysis device detects the spines of the books included in the photographed images through image analysis, extracts the text information on the spine of each book through OCR processing of the spine image, creates an identification database in which the images of the book's spine and the text information on the spine of that book are registered in correspondence with each other, extracts features of the spine image of each book using an AI model and stores the features in correspondence with each book.The management device obtains information about each book, including at least the image of the spine of each book and the text information on the spine, and uses the obtained information to create a management database for book collection management.

Description

[Technical Field]

[0001] The present invention relates to a book collection management system and a book collection management method for managing a library's book collection. [Background technology]

[0002] Traditionally, libraries have conducted periodic inventory checks to confirm the availability of books. During inventory checks, barcodes or IC tags attached to books are manually read to obtain information about the books, and books are checked to see if they are missing or in the correct location on the shelves.

[0003] Library inspections are often conducted over a 10-day closure period, during which the library is closed. The larger the number of books in a library, the greater the workload.

[0004] For this reason, systems for efficiently inspecting books have been proposed (for example, Patent Document 1). The system in Patent Document 1 discloses that it analyzes images of a portion of a bookshelf to extract information about the books, and compares this information with book collection data, which is a list of the library's books, to manage the books. Since it is possible to extract information about multiple books at once from the captured image, it is said that it is possible to efficiently inspect books. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-77368 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the above system requires at least a database of the library's book collection as information to be used for matching. Furthermore, the above system relies on a photographing method in which workers use terminal devices such as tablets, which takes time and must be carried out when the library is closed and there are no users. Furthermore, Patent Document 1 also mentions that a robot equipped with a terminal device may be used, but does not specifically disclose how to do this.

[0007] Therefore, an object of the present invention is to provide a book management system that enables efficient and accurate book inspection. [Means for solving the problem]

[0008] The embodiments of the present application have been made to solve the above problems, and the gist of the embodiments is as follows.

[0009] (1) a mobile imaging device that is equipped with an imaging device and moves around the library to photograph books on the bookshelf; an analysis device that analyzes the captured image including the spine of the book captured by the mobile image capture device; A book management system including a management device for managing information about a library's book collection, The analysis device A process of acquiring photographed images of books on the bookshelf from the mobile photographing device; A process of detecting the spines of the books included in the photographed images by image analysis, and extracting text information from the spines of the books by OCR processing of the detected spine images; A process of creating an identification database in which the image of the spine of the book and the character information of the spine of the book are registered in association with each other; extracting features of the spine image of each book using an AI model, and storing the information on the features in association with each book; A book management system characterized in that the management device obtains information about each book, including at least an image of the spine of each book and text information about the spine, from the analysis device, and uses the obtained information to create a management database for book management.

[0010] (2) The library management system described in (1) above, characterized in that the mobile imaging device is an autonomously mobile device of the wheeled type or drone type.

[0011] (3) The book management system described in (1) above, characterized in that the mobile photographing device moves to the location of the target bookshelf by referring to a library map created in advance and photographs the book.

[0012] (4) The book management system described in (1) above is characterized in that the analysis device compares the extracted spine text information with the library's existing book database, extracts information corresponding to the book having the spine text information from the existing book database, and incorporates the extracted information into the identification database.

[0013] (5) When inspecting the library's books after creating the management database, the mobile camera photographs the bookshelf and transmits the obtained inspection photographed image to the analysis device; The analysis device uses the AI ​​model to analyze the image of the spine of the book to be compared that is included in the transmitted inspection photograph image, and uses the stored feature amount to compare it with any spine image registered in the identification database to identify the book to be compared; and transmitting to the management device information on the book inventory inspection result, the information including information indicating that the identified book was present on the bookshelf that was inspected. The book management system according to (1) above, wherein the management device uses the book inspection result information sent to register the book inspection result in the management database.

[0014] (6) The book management system according to (5) above, wherein the mobile photographing device photographs the bookshelves for book inspection purposes during library closing hours.

[0015] (7) a mobile imaging device that is equipped with an imaging device and moves around the library to photograph books on the bookshelf; an analysis device that analyzes the captured image including the spine of the book captured by the mobile image capture device; A book management method executed in a book management system including a management device that manages information about a library's book collection, Acquire a photographed image of a book on the bookshelf; Detecting the spines of the books included in the photographed image by image analysis, and extracting text information from the spines of the books by OCR processing of the detected spine images; creating an identification database in which the image of the spine of the book and the character information of the spine of the book are registered in association with each other; Extracting features of the spine image of each book using an AI model, and storing the information of the features in association with each book; A book management method characterized by acquiring information about each book including at least an image of the spine of each book and text information on the spine, and using the acquired information to create a management database for book management.

[0016] (8) The book collection management method described in (7) above, characterized in that the mobile imaging device is an autonomously mobile device of the wheeled type or drone type.

[0017] (9) The book collection management method according to (7) above, characterized in that the mobile photographing device moves to the location of the target bookshelf by referring to a library map created in advance and photographs the book.

[0018] (10) A book management method as described in (7) above, characterized in that the extracted text information on the spine is compared with the library's existing book database, information corresponding to the book having the text information on the spine is extracted from the existing book database, and the extracted information is incorporated into the identification database.

[0019] (11) When inspecting the library's books after creating the management database, the bookshelf is photographed and the obtained inspection photograph image is transmitted; Using the AI ​​model, an image of the spine of the book to be compared that is included in the transmitted inspection photograph image is analyzed, and the image is compared using the stored feature amount to determine which spine image registered in the identification database matches, thereby identifying the book to be compared; Transmitting book inspection result information including information indicating that the identified book was present on the inspected bookshelf; The book collection management method according to (7) above, characterized in that the book collection inspection result is registered in the management database using the book collection inspection result information transmitted.

[0020] (12) The book collection management method according to (11) above, characterized in that the bookshelves are photographed for book collection inspection during library closing hours. [Effects of the Invention]

[0021] According to one aspect of the embodiment, it is possible to provide a book management system that is capable of efficiently and accurately inspecting book collections. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a schematic diagram of a book collection management system according to an embodiment of the present invention; [Figure 2] FIG. 1 is a diagram illustrating an example of a mobile imaging device. [Figure 3] FIG. 10 is a diagram illustrating an example of a data structure of an identification database. [Figure 4] FIG. 2 illustrates an example of a data structure of a management database. [Figure 5]10 is a flowchart showing the flow of preparation processing by the library management system. [Figure 6] 10A and 10B are diagrams illustrating an example of a method for photographing books on a bookshelf with a camera of a mobile photographing device. [Figure 7] 10 is a flowchart showing the flow of a book collection inspection process by the book collection management system. DETAILED DESCRIPTION OF THE INVENTION

[0023] An embodiment of a book management system and a book management method will be described with reference to the drawings. The book management system according to this embodiment analyzes images taken by an autonomously moving mobile camera using an AI model and performs processing to identify books, thereby conducting book inspections and managing book collections in libraries and the like.

[0024] FIG. 1 is a schematic diagram of a book collection management system 1 of this embodiment. The book collection management system 1 includes a mobile camera device 100, an analysis device 200, and a management device 300, which are communicatively connected to one another via a network 50. A plurality of mobile camera devices 100 may be used in the book collection management system 1. The mobile camera device 100 is placed in the library where the inspection is performed. The analysis device 200 and the management device 300 may be placed in the library where the book collection inspection is performed, or may be placed outside the library. Furthermore, the analysis device 200 and the management device 300 may be configured as a single device having the respective functions.

[0025] (Mobile camera) The mobile camera 100 is a device that moves around a library and photographs books on a bookshelf (shelf). FIG. 2 shows an example of the configuration of the mobile camera 100. FIG. 2 shows the mobile camera 100 photographing a bookshelf, viewed from the side of the bookshelf. The mobile camera 100 is an autonomously moving robot (device) equipped with an imaging device. In this embodiment, the mobile camera 100 is, as an example, a wheeled device. The mobile camera 100 is equipped with a control unit 102, a memory unit 104, a communication unit 106, a camera 110, a traveling unit 120, etc.

[0026] The control unit 102 controls various operations and processes executed by the mobile image capture device 100. The control unit 102 includes a processor, a memory, and the like. The processor is a circuit that executes control programs stored in the storage unit 104 or the like to perform various processes. In this embodiment, the processor executes processes such as photographing bookshelves in a library, moving the mobile image capture device 100 itself to a predetermined position for image capture, and transmitting captured images to an external device. The processor is, for example, a central processing unit (CPU) or a micro processing unit (MPU). The memory is, for example, a semiconductor memory, such as a random access memory (RAM) that provides a temporary working area for the processor or a read only memory (ROM) that stores a control program for the processor. The control unit 102 may also include an application specific integrated circuit (ASIC) that executes some or all of the functions realized by executing a program.

[0027] The memory unit 104 stores an OS and other control programs for controlling the mobile image capture device 100. The memory unit 104 also stores images captured by the camera 110. The memory unit 104 can also store library map information that is used when the mobile image capture device 100 moves to capture images of books on the bookshelves. The memory unit 104 can also externally acquire and store the details of the book inspection process (information specifying the bookshelves to be photographed, the time of the inspection, etc.).

[0028] The storage unit 104 may be, for example, a hard disk drive (HDD) or other magnetic storage device, an optical storage device, a semiconductor storage device such as a flash memory, or any combination of these.

[0029] Communication unit 106 is a communication interface for communicating with the outside via network 50. Communication unit 106 performs communication, for example, when transmitting an image captured by camera 110 to analysis device 200. Communication unit 106 is connected to network 50 wirelessly or via a wire.

[0030] The camera 110 is an imaging device, and in this embodiment, is used to capture images of books on bookshelves in a library. More specifically, the camera 110 captures images of books arranged on the bookshelves and generates a captured image including multiple spines.

[0031] There are no particular limitations on camera 110 as long as it includes an image sensor such as a CCD (Charge Coupled Device) and an imaging optical system such as a lens. It is preferable that camera 110 captures color images.

[0032] As shown in FIG. 2, camera 110 is preferably adjustable in height so that it can capture every shelf in the library, from the top shelf to the bottom shelf. For example, camera 110 may have a mechanism for sliding up and down, a neck supporting camera 110 that extends and retracts, or other parts of the body that can raise and lower to change the height of camera 110. Image capture-related processes, such as image capture by camera 110 and adjustment of the image capture position, may be controlled by control unit 102. Instead of or in addition to changing the height, a mechanism for changing the image capture direction of camera 110 may be provided so that each shelf can be captured. Camera 110 does not need to be adjustable in height or direction; it may be a fixed camera if it is a wide-angle, high-resolution camera capable of capturing images of the spines of each book that can be subjected to OCR processing, machine learning, image matching, and other processes, as described below.

[0033] The running unit 120 is a means for moving the mobile imaging device 100. The running unit 120 has wheels, a motor for driving the wheels, and the like. The running unit 120 preferably has wheels that allow it to move freely within the library. Furthermore, the running unit 120 may have other running means, such as caterpillars, in addition to or instead of wheels.

[0034] The traveling unit 120 is controlled by the control unit 102. The control unit 102 controls the traveling unit 120 to move the mobile imaging device 100 itself to a predetermined imaging position. The control unit 102 also controls the traveling unit 120 based on the image of the camera 110 and detection signals from sensors for detecting other obstacles, to make the traveling unit 120 travel without hitting any obstacles.

[0035] Although the mobile image capturing device 100 is a wheeled device that moves on wheels, it is not limited to this and may be any device that has a camera 110, moves autonomously, and can capture images of books on a bookshelf. For example, the mobile image capturing device 100 may be a drone-type or walking-type autonomously moving device.

[0036] The mobile camera device 100 can operate by receiving inputs for operations and settings related to the preparation process for introducing the system to a library and the subsequent book collection inspection process from an external computer with which it communicates or is directly connected via the network 50. Specifically, the mobile camera device 100 receives instructions to specify the bookshelves to be photographed in the preparation process, instructions to carry out a book collection inspection, and instructions to specify the bookshelves to be inspected. The external computer that issues instructions to the mobile camera device 100 may be an analysis device 200 or a management device 300, or may be another computer, tablet terminal, smartphone, etc., as long as it has software installed to operate the mobile camera device 100. This concludes the configuration of the mobile camera device 100.

[0037] (Analysis device) Next, analysis device 200 is a server that analyzes the captured image including the spine of a book captured by mobile image capture device 100 and performs processing to identify the book included in the captured image. Analysis device 200 may be configured as a server computer or may be a general-purpose computer. Analysis device 200 includes control unit 202, storage unit 204, communication unit 206, etc. Analysis device 200 may further include an operation input unit such as a keyboard, a display unit (monitor), etc.

[0038] The control unit 202 controls various operations and processes executed by the analysis device 200. The control unit 202 includes a processor 202a and a memory 202b. The processor 202a is a circuit that executes control programs stored in the storage unit 204 or the like to perform various processes. The processor 202a is, for example, a central processing unit (CPU) or a micro processing unit (MPU). The memory 202b is, for example, a semiconductor memory, such as a random access memory (RAM) that provides a temporary working area for the processor or a read only memory (ROM) that stores the control program for the processor. The control unit 202 may include an application specific integrated circuit (ASIC) that implements some or all of the functions implemented by executing programs stored in the memory 202b or the like.

[0039] The storage unit 204 stores an OS and other control programs for controlling the processing of the analysis device 200. The storage unit 204 also stores an AI model 210, an identification database (hereinafter referred to as an identification DB) 220, an OCR program 230, a DB matching program 240, etc. The storage unit 204 may be, for example, a hard disk drive (HDD) or other magnetic storage device, an optical storage device, a semiconductor storage device such as a flash memory, or any combination of these.

[0040] The AI ​​model 210 analyzes captured images of the bookshelf, classifies and detects the spine of each book, and compares the detected spine image with any of the multiple spine images registered in the identification DB 220. To perform the comparison process, the AI ​​model 210 first extracts features from the spine image captured during preparation for installing the book management system 1. During book inventory inspection processing by the system 1, the AI ​​model 210 compares the spine image to be compared, captured during inspection, with the multiple registered spine images using the features, finds a matching image, and identifies which book the image to be compared is. The AI ​​model 210 may be a trained AI model trained using spine images. The AI ​​model 210 may be trained to recognize and detect (classify) the spine area of ​​each book from the captured image.

[0041] In this embodiment, the process of detecting the spine from the captured image and the process of matching the spine image to be compared with registered spine images are described as being performed by a single AI model 210, but the analysis device 200 may have separate AI models corresponding to each process.

[0042] The identification DB 220 is a database used to identify books to be inspected during the book collection inspection process. The identification DB 220 is created during the preparation process when the system is introduced. The identification DB 220 is a database in which at least images of the spine of each book photographed during the preparation process and text information extracted from the spine images are registered in association with each other.

[0043] Fig. 3 shows an example of the data structure of the identification DB 220. The identification DB 220 shown in Fig. 3 includes image extraction information 220a and related information 220b.

[0044] The image extraction information 220a is a spine image, character information extracted from the spine image, and feature values ​​extracted from the spine image by the AI ​​model 210, and is information obtained by analyzing images captured during preparation processing. The spine image is an image file used for comparison during inspection, and the file format is not particularly limited, and JPEG, TIFF, GIF, etc. may be used. The spine image may also be a link to the image file.

[0045] The extracted character information is the book title extracted from the spine image. The title character information is obtained by OCR (Optical Character Recognition) processing of the spine image. The extracted character information can be used to identify the book corresponding to the spine image to be registered. The extracted character information can also be used to match the identification DB 220 with the library's existing book database. The extracted character information may include the author name and publisher name printed on the spine in addition to the book title. If a label displaying a call number or the like is attached to the spine, the extracted character information may also include the character information of the label.

[0046] The related information 220b is various information about each book extracted from the library's existing book database, etc. The existing book database is a database in which, for example, the title, author name, publisher, ISBN, etc. are associated with each book management number in the library, and is used for book collection management and lending management. Furthermore, the existing book database is not limited to the library's book collection database, and an external database may also be used.

[0047] By performing a matching process with an existing book database based on the extracted character information, it is possible to register the image extraction information 220a and related information 220b in association with each other. Note that if there is no existing book database or if an existing book database is not used, the related information 220b does not need to be registered. In that case, the identification DB 220 only needs to be composed of the image extraction information 220a.

[0048] Next, the OCR program 230 is a program for realizing an OCR processing function that extracts character information from the spine image of a book when executed by the control unit 202. As the OCR program 230, a known program can be used.

[0049] The DB matching program 240 is a program executed by the control unit 202 to perform a process of matching character information (book titles) extracted from the spines of photographed images of a bookshelf with the library's existing book database. If matching processing is not performed, the storage unit 204 does not need to store the DB matching program 240.

[0050] The communication unit 206 is a communication interface for communicating with the outside via the network 50, and performs communication, for example, when receiving captured images from the mobile imaging device 100 or when transmitting inspection result information to the management device 300 after performing a library inspection process. The communication unit 206 is connected to the network 50 wirelessly or via a wired connection. The above is the configuration of the analysis device 200.

[0051] (Management device) Next, the management device 300 is a server that records and manages the results of book inspections for each book when a book inspection is performed in the book management system 1. The management device 300 can manage the results of the book inspection in association with various other information about each book. The management device 300 may be configured as a server computer or may be a general-purpose computer. The management device 300 includes a control unit 302, a memory unit 304, a communication unit 306, etc. The management device 300 may further include an operation input unit such as a keyboard, a display unit (monitor), etc.

[0052] The control unit 302 controls various operations and processes executed by the management device 300. The control unit 302 includes a processor 302a and a memory 302b. The processor 302a is a circuit that executes control programs stored in the storage unit 304 or the like to perform various processes. The processor 302a is, for example, a central processing unit (CPU) or a micro processing unit (MPU). The memory 302b is, for example, a semiconductor memory, such as a random access memory (RAM) that provides a temporary working area for the processor or a read only memory (ROM) that stores the processor's control program. The control unit 302 may include an application specific integrated circuit (ASIC) that implements some or all of the functions implemented by executing programs stored in the memory 302b or the like.

[0053] The storage unit 304 stores an OS and other control programs for controlling the processing of the management device 300. The storage unit 304 also stores a management database (hereinafter referred to as a management DB) 320. The storage unit 304 may be, for example, a hard disk drive (HDD) or other magnetic storage device, an optical storage device, a semiconductor storage device such as a flash memory, or any combination of these.

[0054] The management DB 320 is a database that registers the results of book inspections for each book and manages the location of the books. Information about each book and the book inspection result information for that book are registered in association with each other in the management DB 320. An example of the data structure of the management DB 320 is shown in FIG. 4. For example, the management DB 320 registers information about each book for each library management number assigned to each book. The book information includes spine images of each book captured during preparation processing when the system is installed, as well as the book title, author name, publisher name, and ISBN code extracted from the spine image. Information about the book's lending status may also be associated with the information.

[0055] The management DB 320 can be created by combining information from an existing book database used for book collection management with the spine image and spine text information acquired from the analysis device 200 during the preparation process. Also, if information from an existing book database is incorporated into the identification DB 220, the information from that identification DB 220 may be acquired from the analysis device 200 and used as is as the management DB 320. If there is no existing book collection database, it is sufficient that at least the spine image and extracted text information are associated with book collection inspection result information and registered in the management DB 320.

[0056] Book inspection result information is information such as the presence or absence of books confirmed in the book inspection process by the book management system 1, and the date and time of the inspection. As inspection result information, any information necessary can be registered as appropriate. For example, a bookshelf number may be recorded as information specifying the location where the book was confirmed. Book inspection result information may also be registered multiple times. Figure 4 shows an example in which the latest inspection result is registered as inspection result 1, and the previous inspection result is registered as inspection result 2, in that order.

[0057] The information in the management DB 320 can be viewed on a display unit connected to the management device 300. The information may also be viewed from an external computer connected to the management DB 320 via the network 50. The above is the configuration of the management device 300.

[0058] The network 50 may be a wired or wireless network such as the Internet, a local area network (LAN), a wide area network (WAN), or a mobile communication network.

[0059] (Book collection management method) Next, we will explain the book collection management method performed by the book collection management system 1. The book collection management method includes a preparation process and a book collection inspection process that is performed after the preparation process is completed.

[0060] (Preparation process) First, we will explain the flow of the preparation process performed when introducing the book collection management system 1 to a library. The preparation process includes a process of photographing the library's bookshelves to acquire spine images of the books to be managed, a process of extracting text information about the spine from the spine image, associating the spine image with the text information and registering it in a database (identification DB 220), and a process of having AI model 210 extract features from the acquired spine image. Figure 5 shows a flowchart explaining the flow of the preparation process.

[0061] The mobile camera 100 photographs all bookshelves that are to be managed by this system, and acquires photographed images (step 101). Fig. 6 is a diagram illustrating an example of a method for photographing with the camera 110 of the mobile camera 100. Fig. 6 shows a bookshelf 10 that is part of a library's bookshelves. A plurality of books 20 are stored on each shelf of the bookshelf 10. The mobile camera 100 photographs the spines of the books on the bookshelf while moving the photographing range 110a of the camera 110. For example, as shown in Fig. 6, the camera 110 starts photographing from the top left of the bookshelf 10, photographs the top shelf while moving to the right, then moves down to the lower shelf and photographs from left to right in the same way. This operation is repeated to photograph the bookshelf 10. Once photographing of the bookshelf 10 is complete, the camera moves to the next bookshelf and photographs again in the same way. The order and route for photographing the spines are not limited to the above method, and can be done in any order.

[0062] The position of each bookshelf to be photographed in the library and the order in which the photographs will be taken can be preset for the mobile camera device 100 using operating software on an external computer or the like. Specifically, a map of the library is registered in the storage unit 104 of the mobile camera device 100. A user of the present system 1 then sets the position of each bookshelf on the map in advance. The control unit 102 then controls the traveling unit 120 of the mobile camera device 100, allowing it to move to the position of the target bookshelf and take photographs while referring to the map. Furthermore, if the order in which each bookshelf will be photographed is specified in advance, the mobile camera device 100 can move to each bookshelf while referring to the map and take photographs in the specified order.

[0063] The map may be created automatically by the mobile image capture device 100 traveling autonomously and detecting obstacles such as shelves using cameras and sensors. Photographing each bookshelf in the preparation process can also be performed manually by a user of the system operating the mobile image capture device 100. The mobile image capture device 100 can also store information such as the location of the captured image and the date and time of the image capture in association with the captured image. The shelf number assigned to the photographed bookshelf or the location information on the map can be used as the location information.

[0064] Next, the mobile image capturing device 100 transmits information such as the captured images to the analysis device 200 (step 102). The transmitted captured images are multiple image files obtained by capturing images of all bookshelves that are the subject of book inventory inspection in the book collection management system 1. Furthermore, information such as location information and the date and time of capture may be transmitted in association with each captured image, as needed.

[0065] Next, the analysis device 200 acquires captured images of the books on the bookshelf from the mobile image capture device 100 and detects the spines of each book in the image (step 103). Specifically, as shown in FIG. 6, the captured image (capture range 110a) may include the spine area, the shelf area, and the like. The control unit 202 of the analysis device 200 performs image analysis to recognize the spine area in the image, detects and classifies each spine portion for each book, and generates a spine image for each book. This process is executed by the control unit 202 of the analysis device 200 performing image recognition processing using the AI ​​model 210 stored in the storage unit 204.

[0066] Next, the analysis device 200 performs a process to extract character information from the spine image of each book (step 104). Specifically, the control unit 202 executes the OCR program 230 stored in the storage unit 204, performs OCR processing on the spine image of each book, and extracts character information printed on the spine. The extracted character information needs to include at least the book title, but may also include other character information printed on the spine, such as the author name or publisher name. Furthermore, if a label is affixed to the spine and that information is also used, the extracted character information may include the character information on the label.

[0067] The analysis device 200 performs a process to extract features from the spine image of each book (step 105). Specifically, the control unit 202 causes the AI ​​model 210 to analyze each spine image and execute a process to extract features. The features are used by the AI ​​model 210 when it identifies which book the spine image input during the book collection inspection process belongs to. The features may be any information that the AI ​​model 210 can use to identify images, and it is sufficient if the AI ​​model 210 itself extracts appropriate features. Of course, the features may be automatically extracted by the AI ​​model 210 and set by a system administrator in combination. For example, the features may be automatically extracted by the AI ​​model 210 and then modified by the administrator. The order of steps 105 and 104 does not matter, and they may be performed in parallel.

[0068] Next, analysis device 200 creates identification DB 220 by associating the spine image generated by the processing of step 103, the text information on the spine of the book generated in step 104, and the feature amounts generated in step 105 (step 106). At this step, identification DB 220 is composed of spine images, text information on the spine corresponding to the images (titles, etc.), and feature amounts. Note that the feature amounts may be managed in a separate database or stored in a predetermined storage area for AI model 210, as long as they are linked to information that identifies each book.

[0069] Next, the analysis device 200 performs a process of matching the identification DB 220 created in step 106 with the library's existing book database (step 107). Specifically, based on the spine character information extracted by the OCR process, the existing book database is matched, and information (records) corresponding to the spine character information (of books having the spine character information) is extracted from the existing book database and incorporated into the identification DB 220. The information extracted from the existing book database corresponds to the related information 220b shown in Fig. 3. This process results in the identification DB 220 consisting of image extraction information 220a and other related information 220b.

[0070] If an existing book database is not used or if no such database exists, step 107 can be omitted. In this case, the identification DB 220 will be composed only of image extraction information 220a obtained from the captured image. Even in this case, the text information on the spine can be used as identification information to identify the book, so the book inspection process can be carried out.

[0071] Next, the analysis device 200 transmits to the management device 300 at least information (data table) in which identification information for identifying a book and the spine image of the book are associated with each other (step 108). The identification information for identifying a book can use character information on the spine extracted from the image. Alternatively, other information (such as a library control number) integrated in the matching process of step 107 may be used. Alternatively, the information in the identification DB 220 may be transmitted directly to the management device 300.

[0072] After acquiring the spine image and the text information on the spine (or other identification information that can identify the book), the management device 300 uses that information to create a management DB 320 for the management device 300 (step 109). If the management device 300 can use the library's existing book collection database, it can create the management DB 320 by associating the acquired spine images with the information on the corresponding books in the existing book collection database. Also, if information from the existing book collection database has been incorporated into the identification DB 220 in step 107, the information in the identification DB 220 may be acquired from the analysis device 200 and used as is as the management DB 320.

[0073] The above is the flow of the preparation process in the book management system 1 of this embodiment. Once the preparation process is complete, the book inventory process can be performed using the book management system 1. Note that in steps 101 and 102, the mobile image capture device 100 photographs all of the target bookshelves and then transmits all of the images together to the analysis device 200, but the captured images may also be transmitted to the analysis device 200 at any timing. For example, the mobile image capture device 100 may transmit an image to the analysis device 200 immediately after each image capture, or may transmit the images in multiple batches at appropriate timings. Furthermore, if there are multiple mobile image capture devices 100, each mobile image capture device 100 may photograph the bookshelves assigned to it, and each mobile image capture device 100 may transmit the captured images separately.

[0074] (Book collection inspection process) Next, we will explain the flow of the book inventory inspection process performed by the book inventory management system 1. The book inventory inspection process is a process that uses the databases (identification DB 220, management DB 320) created by the preparation process explained in Fig. 5 and the AI ​​model 210 to identify each book from an image of the bookshelf taken at the time of inspection, and check whether or not the managed book is on the bookshelf.

[0075] FIG. 7 shows a flowchart illustrating the flow of the book inspection process. First, the mobile camera 100 photographs the bookshelf to be inspected (step 201). For example, when the mobile camera 100 receives an instruction to inspect the bookshelf from an external computer or the like, it starts photographing the bookshelf to be inspected. Furthermore, if the mobile camera 100 is set to inspect the bookshelf at a pre-specified time, such as during a book inspection or when the library is closed and no users are present, it can start the photographing process at the specified time. The bookshelves to be inspected may be all the bookshelves in the library, or only some of the shelves. For example, when the inspection process is to be performed in multiple sessions, the bookshelves to be inspected for each session can be specified and inspected.

[0076] Based on the library map stored in the memory unit 102, the mobile image capturing device 100 moves to the location of the bookshelf to be photographed and takes a photograph with the camera 110. If the bookshelves to be photographed and the order in which the bookshelves are to be photographed are specified, the control unit 102 controls the camera 110, the traveling unit 120, etc. to take a photograph based on the specification.

[0077] When photographing a bookshelf, the mobile camera 100 moves within the photographing range as shown in Fig. 6, photographing so as to capture images of all the spines of the books in the bookshelf. The control unit 102 controls the mechanism that raises and lowers the photographing position of the camera 110 to photograph each shelf. After photographing one bookshelf, the mobile camera 100 moves to the next bookshelf (photographing position) and photographs the next bookshelf, repeating this process. The mobile camera 100 can associate information such as location information that identifies the location (bookshelf) where the photograph was taken with the photographed image, and the date and time of the photograph.

[0078] Once the photographs of the bookshelves that are the subject of the book inspection have been taken, the mobile image capture device 100 transmits all of the images captured during the book inspection (inspection photographed images) to the analysis device 200 (step 202). If all of the bookshelves in the library are photographed in one inspection, the images of all of the bookshelves are transmitted. If the inspection is conducted in multiple sessions, the images captured during that inspection are transmitted. The mobile image capture device 100 can also transmit location information associated with the photographed images, the date and time of the photograph, and so on, along with the photographed images.

[0079] When the analysis device 200 acquires the captured image from the mobile image capture device 100, it performs image analysis on the captured image and detects the spine of each book in the image (step 203). This process is the same as step 103 of the preparation process shown in the flowchart of Figure 5. The control unit 202 of the analysis device 200 uses the AI ​​model 210 to detect the spine portion of each book from the captured image that includes multiple spines, and generates a spine image of each book.

[0080] Next, analysis device 200 uses AI model 210 to perform image analysis of the spine image, and compares it with the spine image of a book registered in identification DB 220 to identify the book corresponding to each spine image (step 204). Specifically, control unit 202 uses AI model 210 to compare the spine image of the book to be compared with the spine image registered in identification DB 220 to identify the spine image. For example, AI model 210 extracts features of the spine image to be compared and compares them with the features of each image registered in identification DB 220 (matching), thereby finding a matching image. Then, control unit 202 can identify the book to which the spine image to be compared belongs, using identification information (library control number, book title, etc.) of the book associated with the matching image in identification DB 220.

[0081] Next, the analysis device 200 transmits the book inspection result information to the management device 300 (step 205). The book inspection result information is information indicating the results of the book inspection, and is information indicating which books were confirmed in the book inspection process. For example, a list (data table) can be used in which information indicating that the book identified from the image in step 204 was present on the bookshelf where the inspection was performed is associated with identification information that can identify the book. The book inspection result information may also include a list in which, for books whose presence was not confirmed during the inspection, the identification information of the book is associated with information indicating that the book has not been confirmed. The book inspection result information may also include the date and time of the photo (inspection) corresponding to the time the book was confirmed, and information about the location of the bookshelf where the book was present (such as the location information of the photographed image).

[0082] Next, the management device 300 acquires the book inspection result information from the analysis device 200, and based on the acquired information, registers the book inspection result in the management DB 320 (step 206). The control unit 302 of the management device 300 registers the inspection result in the record of the corresponding book in the management DB 320 based on the identification information of each book in the acquired book inspection result information.

[0083] For example, in the case of the management DB 320 shown in Figure 4, once the inspection results for books with library control numbers "ABCDE" and "FGHIJ" are obtained, the inspection results are registered for those book records based on the book inspection result information. In the example of Figure 4, for the book with control number "ABCDE," it was confirmed from the image captured during the book inspection process on August 10 that it was placed on a bookshelf, so the fact that the book was on the bookshelf and the date (present: 8 / 10) are registered. Also, for the book with control number "FGHIJ," it is registered that it was not present during the inspection on August 10. Note that the fact that a book was not present can be registered only after inspection of all bookshelves has been completed and it has been confirmed that the book was not present on any of the bookshelves.

[0084] The information to be registered as the inspection result can be selected by the user as appropriate. For example, the number (location information) of the bookshelf on which the book was found may be registered. This allows the user to check whether the found book is placed on the correct bookshelf.

[0085] The above is the flow of processing during a book inventory inspection. After the inspection process is completed, users of this system 1, such as library administrators, can check the inspection results registered in the management DB 320 to confirm whether or not a book is available. In the example of Figure 4, as described above, it can be confirmed that the book with management number "ABCDE" was on the bookshelf at the time of the most recent inspection (inspection result 1), and that the book with "FGHIJ" was not on any bookshelf. Furthermore, by checking the loan status of book "FGHIJ," it can be determined that the book is currently on loan, so it can be confirmed that it is not lost. Furthermore, books that are not currently on loan and that were not found during the book inventory process can be determined to be whereabouts unknown.

[0086] 7, the mobile image capture device 100 photographs all bookshelves to be inspected during the book inventory inspection and then transmits the images to the analysis device 200. However, this order is not limited to this, and the captured images may be transmitted at any timing. For example, the mobile image capture device 100 may transmit each captured image to the analysis device 200 as it is taken, or may capture and transmit multiple images. In these cases, the analysis device 200 may perform comparison as it acquires the captured images. Furthermore, if there are multiple mobile image capture devices 100, each mobile image capture device 100 may photograph the bookshelves it is responsible for, and each mobile image capture device 100 may transmit the captured images to the analysis device 200.

[0087] The book management system 1 of this embodiment described above allows for automated book inspections even without library staff. For example, if the book management system 1 is configured to perform book inspections during times when there are no library users (i.e., after closing time), the book inspection process is automatically performed during that time, and staff can check the results of the book inspection at any time after the inspection is completed. Therefore, this system eliminates the need to close the library to conduct book inspections. Furthermore, if the mobile camera 100 is capable of autonomously moving while avoiding obstacles such as people, the book inspection process can be performed during open hours. Furthermore, the book management system 1 eliminates the need to remove books from bookshelves and read their tags during inspections, allowing for more efficient inspections. Furthermore, by using the mobile camera 100, which is an autonomously moving device (robot), coordinate information that can identify the locations of shelves and books in the library can be acquired and used during the preparation process for system implementation. Therefore, even in book inventory inspection processing, the mobile photography device 100 can automatically and smoothly move around the library to take photographs, and the location of the photographed book can be automatically and accurately determined, allowing book inventory management to be carried out efficiently and with high precision without relying on personnel.

[0088] Furthermore, during the initial preparation process, the book management system 1 can extract text information from the spine using OCR processing, detect the spine in the captured image, and extract features of that image to create a database that associates text information identifying a book with the spine image of that book. Therefore, even if a library does not have an existing book database, it can create the databases (identification DB 220 and management DB 320) necessary for book collection inspection processing. Therefore, even if a library does not have an existing book database, it can create the management DB 320 as a book list and use that database to manage its book collection. If there is an existing book database, the database can be matched and a book collection inspection can be performed using a database that combines existing information.

[0089] In addition, the library management system 1 of this embodiment stores spine images captured during the preparatory process, and then uses the AI ​​model to extract features from the saved spine images in the analysis device 200. During the book inventory process, the AI ​​model uses the features to match the spine image captured during the inspection with a saved spine image. This method allows for accurate and rapid book identification, even for books with spines whose titles are difficult to read using OCR. Furthermore, since the spine images from the library's actual books are used as the spine images from which features are extracted during the preparatory process, matching can be performed using images of the actual books captured during the book inventory. Therefore, books can be accurately identified even if, for example, the spine is damaged, discolored, or has a label attached. Similarly, books in the library's inventory can be accurately identified even if there are multiple editions of the same book.

[0090] The programs and AI models for implementing the book collection management system 1 of this embodiment may be stored in a computer-readable storage medium. By installing the programs from the computer-readable storage medium into a corresponding device, the functions of the book collection management system 1 can be realized and the book collection management process of this embodiment can be executed. [Explanation of symbols]

[0091] 1 Library management system 100 Mobile camera 102 Control section 104 Storage section 106 Communications Department 110 Camera 120 Running part 200 Analysis equipment 202 Control section 204 Storage section 206 Communications Department 210 AI models 220 Specific Database (DB) 230 OCR Program 240 DB matching program 300 Management device 302 Control Unit 304 Storage section 306 Communications Department 320 Management Database (DB) 50 Network

Claims

1. a mobile imaging device that is equipped with an imaging device and moves around the library to photograph books on the bookshelves; an analysis device that analyzes the captured image including the spine of the book captured by the mobile image capture device; A book management system including a management device for managing information about a library's book collection, The analysis device A process of acquiring photographed images of books on the bookshelf from the mobile photographing device; A process of detecting the spines of the books included in the photographed images by image analysis, and extracting character information of the spines of the books by OCR processing of the detected spine images; A process of creating an identification database in which the image of the spine of the book and the character information of the spine of the book are registered in association with each other; extracting features of the spine image of each book using an AI model, and storing the information on the features in association with each book; A book management system characterized in that the management device obtains information about each book, including at least an image of the spine of each book and text information about the spine, from the analysis device, and uses the obtained information to create a management database for book management.

2. The book management system according to claim 1, wherein the mobile imaging device is an autonomously mobile device of a wheeled type or a drone type.

3. 2. The book management system according to claim 1, wherein the mobile image capturing device moves to the location of the target bookshelf and captures the image by referring to a library map created in advance.

4. The book management system of claim 1, characterized in that the analysis device compares the extracted spine text information with the library's existing book database, extracts information corresponding to the book having the spine text information from the existing book database, and incorporates the extracted information into the identification database.

5. When inspecting the library's books after creating the management database, the mobile camera takes a picture of the bookshelf and transmits the obtained inspection photographed image to the analysis device; The analysis device uses the AI ​​model to analyze the image of the spine of the book to be compared that is included in the transmitted inspection-photographed image, and uses the stored feature amount to compare it with any spine image registered in the identification database to identify the book to be compared; and transmitting to the management device information on the book inspection result, the information including information indicating that the identified book was present on the bookshelf that was inspected.

2. The book management system according to claim 1, wherein the management device uses the book inspection result information transmitted to it to register the book inspection result in the management database.

6. 6. The book management system according to claim 5, wherein said mobile image capturing device captures images of said bookshelves for book inspection purposes during library closing hours.

7. a mobile imaging device that is equipped with an imaging device and moves around the library to photograph books on the bookshelves; an analysis device that analyzes the captured image including the spine of the book captured by the mobile image capture device; A book management method executed in a book management system including a management device that manages information about a library's book collection, Acquire a photographed image of a book on the bookshelf; Detecting the spines of the books included in the photographed images by image analysis, and extracting character information from the spines of the books by OCR processing of the detected spine images; creating an identification database in which the image of the spine of the book and the character information of the spine of the book are registered in association with each other; Extracting features of the spine image of each book using an AI model, and storing the information of the features in association with each book; A book management method characterized by acquiring information about each book including at least an image of the spine of each book and text information on the spine, and using the acquired information to create a management database for book management.

8. The book collection management method according to claim 7, wherein the mobile imaging device is an autonomously mobile device of a wheeled type or a drone type.

9. 8. The book collection management method according to claim 7, wherein the mobile image capturing device refers to a library map prepared in advance, moves to the location of the target bookshelf, and captures the image.

10. The book management method according to claim 7, characterized in that the extracted text information on the spine is compared with an existing book database of the library, information corresponding to the book having the text information on the spine is extracted from the existing book database, and the extracted information is incorporated into the identification database.

11. When inspecting the library's books after creating the management database, the bookshelf is photographed and the obtained inspection photographed image is transmitted; Using the AI ​​model, an image of the spine of the book to be compared that is included in the transmitted inspection photograph image is analyzed, and the image is compared using the stored feature amount to determine which spine image registered in the identification database matches, thereby identifying the book to be compared; Transmitting book inspection result information including information indicating that the identified book was present on the inspected bookshelf; 8. A book management method according to claim 7, wherein the book inspection result is registered in the management database using the transmitted book inspection result information.

12. 12. The book collection management method according to claim 11, wherein the bookshelves are photographed for book collection inspection during library closing hours.

Images