Library book position information confirmation method and related equipment

Abstract

The embodiment of the invention provides a library book position information confirmation method and related equipment, and belongs to the technical field of book management. After to-be-queried book information uploaded by a user side is received, a target spine vector matched with the to-be-queried information is retrieved in a spine vector library, a target bookshelf vector corresponding to the target spine vector is determined, and the bookshelf position of the target bookshelf vector and the position of the target spine vector in a target bookshelf are combined, so that the bookshelf position of the to-be-queried book information is determined. Finally, the real-time position of the to-be-queried book in the library is determined, the real-time position of the book is fed back to the reader, the reader can quickly query the to-be-queried book according to the real-time position, and the reading experience of the reader is improved.

Description

Technical Field

[0001] This application relates to the field of library management technology, and in particular to a method and related equipment for confirming the location information of library books. Background Technology

[0002] Each bookshelf in the library is filled with numerous books, each with its designated place. When readers want to borrow a book, they often find it difficult to locate the desired book from the densely packed shelves, thus diminishing their reading experience.

[0003] Therefore, there is an urgent need for a solution that can efficiently and accurately confirm the location information of library books and support natural query methods. Summary of the Invention

[0004] The main objective of this application is to achieve rapid and accurate confirmation of the location information of books in a library.

[0005] To achieve the above objectives, one aspect of this application proposes a method for confirming the location information of library books, the method comprising: The system receives information about books to be searched input from a user terminal. This information represents the books that the user needs to search for, and can be in any of the following formats: text, voice, or images. Encode the book information to be queried into a query vector; Nearest neighbor retrieval is performed on the query vector to obtain one or more target spine vectors that match the query vector from the spine vector library, which stores visual vectors obtained by identifying the spines of books on each bookshelf in the library. Determine the target bookshelf vector corresponding to the target spine vector from the bookshelf vector library; The real-time location information of the target bookshelf vector and the target spine vector in the library is determined, and the real-time location information is fed back to the user terminal.

[0006] In some embodiments, Encoding the book information to be queried into a query vector includes: If the book information to be queried is text information, then the book information to be queried is encoded into a text query vector; If the book information to be queried is audio information, extract the spectral features from the audio information to obtain the spectral query vector; If the book information to be queried is image information, then the image information is encoded into a visual query vector; The query vector is obtained by concatenating at least one of the text query vector, the spectrum query vector, and the visual query vector, and aligning the fusion vector to a unified semantic space.

[0007] In some embodiments, determining the real-time location information of the target bookshelf vector and the target spine vector in the library, and feeding back the real-time location information to the user terminal, includes: Determine the position of the target spine vector within the target bookshelf vector; Determine the target bookshelf photo corresponding to the target bookshelf vector from the bookshelf photo library; the target bookshelf photo contains the bookshelf position information of the target bookshelf vector. Determine the spine position information of the target spine vector in the target bookshelf vector; The target bookshelf photo is fed back to the user terminal, and the target bookshelf photo is used to indicate the location information of the book spine.

[0008] In some embodiments, determining the target bookshelf vector corresponding to the target spine vector from the bookshelf vector library includes: The book database is used to query the bookshelf vector position corresponding to the target spine vector. The book database stores the correspondence between each bookshelf vector and each spine vector. Based on the bookshelf vector position, the target bookshelf vector is determined in the bookshelf vector library.

[0009] In some embodiments, feeding back the target bookshelf photo to the user terminal includes: Based on the position of the target spine vector in the target bookshelf vector, a graphic marker box is generated in the target bookshelf image to highlight the position of the book to be queried. The target bookshelf photo with the graphic identifier frame is fed back to the user terminal.

[0010] In some embodiments, before receiving the book information to be queried input by the user terminal, the method further includes: Obtain at least one bookshelf image uploaded by the administrator terminal, which contains at least one bookshelf and multiple books are stored on the bookshelf; The target detection model is used to detect each spine region in the bookshelf image, and the bookshelf image is cropped based on the detection box information to obtain a spine sub-image; A multimodal semantic alignment model is used to extract features from the bookshelf image and the spine sub-image to obtain the bookshelf vector set and the spine vector set; The bookshelf image is entered into the bookshelf photo library, the bookshelf vector set is entered into the bookshelf vector library, and the book spine vector set is entered into the book spine vector library.

[0011] In some embodiments, the method further includes: The correspondence between the bookshelf vector set and the spine vector set is entered into the book database.

[0012] To achieve the above objectives, another aspect of this application provides a library book positioning device, the device comprising: The receiving module is used to receive the book information to be queried input by the user terminal. The book information to be queried is used to represent the book to be queried by the user. The book information to be queried can be any one of text, voice or image. The conversion module is used to encode the book information to be queried into a query vector; The retrieval module is used to perform nearest neighbor retrieval on the query vector and obtain one or more target spine vectors that match the query vector from the spine vector library. The spine vector library stores visual vectors obtained by identifying the spine parts of books on each bookshelf in the library. The positioning module is used to determine the target bookshelf vector corresponding to the target spine vector from the bookshelf vector library; The feedback module is used to determine the real-time location information of the target bookshelf vector and the target spine vector in the library, and to feed back the real-time location information to the user terminal.

[0013] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0014] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.

[0015] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method.

[0016] The embodiments of this application include at least the following beneficial effects: This application provides a method, apparatus, electronic device, storage medium, and program product for confirming the location information of library books. This solution receives query book information input by a user terminal. The query book information represents the book requested by the user and can be in any form of text, voice, or image. The query book information is encoded into a query vector. Nearest neighbor retrieval is performed on the query vector to obtain one or more target spine vectors matching the query vector from a spine vector library. The spine vector library stores visual vectors obtained by identifying the spines of books on various bookshelves in the library. A target bookshelf vector corresponding to the target spine vector is determined from the bookshelf vector library. The real-time location information of the target bookshelf vector and the target spine vector in the library is determined and fed back to the user terminal. After receiving the book information uploaded by the user, this embodiment retrieves the target spine vector matching the query information from the spine vector library, determines the target bookshelf vector corresponding to the target spine vector, and combines the bookshelf position of the target bookshelf vector with the position of the target spine vector within the target bookshelf to finally determine the real-time location of the book in the library. The real-time location of the book is then fed back to the reader, allowing the reader to quickly find the book based on the real-time location, thus improving the reader's reading experience. Attached Figure Description

[0017] Figure 1 This is a flowchart of a method for confirming the location information of library books provided in an embodiment of this application; Figure 2 This is a flowchart of another method for confirming the location information of library books provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of the library book location information confirmation device provided in the embodiments of this application; Figure 4 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0020] In view of this, this application provides a method and related equipment for confirming the location information of library books. This method involves receiving information about books to be queried input from a user terminal. This information represents the books the user requests and can be in text, voice, or image format. The information is then encoded into a query vector. Nearest neighbor retrieval is performed on the query vector to obtain one or more target spine vectors matching the query vector from a spine vector library. The spine vector library stores visual vectors obtained by identifying the spines of books on various bookshelves in the library. A target bookshelf vector corresponding to the target spine vector is determined from the bookshelf vector library. The real-time location information of the target bookshelf vector and the target spine vector in the library is determined and fed back to the user terminal. After receiving the book information uploaded by the user, this embodiment retrieves the target spine vector matching the query information from the spine vector library, determines the target bookshelf vector corresponding to the target spine vector, and combines the bookshelf position of the target bookshelf vector with the position of the target spine vector within the target bookshelf to finally determine the real-time location of the book in the library. This provides readers with accurate feedback on the real-time location of the book, allowing them to quickly find the book based on the real-time location and improving their reading experience.

[0021] The library book location information confirmation method provided in this application relates to the field of library management technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the library book location information confirmation method, but is not limited to the above forms.

[0022] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0023] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0024] This application provides a library book location information confirmation system. This system can be deployed on a server or cloud platform and communicates with administrator terminals and user terminals (such as mobile phones, tablets, and kiosks) via a network. The core of the system lies in constructing a multi-layered visual and semantic information database and achieving fast and accurate book location based on vector retrieval technology.

[0025] The main functions of the system include: 1. Receive bookshelf images uploaded by the administrator and perform image preprocessing, spine detection, feature extraction, and other operations.

[0026] 2. Construct a multimodal vector library, including a bookshelf photo library, a bookshelf vector library, a spine vector library, and a book database for storing relationships.

[0027] 3. Perform book retrieval, receive reader queries, convert them into query vectors, and perform similarity retrieval in the spine vector database.

[0028] 4. Feedback on the location information of the book to be searched: Based on the search results, combined with the book database and bookshelf photo library, determine the specific location of the book in the library and provide feedback in a format that readers can view.

[0029] The library book location information confirmation method provided in this application mainly includes two stages: Phase 1. Book Information Collection and Database Construction During this stage, after organizing the bookshelves, the administrator takes photos of each bookshelf. The system automatically processes these images, extracts feature vectors from the bookshelves and spines, and establishes corresponding vector libraries and related databases.

[0030] Phase Two: Book Location Search and Feedback Phase During this stage, readers input information about the books they want to search for in various ways. The system uses vector retrieval and related queries to locate the specific location of the books in the library and provides feedback by visually marking them on photos of the bookshelves.

[0031] The following will provide a detailed explanation of these two stages and each step, with reference to specific embodiments.

[0032] In Phase One, the system digitizes the visual information of all bookshelves and books in the library and transforms it into a vector format that allows for efficient retrieval. Phase One is as follows: Figure 1 As shown, the specific steps include: Step S101: Obtain at least one bookshelf image uploaded by the administrator terminal, which contains at least one bookshelf and multiple books are stored on the bookshelf.

[0033] Administrators will use a camera-enabled device (such as a high-definition camera, smartphone, or dedicated library management equipment) to photograph each bookshelf in the library. At least one clear image showing the spines of all books on each shelf should be taken. For easier processing, it is recommended to take the photos under uniform lighting conditions without severe perspective distortion. After the images are uploaded to the system, a unique identifier (such as an image ID) will be generated for each image and linked to the actual physical location information of the bookshelf (e.g., "Section A, Row 3, Shelf 2"). This linking can be pre-entered by the administrator or extracted from the bookshelf label using image recognition technology.

[0034] Step S102: Use the object detection model to detect each spine region in the bookshelf image, and crop the bookshelf image based on the detection box information to obtain the spine sub-image.

[0035] When acquiring the bookshelf image uploaded from the administrator terminal, image preprocessing can be performed first. Preprocessing methods can include: wide-angle correction, perspective transformation, and brightness and contrast adjustment.

[0036] From the preprocessed image, identify and extract information to uniquely identify the bookshelf. The extracted identification information (such as "F-12-05") will serve as the bookshelf's digital identity information.

[0037] The system uses a pre-trained object detection model (such as a model based on YOLO, Faster R-CNN, etc.) to analyze the pre-processed bookshelf image and automatically detect the spine region of each book in the image. The model outputs a bounding box for each detected spine region, which contains the precise coordinate information of the spine in the original image (such as the coordinates of the top left corner (x1, y1) and the bottom right corner (x2, y2)).

[0038] Subsequently, based on these bounding box coordinates, the system crops each spine region from the original bookshelf image, forming an independent spine sub-image. Each spine sub-image is associated with its source bookshelf image ID, bounding box coordinate information, and the bookshelf identifier information to which it belongs.

[0039] Step S103: Use a multimodal semantic alignment model to extract features from the bookshelf image and the spine sub-image to obtain the bookshelf vector set and the spine vector set.

[0040] This application employs a multimodal semantic alignment model (such as CLIP, ALIGN, and other pre-trained models) to extract visual feature vectors with rich semantic information. Specifically, this embodiment uses a multimodal semantic alignment model to extract feature information from bookshelf images and spine sub-images.

[0041] The entire preprocessed bookshelf image is input into the visual encoder of the multimodal semantic alignment model, which outputs a high-dimensional feature vector as the bookshelf vector. This vector encodes the global visual and semantic information of the entire bookshelf. Each cropped spine image is also input into the visual encoder of the multimodal semantic alignment model, which outputs a corresponding spine vector. This vector encodes the visual features and potential semantic information of a single book spine (such as information about the book title and author implied through spine design, color, and text layout).

[0042] Since the number of bookshelf images and spine sub-images for feature extraction is generally large, when performing feature extraction in batches, the final extracted bookshelf vector set is called the bookshelf vector set, and the spine vector set is called the spine vector set.

[0043] Step S104: Input the bookshelf image into the bookshelf photo library, input the bookshelf vector set into the bookshelf vector library, and input the book spine vector set into the book spine vector library.

[0044] The bookshelf photo library is used to store original bookshelf image files uploaded by the administrator terminal, indexed by bookshelf identification information or image ID.

[0045] The bookshelf vector library is used to store all bookshelf vectors generated in step S103. Each bookshelf vector record is associated with the corresponding bookshelf identification information and bookshelf image ID.

[0046] The spine vector library is used to store all spine vectors generated in step S103. This is the direct object library for subsequent vector retrieval.

[0047] The book database is used to establish and maintain the relationships between all elements. Each record must contain at least the following fields: Spine Vector ID: A unique identifier pointing to a specific vector in the spine vector library.

[0048] Bookshelf Vector ID: A vector identifier pointing to the bookshelf where the spine of this book is located.

[0049] Bookshelf identification information: The actual location number of the bookshelf.

[0050] Bookshelf Image ID: The ID of the original photo.

[0051] Spine bounding box coordinates: The specific position of the spine in the image of the bookshelf to which it belongs (x1, y1, x2, y2).

[0052] Through the above steps, the system has completed the digitization of library book information. This process maps bookshelves and books in the physical world into vectors and related data in the digital world, laying the foundation for efficient semantic retrieval. It is worth noting that this solution does not rely on traditional barcode or RFID book-by-book scanning; instead, it significantly improves data entry efficiency through batch image processing.

[0053] When a reader needs to find a book, they can initiate a search through their user terminal. This process, namely stage two mentioned above, transforms fuzzy user input into precise book location information, such as... Figure 2 As shown, the specific steps include: Step S201: Receive the book information to be queried input by the user terminal. The book information to be queried is used to represent the book to be queried by the user. The book information to be queried can be any one of text, voice or image.

[0054] The system receives query requests from user terminals. The forms of information queried are flexible and varied, including: Users can directly enter the book title, author, ISBN number, or keywords (such as "Introduction to Artificial Intelligence").

[0055] Users can speak the title of the book they want to find, and the system backend will convert it into text using automatic speech recognition technology.

[0056] Image format: Users can upload photos of a book cover, some interior pages, or even a blurry screenshot.

[0057] Step S202: Encode the book information to be queried into a query vector.

[0058] The system needs to unify different forms of query information into the same semantic space as the spine vector in order to perform similarity comparisons.

[0059] If the input is text, the text encoder in the same multimodal semantic alignment model as in the database building phase is used to encode the text description into a text query vector.

[0060] If the input is speech, the speech encoder in the multimodal semantic alignment model is used to encode the speech into a spectral query vector.

[0061] If the input is an image, the model's visual encoder is used to encode the image into a visual query vector.

[0062] Regardless of whether the query information is a single query vector such as a text query vector, a spectrum query vector, or a visual query vector, the system directly inputs the single type of query vector into the model.

[0063] When the information to be queried includes multiple information formats, in order to ensure that the generated query vector contains at least two types of query vectors, such as text query vectors, spectrum query vectors, and visual query vectors, the system can fuse the text query vectors and visual query vectors (such as vector concatenation, weighted averaging, etc.). Through the model's inherent alignment mechanism, it ensures that the fused query vector is in a unified and comparable semantic space with the vectors in the spine vector library.

[0064] Step S203: Perform nearest neighbor retrieval on the query vector and obtain one or more target spine vectors that match the query vector from the spine vector library. The spine vector library stores visual vectors obtained by identifying the spines of books on each bookshelf in the library.

[0065] Similarity retrieval is performed in a spine vector database. Retrieval algorithms typically employ nearest neighbor search, using metrics such as cosine similarity or Euclidean distance to find one or more target spine vectors in the database that are most similar to the query vector. Thanks to the efficiency of vector retrieval techniques (such as using approximate nearest neighbor databases like FAISS and Annoy), results can be returned in milliseconds, even in databases containing millions of spine vectors.

[0066] Step S204: Determine the target bookshelf vector corresponding to the target spine vector from the bookshelf vector library.

[0067] After obtaining one or more target spine vectors, the system needs to locate which bookshelf these spines are located on. At this point, the book database plays a crucial role. Based on the target spine vector ID, the system queries the book database for its corresponding record, thereby obtaining at least the bookshelf vector ID from the book database. Based on the corresponding bookshelf vector ID, the system can locate the corresponding target bookshelf vector from the bookshelf vector library. This step is achieved through a fast query of a relational database, linking the vector retrieval results with specific physical location information.

[0068] Step S205: Determine the real-time location information of the target bookshelf vector and the target spine vector in the library, and feed back the real-time location information to the user terminal.

[0069] The system ultimately needs to present the location information to the user in a visually appealing way: First, based on the image ID of the corresponding bookshelf retrieved from the book database, the system retrieves the original bookshelf image (target bookshelf photo) from the bookshelf photo library. Second, based on the spine bounding box coordinates in the same record, the system generates a highlighted graphic marker (such as a red rectangle or flashing box) at the corresponding coordinate position on the retrieved target bookshelf photo, precisely circling the location of the book to be queried. Finally, the system sends this target bookshelf photo with location markings, along with the actual location information of the bookshelf (e.g., "located on the second floor, in the Natural Sciences section, row B, shelf 7, the 3rd shelf from the top"), to the user terminal.

[0070] Readers can use their mobile phones or other devices to see a picture of the precise location of a book on a specific bookshelf, allowing them to quickly navigate to and find the book.

[0071] To illustrate this solution more clearly, let's consider a specific application scenario: Scene: Student Zhang wants to borrow "Dream of the Red Chamber" from the library.

[0072] Implementation process: Zhang used the library's query terminal or his own mobile app to voice-input "I want to find 'Dream of the Red Chamber'".

[0073] The system converts the speech into the text "Dream of the Red Chamber" and generates a query vector Q using a text encoder.

[0074] The system searches the spine vector library for the spine vector that is most similar to Q. It is assumed that the most matching vector found corresponds to the spine of the book "Dream of the Red Chamber (People's Literature Publishing House)".

[0075] The system found through the book database that the record associated with the spine vector ID is located at the bookshelf vector "SV_205", bookshelf identifier "2F-LIT-A12", image "IMG_205.jpg", and coordinates (150, 300, 250, 450).

[0076] The system retrieves IMG_205.jpg from the bookshelf photo library and draws a red box in the area from (150,300) to (250,450) of the image.

[0077] Zhang's phone screen showed a photo of a bookshelf, in which a copy of "Dream of the Red Chamber" was highlighted in red, with the text: "This book is located in the Literature Section on the second floor (2F-LIT), row A, shelf 12, in the middle."

[0078] This embodiment's technical solution is applied to readers searching for books, supporting multiple input methods such as text, voice, and images, conforming to user habits and lowering the barrier to entry. When displaying the final search results, it directly provides real-life photos of bookshelves with visual markers, which is more intuitive and easier to locate than traditional text descriptions of "section-row-shelf-layer."

[0079] In terms of database construction, this embodiment can replace manual book-by-book scanning with batch image processing; the query phase utilizes efficient vector retrieval technology, resulting in a fast response time. Furthermore, the solution relies on the latest collected bookshelf photos to reflect the real-time placement of books. If a book is misplaced, the system can update its location information after the administrator re-photographs and re-builds the database, resolving the problem of books being misplaced and difficult to find.

[0080] Please see Figure 3 This application also provides a library book location information confirmation device, which can implement the above method. The device includes: The receiving module 31 is used to receive the book information to be queried input by the user terminal. The book information to be queried is used to represent the book to be queried by the user. The book information to be queried can be any one of text, voice or image. The conversion module 32 is used to encode the book information to be queried into a query vector; The retrieval module 33 is used to perform nearest neighbor retrieval on the query vector and obtain one or more target spine vectors that match the query vector from the spine vector library. The spine vector library stores visual vectors obtained by identifying the spine parts of books on each bookshelf in the library. The positioning module 34 is used to determine the target bookshelf vector corresponding to the target spine vector from the bookshelf vector library; Feedback module 35 is used to determine the real-time location information of the target bookshelf vector and the target spine vector in the library, and to feed back the real-time location information to the user terminal.

[0081] In some embodiments, the conversion module 32 is configured to: If the book information to be queried is text information, then the book information to be queried is encoded into a text query vector; If the book information to be queried is audio information, extract the spectral features from the audio information to obtain the spectral query vector; If the book information to be queried is image information, then the image information is encoded into a visual query vector; The query vector is obtained by concatenating at least one of the text query vector, the spectrum query vector, and the visual query vector, and aligning the fusion vector to a unified semantic space.

[0082] In some embodiments, the feedback module 35 is used for: Determine the position of the target spine vector within the target bookshelf vector; Determine the target bookshelf photo corresponding to the target bookshelf vector from the bookshelf photo library; the target bookshelf photo contains the bookshelf position information of the target bookshelf vector. Determine the spine position information of the target spine vector in the target bookshelf vector; The target bookshelf photo is fed back to the user terminal, and the target bookshelf photo is used to indicate the location information of the book spine.

[0083] In some embodiments, the positioning module 34 is used for: The book database is used to query the bookshelf vector position corresponding to the target spine vector. The book database stores the correspondence between each bookshelf vector and each spine vector. Based on the bookshelf vector position, the target bookshelf vector is determined in the bookshelf vector library.

[0084] In some embodiments, the feedback module 35 is used for: Based on the position of the target spine vector in the target bookshelf vector, a graphic marker box is generated in the target bookshelf image to highlight the position of the book to be queried. The target bookshelf photo with the graphic identifier frame is fed back to the user terminal.

[0085] In some embodiments, the library book location information confirmation device further includes: a database creation module 30, used for: Obtain at least one bookshelf image uploaded by the administrator terminal, which contains at least one bookshelf and multiple books are stored on the bookshelf; The target detection model is used to detect each spine region in the bookshelf image, and the bookshelf image is cropped based on the detection box information to obtain a spine sub-image; A multimodal semantic alignment model is used to extract features from the bookshelf image and the spine sub-image to obtain the bookshelf vector set and the spine vector set; The bookshelf image is entered into the bookshelf photo library, the bookshelf vector set is entered into the bookshelf vector library, and the book spine vector set is entered into the book spine vector library.

[0086] In some embodiments, the library creation module 30 is used for: The correspondence between the bookshelf vector set and the spine vector set is entered into the book database.

[0087] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0088] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0089] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0090] Please see Figure 4 , Figure 4 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 401 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 402 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 402 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 402 and is called and executed by the processor 401 using the methods described in the embodiments of this application. Input / output interface 403 is used to implement information input and output; The communication interface 404 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 405 transmits information between various components of the device (e.g., processor 401, memory 402, input / output interface 403, and communication interface 404); The processor 401, memory 402, input / output interface 403 and communication interface 404 are connected to each other within the device via bus 405.

[0091] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0092] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0093] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0094] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0095] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0096] The library book location information confirmation method, apparatus, electronic device, storage medium, and program product provided in this application embodiment receive query book information input by a user terminal. The query book information represents the book requested by the user and can be in the form of text, voice, or image. The query book information is encoded into a query vector. Nearest neighbor retrieval is performed on the query vector to obtain one or more target spine vectors matching the query vector from a spine vector library. The spine vector library stores visual vectors obtained by identifying the spines of books on various bookshelves in the library. A target bookshelf vector corresponding to the target spine vector is determined from the bookshelf vector library. The real-time location information of the target bookshelf vector and the target spine vector in the library is determined and fed back to the user terminal. After receiving the book information uploaded by the user, this embodiment retrieves the target spine vector matching the query information from the spine vector library, determines the target bookshelf vector corresponding to the target spine vector, and combines the bookshelf position of the target bookshelf vector with the position of the target spine vector within the target bookshelf to finally determine the real-time location of the book in the library. This provides readers with accurate feedback on the real-time location of the book, allowing them to quickly find the book based on the real-time location and improving their reading experience.

[0097] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0098] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0099] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0100] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0101] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0102] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0103] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0104] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0105] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0106] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0107] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for confirming the location information of library books, characterized in that, The method includes: The system receives information about books to be searched input from a user terminal. This information represents the books that the user needs to search for, and can be in any of the following formats: text, voice, or images. Encode the book information to be queried into a query vector; Nearest neighbor retrieval is performed on the query vector to obtain one or more target spine vectors that match the query vector from the spine vector library, which stores visual vectors obtained by identifying the spines of books on each bookshelf in the library. Determine the target bookshelf vector corresponding to the target spine vector from the bookshelf vector library; The real-time location information of the target bookshelf vector and the target spine vector in the library is determined, and the real-time location information is fed back to the user terminal.

2. The method according to claim 1, characterized in that, Encoding the book information to be queried into a query vector includes: If the book information to be queried is text information, then the book information to be queried is encoded into a text query vector; If the book information to be queried is audio information, extract the spectral features from the audio information to obtain the spectral query vector; If the book information to be queried is image information, then the image information is encoded into a visual query vector; The query vector is obtained by concatenating at least one of the text query vector, the spectrum query vector, and the visual query vector, and aligning the fusion vector to a unified semantic space.

3. The method according to claim 1, characterized in that, The step of determining the real-time location information of the target bookshelf vector and the target spine vector in the library, and feeding back the real-time location information to the user terminal, includes: Determine the position of the target spine vector within the target bookshelf vector; Determine the target bookshelf photo corresponding to the target bookshelf vector from the bookshelf photo library; the target bookshelf photo contains the bookshelf position information of the target bookshelf vector. Determine the spine position information of the target spine vector in the target bookshelf vector; The target bookshelf photo is fed back to the user terminal, and the target bookshelf photo is used to indicate the location information of the book spine.

4. The method according to claim 1, characterized in that, Determining the target bookshelf vector corresponding to the target spine vector from the bookshelf vector library includes: The book database is used to query the bookshelf vector position corresponding to the target spine vector. The book database stores the correspondence between each bookshelf vector and each spine vector. Based on the bookshelf vector position, the target bookshelf vector is determined in the bookshelf vector library.

5. The method according to claim 3, characterized in that, The step of feeding back the target bookshelf photo to the user terminal includes: Based on the position of the target spine vector in the target bookshelf vector, a graphic marker box is generated in the target bookshelf image to highlight the position of the book to be queried. The target bookshelf photo with the graphic identifier frame is fed back to the user terminal.

6. The method according to claim 1, characterized in that, Before receiving the book information to be queried input from the user terminal, the method further includes: Obtain at least one bookshelf image uploaded by the administrator terminal, which contains at least one bookshelf and multiple books are stored on the bookshelf; The target detection model is used to detect each spine region in the bookshelf image, and the bookshelf image is cropped based on the detection box information to obtain a spine sub-image; A multimodal semantic alignment model is used to extract features from the bookshelf image and the spine sub-image to obtain the bookshelf vector set and the spine vector set; The bookshelf image is entered into the bookshelf photo library, the bookshelf vector set is entered into the bookshelf vector library, and the book spine vector set is entered into the book spine vector library.

7. The method according to claim 6, characterized in that, The method further includes: The correspondence between the bookshelf vector set and the spine vector set is entered into the book database.

8. A library book location information confirmation device, characterized in that, The device includes: The receiving module is used to receive the book information to be queried input by the user terminal. The book information to be queried is used to represent the book to be queried by the user. The book information to be queried can be any one of text, voice or image. The conversion module is used to encode the book information to be queried into a query vector; The retrieval module is used to perform nearest neighbor retrieval on the query vector and obtain one or more target spine vectors that match the query vector from the spine vector library. The spine vector library stores visual vectors obtained by identifying the spine parts of books on each bookshelf in the library. The positioning module is used to determine the target bookshelf vector corresponding to the target spine vector from the bookshelf vector library; The feedback module is used to determine the real-time location information of the target bookshelf vector and the target spine vector in the library, and to feed back the real-time location information to the user terminal.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images