Library book sorting system based on image recognition

Abstract

The invention relates to the technical field of book management, in particular to a library book sorting system based on image recognition, which comprises an image acquisition module; an image processing module; the interdisciplinary association analysis module is used for analyzing the content association between the interdisciplinary books after the preliminary book sorting, and the interdisciplinary association analysis module is used for analyzing the content association between the interdisciplinary books; the knowledge path recommendation module is used for automatically identifying the same knowledge path book based on an output result of the interdisciplinary association analysis module and generating an optimized virtual knowledge path sorting scheme; providing an interactive interface for a borrower to check and select a highlighted knowledge path recommendation book; and the data storage module is used for storing the images, the classification labels, the associated information and the borrowing historical records of the books. According to the method and the device, the books can be optimized and sorted according to the content relevance and the requirements of the readers, so that the efficiency of searching the relevant books by the readers is remarkably improved, and the problem that the book classification and recommendation are not accurate enough in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of book management, and in particular to a library book sorting system based on image recognition. Background Art

[0002] As the scale of libraries continues to expand, the number of books is increasing day by day, and the management and sorting of books has become more and more complicated. Traditional book management systems mostly rely on technologies such as barcodes and RFID, identifying books through manual scanning or electronic tags, and then placing them according to the library's classification rules. However, this method is less efficient when facing large-scale book management, especially in terms of misplaced books and low reader search efficiency. There are many problems.

[0003] Although the current library management system can perform basic book identification and positioning, its classification and recommendation functions are relatively simple, and it is difficult to make intelligent recommendations based on readers' reading habits or the relevance of book content. In addition, existing systems generally lack a deep understanding of book content and are unable to sort and recommend books by analyzing the content relevance or complementarity between books. For example, when a reader borrows a book, the system cannot intelligently recommend other books related to the book, which causes readers to manually search for related books, reducing the user experience.

[0004] Therefore, designing a library book management system that can intelligently sort and recommend books based on image recognition and content analysis combined with historical borrowing records can effectively improve the management efficiency of library books and provide readers with a better search, reading and recommendation experience. Summary of the invention

[0005] The invention provides a library book sorting system based on image recognition.

[0006] A library book sorting system based on image recognition, comprising:

[0007] Image acquisition module: obtains image information of the book cover, and the image acquisition module includes a camera to collect image data of the book cover or spine;

[0008] Image processing module: connected to the image acquisition module, processes the acquired image information, and identifies the subject on the book cover. The image processing module classifies the book cover subject through a convolutional neural network (CNN) model, and preliminarily generates a category label and a sorting number of the book according to the library classification rules;

[0009] Interdisciplinary association analysis module: used to analyze the content association between interdisciplinary books after preliminary book sorting, and the interdisciplinary association analysis module includes:

[0010] The natural language processing unit, based on the subject on the book cover identified by the image processing module, deeply understands the semantics of the book content, analyzes the relevance or complementarity with other subject contents, uses the pre-trained semantic model to deeply understand the book content, extracts the core subject, key concepts and subject connotations of the book, generates a semantic vector representation, and uses cosine similarity to calculate the semantic similarity Sim(A, B) between the semantic vector representation of the book content and the semantic vectors of other books in the library, where A and B represent the semantic vector representation of the current book and other books respectively. Based on the calculation result of the semantic similarity, the unit outputs the relevance or complementarity information between the book and other subject contents;

[0011] Interdisciplinary content association units arrange books with related or complementary contents on adjacent shelves according to their degree of association, making it easier for borrowers to check related books at the same time;

[0012] The knowledge path recommendation module automatically identifies books on the same knowledge path based on the output results of the interdisciplinary association analysis module and generates an optimized virtual knowledge path sorting scheme. The knowledge path recommendation module includes:

[0013] The knowledge path analysis unit analyzes historical borrowing records of related or complementary books based on the correlation analysis of books, identifies combinations of books that are frequently borrowed at the same time, comprehensively analyzes the correlation and historical borrowing frequency, constructs books on the same knowledge path, and arranges the books on the same knowledge path in a logical order;

[0014] A sort number association unit associates the sort numbers of books on the same knowledge path in the virtual space, so that the books on the same knowledge path are arranged in order in the virtual position;

[0015] The path highlighting unit automatically highlights the books on the same knowledge path when the borrower borrows the current book. Based on the book selected by the borrower, it automatically highlights other books related to the currently borrowed book, helping the borrower to quickly find all related books on the same knowledge path (highlighting is performed based on the previous virtual position orderly arrangement, but the actual physical position sorting is still based on the preliminary sorting order, that is, the physical position remains unchanged when highlighting);

[0016] The user interaction module provides an interactive interface for borrowers to view and select recommended books in the highlighted knowledge path, and to view the physical location and related information of the books in the system in real time;

[0017] The data storage module is used to store book images, classification labels, associated information, and borrowing history records.

[0018] Optionally, the image processing module specifically includes:

[0019] A feature extraction unit extracts features from image information through a convolutional neural network (CNN) model and generates a text feature vector. The feature extraction unit performs a convolution operation on the input image information to generate a feature map, then performs a nonlinear transformation on the feature map through an activation function, and then extracts the text feature vector through a pooling operation;

[0020] A subject identification unit, based on the text feature vector, classifies the books by subject through a trained classifier, compares the classification rules with the library, and preliminarily generates a category label for the book;

[0021] After completing the generation of category labels, the sorting number generation unit automatically generates preliminary sorting numbers for books according to the classification rules of books in the library and the current layout of books in the library, so as to facilitate subsequent physical sorting and placement.

[0022] Through the above image processing process, the subject content can be accurately identified from the complex book cover information, and combined with the library's classification system, effective labels and sorting numbers can be generated for book sorting and classification management.

[0023] Optionally, the topic identification unit is based on the text feature vector and further includes a fully connected layer calculation, flattens the feature map after the pooling operation into a one-dimensional vector x, and calculates the score of each classification category through the fully connected layer, which is expressed as:

[0024] y k =W k ·x+b k , where W k is the classification weight matrix, b k is the bias term, y k is the classification score of the kth category;

[0025] Use the Softmax classifier to convert the classification scores into probability values ​​and obtain the probability distribution of each classification: Among them, p k is the probability that the book belongs to the kth category, K is the total number of classification categories, and the category with the largest probability is selected. as the final classification result.

[0026] Optionally, in the sort number generating unit, based on generating a category label of the book, a sort number S is automatically generated according to the classification rules of the library, wherein the classification rules are based on the International Standard Book Number (ISBN) and existing sorting information in the library.

[0027] Optionally, the pre-trained semantic model adopts the BERT semantic model, which captures the semantic relationship between each word and other words by performing a multi-layer self-attention mechanism on the input text (book cover subject information), understands the deep semantic structure in the text through inter-layer transmission and self-attention mechanism calculation, and finally outputs the semantic vector of each word. These semantic vectors represent the content and semantic information of the text and can be used for further analysis or classification tasks. In the present invention, the semantic vectors generated by BERT will be used for interdisciplinary correlation analysis.

[0028] Optionally, the identification of the combination of books that are borrowed simultaneously with high frequency in the knowledge path analysis unit is implemented by a frequent item set mining algorithm, specifically including:

[0029] Calculate support: Calculate the support of each book combination by traversing the borrowing records;

[0030] Filter frequent item sets: filter out frequent item sets according to the support threshold σ;

[0031] Calculate confidence and lift: Calculate the confidence and lift of association rules through frequent item sets;

[0032] Generate association rules: According to the confidence threshold γ, generate highly associated book combinations as candidates for the same knowledge path.

[0033] Optionally, the comprehensive analysis of the relevance and historical borrowing frequency to construct the same knowledge path books includes calculating a comprehensive score: for each book, combining the relevance analysis and the historical borrowing frequency, calculate the comprehensive score of each book s i :s i =α·r i +β·f i , where r i is the relevance of the book, f i is the borrowing frequency of the book, α and β are weight factors, and the books in the same knowledge path are divided into s i The books are sorted in descending order of size, with priority given to books with high relevance and borrowing frequency.

[0034] Optionally, the sequence number association unit specifically includes:

[0035] Assign virtual ranking numbers: After sorting in descending order by comprehensive score, assign virtual ranking numbers V1, V2, ..., V to the books in turn. n , where V1 represents the first book to be displayed in the path, V n For the last book;

[0036] Virtual position association: associate the virtual sort number with the position of the book in the virtual bookshelf to ensure that the books in the same knowledge path are arranged in order in the virtual space.

[0037] Optionally, the path highlighting display unit includes highlighting the same knowledge path books on the virtual bookshelf by color, border or label to highlight the virtual position, and the path highlighting display unit also includes the physical bookshelf position corresponding to the highlighted book.

[0038] Beneficial effects of the present invention:

[0039] The present invention, by combining image recognition technology and natural language processing technology, deeply analyzes the book cover theme and book content, so as to accurately classify and sort the books. By calculating the semantic association, it can automatically identify the content association or complementarity between different books, and generate a knowledge path in combination with historical borrowing records. This weighted sorting method based on association and borrowing frequency ensures that books can be optimized and sorted according to content relevance and reader needs, significantly improves the efficiency of readers in finding related books, and solves the problem of inaccurate book classification and recommendation in the prior art.

[0040] The present invention allocates book sorting numbers in a virtual space and arranges the books in an orderly manner according to a logical order on a virtual bookshelf. With the help of a highlighting function in the virtual bookshelf, readers can quickly identify other books related to the currently borrowed book, and can automatically highlight books on the same path according to the logical order of the knowledge path and the reader's behavior, thereby providing readers with a clear reading navigation path. With the help of the highlighting prompt and physical location synchronization functions of the virtual interface, readers can not only quickly browse on the virtual interface, but also easily find the actual location of related books in the physical library, thereby improving the user's reading experience and borrowing efficiency.

[0041] The present invention automatically generates a knowledge path through a frequent item set mining algorithm combined with the reader's historical borrowing behavior, and optimizes and sorts the books in the path according to their relevance and borrowing frequency. It not only arranges content-related books in order, but also helps readers form personalized knowledge path recommendations based on historical borrowing records. By automatically analyzing the relevance of books and readers' borrowing behavior, it effectively improves the personalization and intelligence of recommendations, allowing readers to more easily discover other high-value books related to the current book, significantly improving the library's book recommendation and management methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 Schematic diagram of a system functional module unit according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. At the same time, it is explained here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments, and are not intended to specifically limit the present invention.

[0045] It should be noted that the references to "one embodiment", "an embodiment", "an exemplary embodiment", "some embodiments" and the like in the specification indicate that the embodiments described may include specific features, structures or characteristics, but not every embodiment may include the specific features, structures or characteristics. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0046] In general, a term can be understood, at least in part, from its use in context. For example, depending, at least in part, on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending, at least in part, on the context, allow for the presence of other factors that are not necessarily explicitly described.

[0047] like Figure 1 As shown, a library book sorting system based on image recognition includes:

[0048] Image acquisition module: obtains image information of the book cover. The image acquisition module includes a camera to collect image data of the book cover or spine;

[0049] Image processing module: connected to the image acquisition module, processes the acquired image information and identifies the subject on the book cover. The image processing module classifies the book cover subject through the convolutional neural network (CNN) model and preliminarily generates the category label and sorting number of the book according to the library classification rules;

[0050] Interdisciplinary correlation analysis module: used to analyze the content correlation between interdisciplinary books after preliminary book sorting. The interdisciplinary correlation analysis module includes:

[0051] The natural language processing unit, based on the subject on the book cover identified by the image processing module, deeply understands the book content, analyzes the relevance or complementarity with other subject contents, uses the pre-trained semantic model to deeply understand the book content, extracts the core subject, key concepts and subject connotations of the book, generates a semantic vector representation, and uses cosine similarity to calculate the semantic similarity Sim(A, B) between the semantic vector representation of the book content and the semantic vectors of other books in the library, where A and B represent the semantic vector representation of the current book and other books respectively. The formula is as follows:

[0052] Among them, v A and v B They represent the semantic vectors of the current book and other books respectively, Sim(A,B) is the semantic similarity; v A ·v B is the inner product of the semantic vector, which calculates the sum of the products of two vectors in each dimension. A | and |v B | is the modulus of the vector, which indicates the length of the vector. The formula is as follows: The cosine similarity is used to obtain the similarity between the semantic vectors of two books. The value is between [-1, 1], where 1 indicates complete similarity and 0 indicates irrelevance. The association results are output in descending order of similarity. The list of related books is used for subsequent sorting optimization and recommendation. Based on the calculation results of semantic similarity, the association or complementarity information between the book and other subject contents is output;

[0053] Interdisciplinary content association units arrange books with related or complementary contents on adjacent shelves according to their degree of association, making it easier for borrowers to check related books at the same time;

[0054] The knowledge path recommendation module automatically identifies books on the same knowledge path based on the output results of the interdisciplinary association analysis module and generates an optimized virtual knowledge path sorting scheme. The knowledge path recommendation module includes:

[0055] The knowledge path analysis unit analyzes historical borrowing records of related or complementary books based on the correlation analysis of books, identifies combinations of books that are frequently borrowed at the same time, comprehensively analyzes the correlation and historical borrowing frequency, constructs books on the same knowledge path, and arranges the books on the same knowledge path in a logical order;

[0056] A sort number association unit associates the sort numbers of books on the same knowledge path in the virtual space, so that the books on the same knowledge path are arranged in order in the virtual position;

[0057] The path highlighting unit automatically highlights the books on the same knowledge path when the borrower borrows the current book. Based on the book selected by the borrower, it automatically highlights other books related to the currently borrowed book, helping the borrower to quickly find all related books on the same knowledge path (highlighting is performed based on the previous virtual position orderly arrangement, but the actual physical position sorting is still based on the preliminary sorting order, that is, the physical position remains unchanged when highlighting);

[0058] The user interaction module provides an interactive interface for borrowers to view and select recommended books in the highlighted knowledge path, and to view the physical location and related information of the books in the system in real time;

[0059] The data storage module is used to store book images, classification labels, associated information, and borrowing history records.

[0060] The image processing module specifically includes:

[0061] The feature extraction unit extracts features from the image information through a convolutional neural network (CNN) model and generates a text feature vector. The feature extraction unit performs a convolution operation on the input image information to generate a feature map, and then performs a nonlinear transformation on the feature map through an activation function, and then extracts the text feature vector through a pooling operation;

[0062] The feature extraction unit is implemented through a convolutional neural network (CNN) model, as follows:

[0063] 1. Convolution operation: Perform convolution operation on the input image I to generate a feature map:

[0064] Among them, I(i,j) is the pixel value of the image, K(m,n) is the convolution kernel, b is the bias term, and F(i,j) is the feature map generated after convolution;

[0065] 2. Activation function: The convolutional feature map is transformed nonlinearly through the ReLU activation function to improve the expressive power of the model: A(i,j)=max(0,F(i,j)), where A(i,j) is the feature value after the activation function processing;

[0066] 3. Pooling operation: The pooling operation reduces the size of the feature map and extracts the main features:

[0067] P(i,j)=max{A(i,j),A(i+1,j),A(i,j+1),A(i+1,j+1)}, where P(i,j) is the feature value after pooling. The maximum pooling operation can extract the local maximum feature.

[0068] The subject recognition unit classifies books by subject based on text feature vectors through a trained classifier, compares it with the library's classification rules, and preliminarily generates category labels for books;

[0069] After completing the generation of category labels, the sorting number generation unit automatically generates preliminary sorting numbers for books according to the classification rules of books in the library and the current layout of books in the library, so as to facilitate subsequent physical sorting and placement.

[0070] Through the above image processing process, the subject content can be accurately identified from the complex book cover information, and combined with the library's classification system, effective labels and sorting numbers can be generated for book sorting and classification management.

[0071] The topic recognition unit is based on the text feature vector and also includes a fully connected layer calculation, which flattens the feature map after the pooling operation into a one-dimensional vector x, and calculates the score of each classification category through the fully connected layer, expressed as:

[0072] y k =W k ·x+b k , where W k is the classification weight matrix, b k is the bias term, y k is the classification score of the kth category;

[0073] Use the Softmax classifier to convert the classification scores into probability values ​​and obtain the probability distribution of each classification: Among them, p k is the probability that the book belongs to the kth category, K is the total number of classification categories, and the category with the largest probability is selected. as the final classification result.

[0074] In the sort number generation unit, based on the category label of the generated book, the sort number S is automatically generated according to the classification rules of the library. The classification rules are based on the International Standard Book Number (ISBN) and the existing sorting information in the library. The calculation of the sort number includes the following steps:

[0075] 1. ISBN code extraction: Identify the ISBN code of the book through the classification model as the initial unique identifier of the book:

[0076] ISBN=Extracted from book cover text via OCR / NLP methods;

[0077] 2. In-library classification mapping: Map the ISBN code with the library's existing classification system to generate the basic classification number C: C = f mapping (ISBN), where f mapping It represents the mapping process, which associates ISBN with the library classification system;

[0078] 3. Bookshelf sort number calculation: Calculate the specific sort number S of the book according to the physical sorting rules of the book in the current classification: Among them, n is the number of times a book is borrowed in this category, and N is the total number of books in this category. In this way, frequently borrowed books are prioritized.

[0079] The pre-trained semantic model adopts the BERT semantic model. The BERT semantic model processes the input text (book cover subject information) through a multi-layer self-attention mechanism to capture the semantic relationship between each word and other words. After inter-layer transmission and calculation of the self-attention mechanism, it understands the deep semantic structure in the text and finally outputs the semantic vector of each word. These semantic vectors represent the content and semantic information of the text and can be used for further analysis or classification tasks. In the present invention, the semantic vectors generated by BERT will be used for interdisciplinary correlation analysis.

[0080] The pre-trained BERT semantic model is used to perform deep semantic understanding on the cover subject information extracted by the image processing module. The pre-trained language model converts the input text into a semantic vector representation. The processing process is as follows:

[0081] Input representation: The input topic information is represented in the BERT input format. The input is text T, which BERT represents as a vector sequence T = {t1, t2, ..., t n}, where t i Represents the vector representation of the i-th word;

[0082] BERT inter-layer transfer: The BERT model consists of a multi-layer self-attention mechanism and a feed-forward neural network. The output update of each layer is expressed as follows:

[0083] h (l) =Attention(h (l-1) )+FeedForward(h (l-1) ), where h (l) represents the word vector of the lth layer, Attention and FeedForward represent the self-attention mechanism and feedforward network respectively;

[0084] Self-attention mechanism: BERT's self-attention mechanism is calculated as follows:

[0085] Among them, Q (query), K (key) and V (value) are matrices obtained by linear transformation of word vectors, d k is a scaling factor, and the softmax function normalizes the attention distribution;

[0086] Output semantic vector: The BERT model finally outputs the semantic vector v of each input word T , which represents the deep semantic information of the input text: v T =BERT(T). In the present invention, the BERT model outputs a multi-dimensional semantic vector of the book cover subject information for subsequent cross-disciplinary correlation analysis.

[0087] The identification of high-frequency book combinations that are borrowed simultaneously in the knowledge path analysis unit is achieved through a frequent item set mining algorithm, which includes:

[0088] Calculate support: Calculate the support of each book combination by traversing the borrowing records;

[0089] Filter frequent item sets: filter out frequent item sets according to the support threshold σ;

[0090] Calculate confidence and lift: Calculate the confidence and lift of association rules through frequent item sets;

[0091] Generate association rules: According to the confidence threshold γ, generate highly associated book combinations as candidates for the same knowledge path.

[0092] 1. Support: Support refers to the frequency with which a group of books are borrowed at the same time in the entire borrowing data set. Let I = {i1,i2,...,i n} represents the book collection, D represents the total number of borrowing records, and the book collection The support degree Support(X) is defined as:

[0093] in, indicates that the book set X is a subset of the borrowing record d, |D| is the total number of borrowing records, is the number of borrowing records containing book set X. The support measures the frequency of a specific book combination in the entire borrowing data. If the support of a book combination is high, it means that these books are often borrowed together and are potential knowledge paths.

[0094] 2. Confidence: Confidence is used to measure the possibility that after borrowing a certain set of books, the borrower will also borrow another set of books. Given two book sets X and Y, the definition of confidence (X→Y) is:

[0095] Among them, X∪Y represents the union of book sets X and Y, represents the frequency of borrowers borrowing these two sets of books at the same time, and Support(X) represents the support of borrowed book set X. Confidence indicates the possibility that if borrowers borrow book X, they will also borrow book Y. High confidence means that X and Y have a strong association and are often borrowed together, which meets the requirements of building knowledge paths.

[0096] 3. Lift: Lift is used to measure the strength of the association between two book sets. If the lift is greater than 1, it means that there is a positive association between books X and Y. The definition of lift (X→Y) is:

[0097]

[0098] The lift represents the ratio of the probability that books X and Y appear in the borrowing record at the same time to the probability that they appear in the borrowing record independently. A lift greater than 1 indicates that they are strongly associated and suitable for generating the same knowledge path.

[0099] 4. Calculation of frequent itemsets: In frequent itemset mining, the above support is used to filter out book combinations that are frequently borrowed together. A book set X is a frequent itemset if and only if its support is higher than the set support threshold σ: Support(X) ≥ σ. By setting a minimum support threshold, book combinations that are frequently borrowed together in borrowing records are filtered out to avoid processing rare and uncommon combinations.

[0100] 5. Generation of association rules: After identifying frequent item sets, generate association rules X→Y, which means that if a borrower borrows book set X, then they will also borrow book set Y. The criterion for rule generation is to meet a certain confidence threshold γ:

[0101] Confidence(X→Y)≥γ, after generating association rules, these rules can be used to recommend the next book combination that the borrower may be interested in, helping to generate the same knowledge path.

[0102] Comprehensively analyze the relevance and historical borrowing frequency to construct the same knowledge path books, including calculating the comprehensive score: for each book, combine the relevance analysis and historical borrowing frequency to calculate the comprehensive score of each book s i :s i =α·r i +β·f i , where ri is the relevance of the book, f i is the borrowing frequency of the book, α and β are weight factors, and the books in the same knowledge path are divided into s i The books are sorted in descending order of size, with priority given to books with high relevance and borrowing frequency.

[0103] The sequence number association unit specifically includes:

[0104] Assign virtual ranking numbers: After sorting in descending order by comprehensive score, assign virtual ranking numbers V1, V2, ..., V to the books in turn. n , where V1 represents the first book to be displayed in the path, V n For the last book;

[0105] Virtual position association: associate the virtual sort number with the position of the book in the virtual bookshelf to ensure that the books in the same knowledge path are arranged in order in the virtual space.

[0106] The path highlighting display unit includes highlighting the books on the same knowledge path on the virtual bookshelf through color, border or label to highlight the virtual position. The path highlighting display unit also includes the physical bookshelf position corresponding to the highlighted books to help readers quickly locate books in the same knowledge path in the actual library.

[0107] Design and function of interactive interface:

[0108] When a borrower enters the interface, the book he is browsing or borrowing is displayed, and the knowledge path related to the book is graphically displayed on the interface. The knowledge path is presented in a visual form, showing other books in the path. These books are arranged in logical order such as relevance and borrowing frequency.

[0109] Each book is displayed on the interface in the form of book cover, title, etc., and related books on the same knowledge path are highlighted, allowing users to easily identify and select them.

[0110] Borrowers can view detailed information by clicking on highlighted books, such as the book's summary, classification, borrowing records of related books, etc. Each highlighted book is accompanied by an operation button, including "View Details", "Borrow", and "Recommend", allowing users to make decisions quickly. If the user clicks on a highlighted book, the current knowledge path will be automatically updated and other books related to the newly selected book will be re-highlighted.

[0111] In the virtual interface, the system can display the physical location of highlighted books in real time. Borrowers can click on the book icon to view its location in the actual library. The physical location display is presented in a map view or a bookshelf view to help users find books quickly. Users can use the search or filter function to filter and display specific book categories by books recommended by the knowledge path, or sort by relevance, subject, etc. for easy query.

[0112] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.

[0113] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A library book sorting system based on image recognition, characterized in that: include: Image acquisition module: obtains image information of the book cover, and the image acquisition module includes a camera to collect image data of the book cover or spine; Image processing module: connected to the image acquisition module, processes the acquired image information, identifies the subject on the book cover, and classifies the book cover subject through a convolutional neural network model, and preliminarily generates a category label and a sorting number of the book according to the library classification rules; Interdisciplinary association analysis module: used to analyze the content association between interdisciplinary books after preliminary book sorting, and the interdisciplinary association analysis module includes: The natural language processing unit, based on the subject on the book cover identified by the image processing module, deeply understands the book content, analyzes the relevance or complementarity with other subject contents, uses the pre-trained semantic model to deeply understand the book content, generates a semantic vector representation, and uses cosine similarity to calculate the semantic similarity Sim(A, B) between the semantic vector representation of the book content and the semantic vectors of other books in the library, where A and B represent the semantic vector representation of the current book and other books respectively. Based on the calculation result of the semantic similarity, the unit outputs the relevance or complementarity information between the book and other subject contents; Interdisciplinary content association units, where books with related or complementary contents are arranged on adjacent shelves according to their degree of association; The knowledge path recommendation module automatically identifies books on the same knowledge path based on the output results of the interdisciplinary association analysis module and generates an optimized virtual knowledge path sorting scheme. The knowledge path recommendation module includes: The knowledge path analysis unit analyzes historical borrowing records of related or complementary books based on the correlation analysis of books, identifies combinations of books that are frequently borrowed at the same time, comprehensively analyzes the correlation and historical borrowing frequency, constructs books on the same knowledge path, and arranges the books on the same knowledge path in a logical order; A sort number association unit associates the sort numbers of books on the same knowledge path in the virtual space, so that the books on the same knowledge path are arranged in order in the virtual position; The path highlighting unit automatically highlights the books on the same knowledge path when the borrower borrows the current book. Based on the book selected by the borrower, it automatically highlights other books related to the currently borrowed book, helping the borrower to quickly find all related books on the same knowledge path. The user interaction module provides an interactive interface for borrowers to view and select recommended books on the highlighted knowledge path; The data storage module is used to store book images, classification labels, associated information, and borrowing history records.

2. A library book sorting system based on image recognition according to claim 1, characterized in that: The image processing module specifically includes: A feature extraction unit extracts features from image information through a convolutional neural network model and generates a text feature vector. The feature extraction unit performs a convolution operation on the input image information to generate a feature map, then performs a nonlinear transformation on the feature map through an activation function, and then extracts the text feature vector through a pooling operation; A subject identification unit, based on the text feature vector, classifies the books by subject through a trained classifier, compares the classification rules with the library, and preliminarily generates a category label for the book; After completing the generation of the category label, the sorting number generation unit automatically generates a preliminary sorting number for the book according to the classification rules of the book in the library and the current layout of the books in the library.

3. A library book sorting system based on image recognition according to claim 2, characterized in that: The topic identification unit is based on the text feature vector and also includes a fully connected layer calculation, which flattens the feature map after the pooling operation into a one-dimensional vector x, and calculates the score of each classification category through the fully connected layer, which is expressed as: y k =W k ·x+b k , where W k is the classification weight matrix, b k is the bias term, y k is the classification score of the kth category; Use the Softmax classifier to convert the classification scores into probability values ​​and obtain the probability distribution of each classification: Among them, p k is the probability that the book belongs to the kth category, K is the total number of classification categories, and the category with the largest probability is selected. as the final classification result.

4. A library book sorting system based on image recognition according to claim 3, characterized in that: In the sequence number generating unit, the sequence number S is automatically generated based on the category label of the generated book and according to the classification rules of the library.

5. The library book sorting system based on image recognition according to claim 1, characterized in that: The pre-trained semantic model adopts the BERT semantic model, which processes the input text through a multi-layer self-attention mechanism to capture the semantic relationship between each word and other words, understands the deep semantic structure in the text through inter-layer transmission and calculation of the self-attention mechanism, and finally outputs the semantic vector of each word.

6. The library book sorting system based on image recognition according to claim 1, characterized in that: The identification of the combination of books that are borrowed at the same time with high frequency in the knowledge path analysis unit is realized by a frequent item set mining algorithm, specifically including: Calculate support: Calculate the support of each book combination by traversing the borrowing records; Filter frequent item sets: filter out frequent item sets according to the support threshold σ; Calculate confidence and lift: Calculate the confidence and lift of association rules through frequent item sets; Generate association rules: According to the confidence threshold γ, generate highly associated book combinations as candidates for the same knowledge path.

7. A library book sorting system based on image recognition according to claim 6, characterized in that: The comprehensive analysis of the relevance and historical borrowing frequency to construct the same knowledge path books includes calculating the comprehensive score: for each book, the comprehensive score of each book is calculated by combining the relevance analysis and the historical borrowing frequency. i :s i =α·r i +β·f i , where r i is the relevance of the book, f i is the borrowing frequency of the book, α and β are weight factors, and the books in the same knowledge path are divided into s i The books are sorted in descending order of size, with books with high relevance and borrowing frequency being prioritized.

8. The library book sorting system based on image recognition according to claim 7, characterized in that: The sequence number association unit specifically includes: Assign virtual ranking numbers: After sorting in descending order by comprehensive score, assign virtual ranking numbers V1, V2, ..., V to the books in turn. n , where V1 represents the first book to be displayed in the path, V n For the last book; Virtual position association: associate the virtual sort number with the position of the book in the virtual bookshelf to ensure that the books in the same knowledge path are arranged in order in the virtual space.

9. The library book sorting system based on image recognition according to claim 1, characterized in that: The path highlighting display unit includes highlighting the same knowledge path books on the virtual bookshelf through color, border or label to highlight the virtual position, and the path highlighting display unit also includes the physical bookshelf position corresponding to the highlighted book.

Images