Book retrieval method and system based on big data

Abstract

The invention discloses a book retrieval method and system based on big data, and relates to the technical field of information retrieval, and the method comprises the steps: collecting book multi-dimensional data through a distributed crawler, updating the data in real time, and generating a structured data set; through a multi-dimensional semantic matching algorithm, the semantic matching degree of the recommendation list and the user retrieval requirement is optimized, the ratio of the popularity to the weight of the reasoning hierarchical knowledge graph is adjusted, and a personalized recommendation book is generated; presenting a personalized recommendation book in real time according to an interactive display mode, recording user behaviors as feedback, and dynamically updating the reasoning hierarchical knowledge graph by utilizing real-time stream processing; in combination with a personalized recommendation scoring model, the knowledge graph is further optimized by interactively and visually presenting the recommendation book and collecting the user feedback, so that the real-time performance, the recommendation precision and the user experience are improved, and efficient and reliable technical support is provided for large-scale book retrieval and intelligent recommendation.

Description

Technical Field

[0001] The present invention relates to the field of information retrieval technology, and in particular to a book retrieval method and system based on big data. Background Art

[0002] In recent years, with the rapid development of big data technologies and the widespread application of deep learning models in natural language processing, the intelligence level of information retrieval systems has been significantly improved. In the field of book retrieval, traditional methods often rely on simple keyword-based matching algorithms or rule-based recommendation systems. While easy to implement, these methods lack a precise understanding of user needs and a degree of personalized recommendation effectiveness. They are unable to effectively tap into users' potential needs and struggle to process the complex relationships within massive amounts of book data. As users' demands for efficient and intelligent retrieval experiences increase, retrieval methods based on knowledge graphs and deep learning technologies are gaining increasing attention.

[0003] While existing knowledge graph-based retrieval methods can achieve a certain degree of semantic association in knowledge, they still have many shortcomings in terms of personalized, real-time, and dynamic updating of book recommendations. For example, existing methods have limited real-time response capabilities to user behavioral feedback, making it difficult to effectively and dynamically adjust the structure and weighting of the knowledge graph. Furthermore, when generating recommendation lists, existing algorithms inadequately optimize semantic matching, resulting in low coverage and relevance of recommendation results to user needs. These shortcomings prevent existing technologies from fully leveraging the advantages of knowledge graphs and big data processing. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a book retrieval method based on big data to solve the problems of real-time dynamic updating of knowledge graphs and optimizing recommendation accuracy in the existing technology.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, the present invention provides a book retrieval method based on big data, which comprises collecting multi-dimensional data of books through a distributed crawler and updating it in real time to generate a structured data set; Based on structured data sets, we use deep learning and relationship extraction algorithms to generate dynamic knowledge graphs, and use the inference engine to deduce implicit relationships to obtain reasoning-level knowledge graphs. Based on the inference hierarchical knowledge graph, combined with time series analysis, book popularity is predicted, book priority scores are generated, and a recommendation list is formed; Through the multi-dimensional semantic matching algorithm, the semantic matching degree between the recommendation list and the user's search requirements is optimized, and the popularity and weight ratio of the inference level knowledge graph are adjusted to generate personalized recommended book lists; Personalized book recommendations are presented in real time through interactive display, user behavior is recorded as feedback, and the inference-level knowledge graph is dynamically updated using real-time stream processing.

[0007] As a preferred solution of the big data-based book retrieval method described in the present invention, the multidimensional book data includes book metadata, book content data and user interaction data.

[0008] As a preferred solution of the book retrieval method based on big data of the present invention, the specific steps of generating a structured data set are as follows: Preprocess book metadata, book content data, and user interaction data; Generate user interest vectors based on preprocessed user interaction data, and fuse the user interest vectors with preprocessed book metadata to form a user-book matrix; The user-book matrix is ​​associated with the book content data through a unique identifier, and a content vector is generated through a natural language processing architecture; Use streaming data to process content vectors and update pre-processed book content data; The real-time updated book content data is stored in a distributed database to form a structured data set.

[0009] As a preferred solution of the big data-based book retrieval method of the present invention, the specific steps of obtaining the reasoning hierarchical knowledge graph are as follows: For structured datasets, we apply the conditional random field model to identify entities and relationships in text and output the semantically annotated text structure. Use a multi-layer long short-term memory network to perform deep semantic modeling on the annotated text structure and generate contextual semantic features of entities and relationships; Input contextual semantic features into the graph neural network, map entities into book nodes in the graph, and map relationships into edges in the graph to construct a knowledge graph; Based on the correlation between book nodes and edges in the knowledge graph, the edges in the knowledge graph are expanded by inferring implicit relationships through the inference engine to form a hierarchical graph structure; According to the correlation degree of each level in the hierarchical graph structure, the derived implicit relationships are sorted to increase the depth of the graph and form an inference hierarchical knowledge graph.

[0010] As a preferred solution of the big data-based book retrieval method described in the present invention, the specific steps are as follows: based on the reasoning hierarchical knowledge graph, combined with time series analysis, the popularity of books is predicted, the book priority score is generated, and a recommendation list is formed. Using a graph neural network algorithm, the embedding vector of each book node is updated by inferring the information of adjacent book nodes in the hierarchical knowledge graph. ; The embedding vector of the book node The user interest vector of the book node is mapped into a multidimensional semantic space, and the semantic similarity score between each book node and the user is calculated; Combine the user's historical interaction data to build a user behavior vector; The user behavior vector is combined with the semantic similarity score through weighted averaging to generate a personalized semantic matching score; The embedding vector of the book node Combined with the popularity data of historical books, a time series prediction model is formed, and the expression is: ; ; in, Represents a book node At a future point in time The predicted value of heat, The index variable representing the book node, represents the long short-term memory network model, Represents a book node Historical heat data series at past time points, An index variable representing a time point, express Model parameters; Based on the combination of personalized semantic matching score and heat prediction value, by introducing the time weight factor Generate the priority score for each book using the expression: ; in, Represents a book node The final priority score of represents a nonlinear activation function, Representation and book nodes The set of directly connected adjacent book nodes, Represents a book node All adjacent book nodes Perform the sum operation, The index variable representing the adjacent book node, Represents a book node and adjacent book nodes The weight value between represents the Hadamard product, Represents adjacent book nodes and users Enhanced similarity measure in multidimensional semantic space, Represents adjacent book nodes At the time point The predicted value of heat, Represents adjacent book nodes The weight value in the recommended state, Represents adjacent book nodes The value of information dissemination intensity between different fields; Defining priority scoring thresholds ; Select Satisfaction Priority Score The books are combined into a recommended book list to form a recommendation list.

[0011] As a preferred solution of the big data-based book retrieval method of the present invention, the specific steps of generating a personalized recommended book list are as follows: Utilize the BERT word embedding model to convert each word in the user's search query into a high-dimensional vector ; Use cosine similarity to evaluate each book in the recommendation list and the high-dimensional vector The semantic matching score between , the expression is: ; in, The semantic vector of the book node representing each book in the recommendation list; Optimize semantic matching scores through quantum particle swarm optimization The matching process; Re-evaluate the popularity and weight ratio of book nodes in the knowledge graph based on the optimized semantic matching score; Combining the optimized semantic matching score and the updated knowledge graph structure to output a personalized score, the expression is: ; in, Represents a book node Personalized ratings, To adjust the parameters; Defining personalized scoring thresholds ; Choose to meet personalized ratings as a personalized recommendation list.

[0012] As a preferred solution of the big data-based book retrieval method of the present invention, the specific steps of dynamically updating the inference level knowledge graph using real-time stream processing are as follows: Use dynamic rendering to visualize personalized recommended book lists and collect feedback data between users and personalized recommended book lists in real time; Based on the feedback data, the feedback weight is calculated in combination with the weight adjustment parameters; Based on the feedback weight, the popularity of book nodes in the knowledge graph is dynamically updated, and the edge weights between book nodes are adjusted to optimize the graph structure and weight distribution; Use the adjusted book nodes to regenerate the graph embedding vector through the graph neural network; The book list finally selected by the user is used as feedback to update the user interest vector.

[0013] In a second aspect, the present invention provides a book retrieval system based on big data, comprising a data acquisition module, a reasoning module, a recommendation algorithm module, an optimization module, and a feedback module; The data collection module is used to collect multi-dimensional data of books through distributed crawlers and update them in real time to generate structured data sets; The reasoning module is used to generate a dynamic knowledge graph based on a structured data set using deep learning and relationship extraction algorithms, and to deduce implicit relationships through an inference engine to obtain a reasoning hierarchical knowledge graph; The recommendation algorithm module is used to predict book popularity based on the inference hierarchical knowledge graph and time series analysis, generate book priority scores, and form a recommendation list; The optimization module is used to optimize the semantic matching degree between the recommendation list and the user's search requirements through a multi-dimensional semantic matching algorithm, adjust the popularity and weight ratio of the reasoning level knowledge graph, and generate a personalized recommended book list; The feedback module is used to present personalized recommended book lists in real time based on an interactive display method, record user behavior as feedback, and dynamically update the inference hierarchical knowledge graph using real-time stream processing.

[0014] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the big data-based book retrieval method as described in the first aspect of the present invention is implemented.

[0015] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the big data-based book retrieval method as described in the first aspect of the present invention.

[0016] The beneficial effects of the present invention are as follows: the present invention utilizes distributed crawlers and big data processing technology to efficiently collect and update multi-dimensional data of books, combines deep learning models and knowledge graph technology to generate inference hierarchical knowledge graphs, and predicts book popularity through time series analysis, and generates comprehensive priority scores to achieve accurate book recommendation functions; dynamically updates the structure and weight distribution of the knowledge graph, makes full use of user behavior feedback, and ensures that the recommendation results can reflect changes in user needs in real time; optimizes semantic matching through quantum particle swarm algorithm, making recommendation results more accurate in relevance and user preferences; combines personalized recommendation scoring model, presents recommended book lists through interactive visualization and collects user feedback to further optimize the knowledge graph, improves real-time performance, recommendation accuracy and user experience, and provides efficient and reliable technical support for large-scale book retrieval and intelligent recommendation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 This is a flow chart of the book retrieval method based on big data in Example 1.

[0019] Figure 2 This is a module diagram of the big data-based book retrieval system in Example 1. DETAILED DESCRIPTION

[0020] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0021] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0022] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0023] Example 1, with reference to Figure 1 and Figure 2, which is the first embodiment of the present invention, provides a book retrieval method based on big data, comprising the following steps: S1. Collect multi-dimensional data of books through distributed crawlers and update them in real time to generate structured data sets.

[0024] Furthermore, the book metadata, book content data and user interaction data are pre-processed; Specifically, the preprocessing of book metadata includes deduplication, filling in missing values ​​(such as ISBN, publication date, etc.), standardizing field formats (such as unifying date formats and price fields into numerical form), and correcting spelling or format errors; the preprocessing of book content data includes removing irrelevant characters, punctuation marks and stop words, unifying text formats, performing word segmentation and stemming, and using natural language processing technology to normalize the text (such as removing duplicate content); the preprocessing of user interaction data includes deduplication and filtering outliers (such as unreasonable ratings or invalid interaction records), filling in missing rating information, and standardizing user behavior data (such as unifying rating standards, timestamp formats, etc.) to ensure data integrity and consistency.

[0025] Generate user interest vectors based on preprocessed user interaction data, and fuse the user interest vectors with preprocessed book metadata to form a user-book matrix; Specifically, the method for generating a user interest vector is as follows: when a user rates a book, the rating is directly used as the user's interest value in the book; for example, the rating is an integer from 1 to 5, which represents the user's interest intensity. The interest vector is generated through the user's click and browsing behavior using a weighting mechanism (such as each click or browse has a weight of 1, and the last behavior has a higher weight, etc.); among them, books purchased or collected by users are regarded as strong interest behaviors and are given a higher weight.

[0026] The user-book matrix is ​​associated with the book content data through a unique identifier (such as ISBN), and a content vector is generated through a natural language processing architecture; Use streaming data to process content vectors and update pre-processed book content data; The real-time updated book content data is stored in a distributed database to form a structured data set.

[0027] S2. Based on structured data sets, deep learning and relationship extraction algorithms are used to generate dynamic knowledge graphs, and implicit relationships are deduced through the inference engine to obtain inference-level knowledge graphs.

[0028] Furthermore, for structured datasets, we apply the Conditional Random Field (CRF) model to identify entities and relations in the text and output the semantically annotated text structure. The conditional random field (CRF) model is a discriminative model that can simultaneously consider the global contextual information in the text sequence. When identifying entities and relationships, it can integrate the dependencies between previous and next words, avoid making decisions based solely on local features, and improve the accuracy and consistency of the annotation results. It is suitable for processing complex natural language texts.

[0029] Use a multi-layer long short-term memory (LSTM) network to perform deep semantic modeling on the annotated text structure and generate contextual semantic features of entities and relationships; Input contextual semantic features into the graph neural network (GNN), map entities into book nodes in the graph, and map relationships into edges in the graph to build a knowledge graph; It should be noted that the contextual semantic features extracted by the multi-layer long short-term memory network (LSTM) are passed as input to the graph neural network (GNN); the graph neural network will further enhance the representation ability of entities and relationships through the structure of book nodes and edges; specifically, entities will be represented as book nodes in the graph, and relationships will be used as edges to connect the corresponding book nodes to form a preliminary graph structure; in this process, GNN can not only use the direct relationship between book nodes to construct the graph, but also obtain deep information of each book node from a global scope through iterative calculation, further improving the expression ability of the knowledge graph; through this method, the knowledge graph not only accurately reflects the direct connection between entities, but also captures more complex semantic associations, and finally constructs a hierarchical and information-rich graph structure, providing a basis for subsequent reasoning and analysis.

[0030] Based on the correlation between book nodes and edges in the knowledge graph, the edges in the knowledge graph are expanded by inferring implicit relationships through the inference engine to form a hierarchical graph structure; Among them, "implicit relationships" mainly include potential connections derived from existing knowledge through reasoning, such as the similarity between books, the potential connection between authors and works, and the implicit connection between user interests and book content.

[0031] According to the correlation degree of each level in the hierarchical graph structure, the derived implicit relationships are sorted to increase the depth of the graph, which is used to ensure that important relationships in the reasoning hierarchical knowledge graph have a higher priority, thereby improving the accuracy of the graph and the query efficiency, and finally forming a reasoning hierarchical knowledge graph.

[0032] S3. Based on the inference hierarchical knowledge graph and combined with time series analysis, the popularity of books is predicted, and book priority scores are generated to form a recommendation list.

[0033] Furthermore, using the graph neural network algorithm, the embedding vector of each book node is updated by inferring the information of adjacent book nodes in the hierarchical knowledge graph. ; The embedding vector of the book node The user interest vector of the book node is mapped into a multidimensional semantic space, and the semantic similarity score between each book node and the user is calculated; Preferably, by calculating the semantic similarity score between each book node and the user, it can accurately match the user's interest preferences, thereby significantly improving the degree of personalization and recommendation accuracy, so as to better understand the user's needs, provide more relevant and personalized book recommendations, and enhance user experience and satisfaction.

[0034] Combine the user's historical interaction data to build a user behavior vector; Specifically, we first collect all the user's past interaction records, including clicks, browsing, purchases, ratings, and comments, and arrange these behaviors in chronological order. Then, we assign different weights to each behavior to reflect its importance. Furthermore, for example, purchasing behavior is weighted higher than clicking behavior. These weighted behaviors are then converted into numerical features, such as using one-hot encoding or embedding vectors to represent different types of interaction behaviors. All behavioral features are then integrated into a fixed-length vector through aggregation functions (such as average, sum, or most recent interaction). Ultimately, a user behavior vector is obtained that can fully reflect the user's interests and preferences. This vector not only captures the user's long-term preferences but also reflects recent changes in interests.

[0035] The user behavior vector is combined with the semantic similarity score through weighted averaging to generate a personalized semantic matching score; The embedding vector of the book node Combined with the popularity data of historical books, a time series prediction model is formed, and the expression is: ; ; in, Represents a book node At a future point in time The predicted value of heat, The index variable representing the book node, represents the long short-term memory network model, Represents a book node Historical heat data series at past time points, An index variable representing a time point, express Model parameters; It should be noted that the specific steps for obtaining the historical popularity data series are as follows: first, behavioral data such as clicks, browsing, purchases, and ratings are extracted from user interaction records, and the timestamps of these behaviors are combined to quantify the popularity of books at different time points. At the same time, the number of mentions and discussion heat on external platforms such as social media, review websites, and news reports are collected. Natural language processing technology is used to analyze the text content to evaluate the social influence of the book. In addition, seasonal and cyclical factors are also considered; Furthermore, for example, the impact of holidays, the start of the school year, or specific festivals on book popularity, all of these multi-source data are cleaned, standardized, and mapped onto a unified timeline. Through weighted averaging or other aggregation methods, the specific popularity value of each book node at each point in time is generated, thus forming a complete and accurate historical book popularity data series.

[0036] Based on the combination of personalized semantic matching score and heat prediction value, by introducing the time weight factor To adjust the importance of the current popularity of the book node to the future time point, generate the priority score of each book, the expression is: ; in, Represents a book node The final priority score of represents a nonlinear activation function, Representation and book nodes The set of directly connected adjacent book nodes, Represents a book node All adjacent book nodes Perform the sum operation, The index variable representing the adjacent book node, Represents a book node and adjacent book nodes The weight value between represents the Hadamard product, Represents adjacent book nodes and users Enhanced similarity measure in multidimensional semantic space, Represents adjacent book nodes At the time point The predicted value of heat, Represents adjacent book nodes The weight value in the recommended state, Represents adjacent book nodes The value of information dissemination intensity between different fields; It should be noted that for obtaining the time weight factor The specific steps are as follows: First, analyze the historical data of book popularity to identify the trends and cyclical patterns of popularity changes in different time periods. Then, based on these patterns, design a decay function. This function can dynamically adjust the weight value according to the time difference between the current time point and the historical data, ensuring that recent data has a greater impact on the prediction results. Specifically, use an exponential decay function to optimize the selection through methods such as cross-validation to ensure that the model can capture both short-term fluctuations and reflect long-term trends.

[0037] Preferably, this time weighting factor The introduction of significantly improves the accuracy and responsiveness of the forecasting model because it allows the model to pay more attention to recent data, thereby better adapting to the rapidly changing market environment and user interests, while avoiding forecast bias caused by over-reliance on outdated information.

[0038] Defining priority scoring thresholds ; It should be noted that in order to define the priority scoring threshold First, collect and analyze historical user behavior data and book popularity data to determine a reasonable rating range; then, based on statistical analysis and business needs, select an initial threshold The results are evaluated through A / B testing or offline verification, and adjustments are made based on feedback until an optimal threshold is found that ensures both recommendation quality and user experience. Specifically, by integrating user interest vectors, historical behaviors, and book priority ratings, we calculate the rating distribution of all books, then use the percentile method or business rules to set a preliminary threshold, and then iterate and adjust it based on the actual recommendation effect to ensure that the final selected threshold is It can effectively screen out high-quality books that are highly relevant to users, while avoiding recommended book lists that are too sparse or too long.

[0039] Select Satisfaction Priority Score The books are combined into a recommended book list to form a recommendation list.

[0040] S4. Optimize the semantic matching degree between the recommendation list and the user's search requirements through a multi-dimensional semantic matching algorithm, adjust the popularity and weight ratio of the reasoning level knowledge graph, and generate a personalized recommended book list.

[0041] Furthermore, the BERT word embedding model is used to convert each word in the user's search query into a high-dimensional vector. ; Preferably, the BERT word embedding model is used to convert each word in the user's search query into a high-dimensional vector, which can capture the semantic information of the word in the context, thereby more accurately understanding the user's intention and improving the accuracy of subsequent semantic matching.

[0042] Use cosine similarity to evaluate each book in the recommendation list and the high-dimensional vector The semantic matching score between , the expression is: ; in, The semantic vector of the book node representing each book in the recommendation list; Optimize semantic matching scores through quantum particle swarm optimization The matching process; It should be noted that each book node is regarded as a particle, and its position is represented by the semantic vector It indicates that the speed reflects the movement trend of the book node in the search space. By simulating the behavior of particles in quantum mechanics, the quantum particle swarm algorithm can efficiently explore the global optimal solution in high-dimensional space.

[0043] Furthermore, we first initialize the position and velocity of each book node: the position is determined by the semantic vector of the book node of each book in the recommendation list. The speed can be randomly generated or set based on historical data. Then, the semantic matching score corresponding to the current position of each particle (i.e. book node) is calculated. As the fitness function value, the higher the fitness, the more relevant the book node is to the user query. Subsequently, according to the rules of the quantum particle swarm optimization (QPSO) algorithm, the position and speed of each particle are dynamically adjusted in combination with the individual extreme value (the best position found by the particle itself) and the global extreme value (the best position found by all particles); this step utilizes the probability amplitude characteristics of quantum mechanics, allowing particles to jump out of the local optimal solution and find better matching results. The above evaluation and update process is repeated in multiple iterative cycles until the convergence conditions are met or the preset maximum number of iterations is reached. As the iteration proceeds, the particles gradually gather in the area with the highest fitness, that is, around those book nodes that are highly relevant to the user query. Finally, after the iteration, several particles (book nodes) with the highest fitness are selected as the optimized matching results. These book nodes not only have higher semantic similarity scores , and the global search capability of quantum particle swarm optimization (QPSO) has been verified to ensure the quality and diversity of recommendation results.

[0044] Preferably, the semantic matching score is optimized by the quantum particle swarm optimization (QPSO) , significantly enhanced the global search capability, making it easier for particles to jump out of the local optimal solution and find the book nodes most relevant to the user's query, improving the accuracy and coverage of recommendations; Specifically, the Quantum Particle Swarm Optimization (QPSO) algorithm leverages quantum properties to accelerate convergence, reduce iterations, improve optimization efficiency, and ensure rapid delivery of high-quality recommendations. The algorithm's randomness and global search capabilities enable the discovery of more personalized and diverse book selections, avoiding monotonous recommendation results. Furthermore, the QPSO algorithm rapidly adapts to user behavior and market changes, ensuring real-time and relevant recommendations. It also exhibits minimal dependence on initial conditions, strong robustness, and stability, even in the presence of high data noise. This provides users with more diverse, personalized, and timely book recommendations, resulting in a better user experience and higher satisfaction.

[0045] Re-evaluate the popularity and weight ratio of book nodes in the knowledge graph based on the optimized semantic matching score; Specifically, when re-evaluating the popularity and weight ratio of book nodes in the knowledge graph, the original popularity value of each book node is first combined with its optimized semantic matching score. Specifically, the updated popularity value is calculated using a weighted formula: the new popularity value is equal to the original popularity value multiplied by an adjustment parameter , plus the semantic matching score multiplied by The adjustment parameters here are Used to balance the importance of historical popularity and current semantic relevance.

[0046] Furthermore, based on these updated heat values, the edge weights between book nodes in the knowledge graph are further adjusted. At the same time, edges connected to high-similarity book nodes will be given higher weights to enhance the influence of these book nodes in the network. The weights of the book nodes themselves will be adjusted accordingly according to their updated heat values ​​to ensure that those book nodes that are highly relevant to user queries occupy a more important position in the subsequent reasoning process.

[0047] Combining the optimized semantic matching score and the updated knowledge graph structure to output a personalized score, the expression is: ; in, Represents a book node Personalized ratings, To adjust the parameters; Defining personalized scoring thresholds ; Specifically, define personalized scoring thresholds Based on the analysis process of statistics and user behavior data, by collecting historical user behavior data and the acceptance rate of recommended book lists, the distribution characteristics of personalized ratings can be calculated. Kernel density estimation or histogram analysis methods can be used to find the distribution peak of the ratings and select the appropriate quantile as the initial threshold. Then, combined with the diverse preferences of the user group, the rating threshold can be adjusted through A / B testing to balance the recommendation accuracy and coverage. Finally, the personalized rating threshold can be dynamically optimized based on real-time feedback. To ensure that the recommended book list can satisfy the user's interests and take into account the efficiency of recommendation Choose to meet personalized ratings as a personalized recommendation list.

[0048] S5. Present personalized book recommendations in real time through interactive display, record user behavior as feedback, and use real-time stream processing to dynamically update the inference-level knowledge graph.

[0049] Use dynamic rendering to visualize personalized recommended book lists and collect feedback data between users and personalized recommended book lists in real time; Specifically, the React.js front-end framework and the WebSocket protocol are combined to achieve dynamic rendering and collect feedback data on user clicks, favorites, and dwell time in real time.

[0050] Based on the feedback data, the feedback weight is calculated in combination with the weight adjustment parameters; Specifically, different weight adjustment parameters are assigned to the real-time feedback data on user clicks, collections, and dwell time to reflect the degree of influence of each behavior type on the feedback weight; the user's feedback weight is calculated through weighted averaging, and this calculation result will represent the user's feedback intensity on the recommended book list.

[0051] It should be noted that to ensure the adaptability of feedback weights in subsequent steps, they must be normalized to map the feedback weights to a standard range. The normalization process adjusts the feedback weights based on the minimum and maximum values ​​in the current data batch, so that the weights of all user feedback are within a uniform range.

[0052] Furthermore, the normalized feedback weights will be used as input to drive the dynamic update of the knowledge graph. Specifically, these feedback weights will adjust the popularity of book nodes and the edge weights between book nodes in the knowledge graph in real time, thereby optimizing the graph structure. This process ensures that users' real-time behavioral feedback can influence the graph structure and further optimize the recommendation logic. Based on the feedback weight, the popularity of book nodes in the knowledge graph is dynamically updated, and the edge weights between book nodes are adjusted to optimize the graph structure and weight distribution; Specifically, feedback weights are generated by real-time calculation of user behavior and used as the basis for dynamic adjustments to the knowledge graph. Specifically, based on user interactions, the popularity of book nodes in the knowledge graph is dynamically updated to reflect changes in their importance. The weights of the edges between book nodes are also adjusted, thereby optimizing the weight distribution and structure of the entire knowledge graph. This process enables real-time response to the graph state, enabling it to dynamically adapt to recommendation needs as user behavior changes, avoiding the problem of traditional static knowledge graphs' inability to respond to timeliness and personalized needs.

[0053] Use the adjusted book nodes to regenerate the graph embedding vector through the graph neural network; Specifically, after dynamically adjusting the weights of book nodes and edges, the updated knowledge graph is embedded using a graph neural network. This process captures the deep semantic information of book nodes and their adjacencies to generate optimized graph embedding vectors. Compared to traditional shallow embedding methods, this process more accurately models the contextual relevance between book nodes. In particular, in the dynamically updated graph, it fully reflects the impact of user behavior feedback on the graph structure.

[0054] The book list finally selected by the user is used as feedback to update the user interest vector.

[0055] Specifically, after a user completes their interaction with a recommended book list, their final choice is used as new behavioral feedback. This is combined with previous behavioral data to update the user's interest vector, enabling it to more accurately reflect the user's latest preference changes. This process dynamically combines a user's long-term and short-term interests, continuously optimizing our understanding of user needs through real-time updates of the user interest vector.

[0056] This embodiment also provides a book retrieval system based on big data, including: a data acquisition module, an inference module, a recommendation algorithm module, an optimization module and a feedback module; the data acquisition module is used to collect multi-dimensional data of books through a distributed crawler and update it in real time to generate a structured data set; the inference module is used to generate a dynamic knowledge graph based on the structured data set using deep learning and relationship extraction algorithms, and deduce implicit relationships through an inference engine to obtain an inference-level knowledge graph; the recommendation algorithm module is used to predict the popularity of books based on the inference-level knowledge graph in combination with time series analysis, generate book priority scores, and form a recommendation list; the optimization module is used to optimize the semantic matching degree between the recommendation list and the user's retrieval needs through a multi-dimensional semantic matching algorithm, adjust the popularity and weight ratio of the inference-level knowledge graph, and generate a personalized recommended book list; the feedback module is used to present the personalized recommended book list in real time according to an interactive display method, record user behavior as feedback, and dynamically update the inference-level knowledge graph using real-time stream processing.

[0057] This embodiment also provides a computer device suitable for the case of a book retrieval method based on big data, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the book retrieval method based on big data proposed in the above embodiment.

[0058] The computer device may be a terminal, comprising a processor, memory, a communication interface, a display, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The internal memory provides an environment for the operating system and computer programs stored in the non-volatile storage media. The communication interface of the computer device is used to communicate with external terminals via wired or wireless communication. Wireless communication may be achieved via Wi-Fi, a carrier network, NFC (near-field communication), or other technologies. The display of the computer device may be a liquid crystal display or an electronic ink display. The input device may be a touchscreen overlay on the display, buttons, a trackball, or a touchpad on the computer device housing, or an external keyboard, touchpad, or mouse.

[0059] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the big data-based book retrieval method proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0060] In summary, the present invention utilizes distributed crawlers and big data processing technologies to efficiently collect and update multi-dimensional data of books, combines deep learning models and knowledge graph technologies to generate reasoning hierarchical knowledge graphs, and predicts book popularity through time series analysis to generate comprehensive priority scores to achieve accurate book recommendation functions; dynamically updates the structure and weight distribution of the knowledge graph, fully utilizes user behavior feedback, and ensures that the recommendation results can reflect changes in user needs in real time; optimizes semantic matching through quantum particle swarm algorithm to make recommendation results more accurate in relevance and user preferences; combines personalized recommendation scoring models, presents recommended book lists through interactive visualization and collects user feedback to further optimize the knowledge graph, improves real-time performance, recommendation accuracy and user experience, and provides efficient and reliable technical support for large-scale book retrieval and intelligent recommendation.

[0061] Example 2, referring to Table 1, is the second embodiment of the present invention. In order to further verify the technical solution of the present invention, experimental simulation data of a book retrieval method based on big data is provided.

[0062] Distributed crawler technology is used to collect book-related data as the basic data required for the experiment, including book metadata (such as title, author, publisher, etc.), book content data (such as chapters, abstracts, keywords, etc.), and user interaction data (such as clicks, reading time, ratings, etc.). This data will be updated in real time to ensure the timeliness of the dataset.

[0063] First, the collected raw data is preprocessed. Book metadata and content data undergo data cleaning and standardization, removing duplicates and unifying the format. User interaction data is organized and converted into user interest vectors. These interest vectors are then integrated with the book metadata to form a user-book matrix. Subsequently, natural language processing techniques (such as the BERT model) are used to generate content vectors for the book content data.

[0064] Secondly, deep learning techniques are applied to generate structured datasets. Conditional Random Field (CRF) algorithms are used to identify entities and relationships within the text and perform semantic annotation. The annotated text is then passed through a multi-layer long short-term memory (LSTM) network for deep semantic modeling, extracting contextual semantic features. Finally, a knowledge graph is constructed using a graph neural network (GNN).

[0065] Next, the inference engine infers the knowledge graph, deriving implicit relationships and expanding the edges of the knowledge graph, thus forming an inference-level knowledge graph. This graph includes not only known explicit relationships but also inferred implicit connections, creating a deeper hierarchical structure.

[0066] Furthermore, based on the inference-based hierarchical knowledge graph, time series analysis is combined to predict book popularity and generate a priority score. A graph neural network is used to update the embedding vectors of book nodes. Based on historical user interaction data, semantic similarity, and a time series popularity prediction model, a personalized recommendation score is calculated. Finally, the personalized score is combined with semantic matching to generate a personalized book recommendation list.

[0067] Finally, to verify the effectiveness of this method, an A / B test scenario was designed. Group A used a traditional keyword-based recommendation algorithm, while Group B used the book retrieval method based on big data and deep learning proposed in this paper. The performance of the two groups was compared by recording metrics such as user click-through rate, conversion rate, and satisfaction.

[0068] The details are shown in Table 1 below: Table 1 Experimental data comparison table

[0069] Comparing the experimental results, Group B (a big data-based book retrieval method) significantly outperformed Group A (a traditional recommendation system) across all metrics. First, Group B achieved a click-through rate of 82%, a 14 percentage point increase compared to Group A's 68%. This demonstrates that recommendation methods based on big data and deep learning can more accurately meet user interests and provide higher click-through rates.

[0070] Group B also demonstrated a significant advantage in user conversion rate, reaching 48%, a 14 percentage point increase compared to Group A's 34%. This increase in conversion rate directly reflects the effectiveness of the personalized recommendation algorithm, which not only encourages users to click on recommended books but also to make further purchases or borrow them.

[0071] User satisfaction scores were also a significant factor in Group B's strong performance. Group B's satisfaction rating was 4.2 / 5, while Group A's was 3.5 / 5, a difference of 0.7 points. This demonstrates that personalized recommendation algorithms based on semantic matching can significantly enhance user experience and increase user acceptance of recommendation results.

[0072] Furthermore, Group B's improvements in book priority ratings and semantic matching demonstrate the effective integration of the inference-level knowledge graph and deep learning model. The optimized popularity prediction value and personalized recommendation score also significantly improved compared to before optimization. Group B's recommendation system uses a graph neural network to update book nodes in the knowledge graph and optimize semantic matching, resulting in more accurate recommendations that can be adjusted in real time based on user interests and behavior.

[0073] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A book retrieval method based on big data, characterized by: include, Collect multi-dimensional data of books through distributed crawlers and update them in real time to generate structured data sets; Based on structured data sets, deep learning and relationship extraction algorithms are used to generate dynamic knowledge graphs, and implicit relationships are derived through the reasoning engine to obtain reasoning hierarchical knowledge graphs. Based on the reasoning hierarchical knowledge graph, combined with time series analysis, the popularity of books is predicted, and the book priority rating is generated to form a recommendation list; The semantic matching degree between the recommendation list and the user's search requirements is optimized through a multi-dimensional semantic matching algorithm, and the heat and weight ratio of the reasoning level knowledge graph are adjusted to generate a personalized recommended book list; Personalized book recommendations are presented in real time through interactive display, user behavior is recorded as feedback, and the inference-level knowledge graph is dynamically updated using real-time stream processing.

2. The book retrieval method based on big data as claimed in claim 1, characterized in that: The multi-dimensional book data includes book metadata, book content data and user interaction data.

3. The book retrieval method based on big data as claimed in claim 2, characterized in that: The specific steps of generating a structured data set are as follows: Preprocess book metadata, book content data, and user interaction data; Generate a user interest vector based on the preprocessed user interaction data, and fuse the user interest vector with the preprocessed book metadata to form a user-book matrix; The user-book matrix is ​​associated with the book content data through a unique identifier, and a content vector is generated through a natural language processing architecture; Use streaming data to process content vectors and update preprocessed book content data; The real-time updated book content data is stored in a distributed database to form a structured data set.

4. The book retrieval method based on big data as claimed in claim 3, characterized in that: The specific steps of obtaining the reasoning level knowledge graph are as follows: For structured data sets, the conditional random field model is applied to identify entities and relations in the text and output the text structure after semantic annotation; Use a multi-layer long short-term memory network to perform deep semantic modeling on the annotated text structure and generate contextual semantic features of entities and relationships; Input contextual semantic features into the graph neural network, map entities to book nodes in the graph, and map relationships to edges in the graph to build a knowledge graph; Based on the correlation between book nodes and edges in the knowledge graph, the edges in the knowledge graph are expanded by inferring implicit relationships through the inference engine to form a hierarchical graph structure. According to the correlation degree of each level in the hierarchical graph structure, the derived implicit relationships are sorted to increase the depth of the graph and form an inference hierarchical knowledge graph.

5. The book retrieval method based on big data as claimed in claim 4, characterized in that: The specific steps of predicting book popularity based on the reasoning hierarchical knowledge graph and combining time series analysis to generate book priority scores and form a recommendation list are as follows: Using the graph neural network algorithm, the embedding vector of each book node is updated by inferring the information of adjacent book nodes in the hierarchical knowledge graph. ; The embedding vector of the book node Map the user interest vector of the book node to the multi-dimensional semantic space and calculate the semantic similarity score between each book node and the user; Combine the user's historical interaction data to build a user behavior vector; The user behavior vector is combined with the semantic similarity score through weighted averaging to generate a personalized semantic matching score; The embedding vector of the book node Combined with the popularity data of historical books, a time series prediction model is formed, and the expression is: ； ； in, Represents a book node At a future time The predicted value of heat The index variable representing the book node, represents the long short-term memory network model, Represents a book node Historical heat data series at past time points, An index variable representing a time point, express The parameters of the model; Based on the combination of personalized semantic matching score and heat prediction value, the time weight factor is introduced Generate the priority score for each book, the expression is: ； in, Represents a book node The final priority score of represents a nonlinear activation function, Representation and Book Node The set of directly connected adjacent book nodes, Represents a book node All adjacent book nodes Perform a sum operation, The index variable representing the adjacent book node, Represents a book node and adjacent book nodes The weight value between represents the Hadamard product, Represents adjacent book nodes and users Enhanced similarity measure in multidimensional semantic space, Represents adjacent book nodes At the point in time The predicted value of heat Represents adjacent book nodes The weight value in the recommended state, Represents adjacent book nodes The value of the intensity of information dissemination between different fields; Defining Priority Scoring Thresholds ; Select Satisfaction Priority Rating The books are composed into a recommended book list to form a recommendation list.

6. The book search method based on big data as claimed in claim 5, characterized in that: The specific steps of generating a personalized recommended book list are as follows: Using the BERT word embedding model, each word in the user's search query is converted into a high-dimensional vector ; Use cosine similarity to evaluate each book in the recommendation list and the high-dimensional vector The semantic matching score between , the expression is: ； in, The book node semantic vector representing each book in the recommendation list; Optimize semantic matching scores through quantum particle swarm algorithm The matching process; Re-evaluate the popularity and weight ratio of book nodes in the knowledge graph based on the optimized semantic matching score; Combining the optimized semantic matching score and the updated knowledge graph structure to output a personalized score, the expression is: ； in, Represents a book node Personalized ratings of To adjust the parameters; Defining personalized scoring thresholds ; Choose to meet personalized ratings as a personalized recommended list.

7. The book search method based on big data as claimed in claim 6, characterized in that: The specific steps of dynamically updating the reasoning level knowledge graph using real-time stream processing are as follows: Use dynamic rendering to visualize the personalized recommended book list and collect feedback data between users and the personalized recommended book list in real time; Based on the feedback data, the feedback weight is calculated in combination with the weight adjustment parameter; According to the feedback weight, the popularity of the book nodes in the knowledge graph is dynamically updated, and the edge weights between book nodes are adjusted to optimize the graph structure and weight distribution; Use the adjusted book nodes to regenerate the graph embedding vector through the graph neural network; The book list finally selected by the user is used as feedback to update the user interest vector.

8. A book search system based on big data, based on the book search method based on big data according to any one of claims 1 to 7, characterized in that: Including data collection module, reasoning module, recommendation algorithm module, optimization module and feedback module; The data collection module is used to collect multi-dimensional data of books through distributed crawlers and update them in real time to generate structured data sets; The reasoning module is used to generate a dynamic knowledge graph based on a structured data set using deep learning and relationship extraction algorithms, and to derive implicit relationships through an inference engine to obtain a reasoning hierarchical knowledge graph; The recommendation algorithm module is used to predict the popularity of books based on the inference hierarchical knowledge graph and combined with time series analysis, generate book priority scores, and form a recommendation list; The optimization module is used to optimize the semantic matching degree between the recommendation list and the user's search requirements through a multi-dimensional semantic matching algorithm, adjust the heat and weight ratio of the reasoning level knowledge graph, and generate a personalized recommended book list; The feedback module is used to present a personalized recommended book list in real time according to an interactive display method, record user behavior as feedback, and dynamically update the reasoning level knowledge graph using real-time stream processing.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the big data-based book retrieval method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the big data-based book retrieval method described in any one of claims 1 to 7 are implemented.

Images