Data processing method, device, equipment and storage medium
The query data is processed through multi-dimensional feature matching rules, which solves the problem of low matching between documents and query data in the search system, and achieves higher matching accuracy and user experience.
Patent Information
- Application Number
- CN202110004255.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-01-04
- Publication Date
- 2025-08-26
- Estimated Expiration
- 2041-01-04
AI Technical Summary
The existing search system does not match the returned document and the query data, resulting in a poor search experience.
The query data is processed using multi-dimensional feature matching rules, including sequential matching rules, hit rate matching rules and tightness matching rules, and searching for target documents matching the query feature sequence from the database.
It improves the matching accuracy of query data, ensures that target documents with a high degree of matching with the query data are searched in the database, and improves the user's query experience.
Smart Images

Figure CN113392309B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, in particular to the field of intelligent search technology, and specifically to a data processing method, a data processing apparatus, a data processing device, and a computer-readable storage medium. Background Art
[0002] The essence of search systems (such as Baidu and Google) is to return relevant documents to users based on a given query (e.g., a search string, a search term, or a search statement). Currently, search systems often return documents that do not match the query, resulting in a poor search experience. Therefore, improving query matching accuracy has become a hot research topic. Summary of the Invention
[0003] The embodiments of the present application provide a data processing method, apparatus, device, and storage medium that can perform feature matching on query data from multiple dimensions, effectively improving the matching accuracy of query data.
[0004] In one aspect, an embodiment of the present application provides a data processing method, the data processing method comprising:
[0005] Obtain query data and extract a query feature sequence from the query data, where the query feature sequence includes N feature data, where N is a positive integer;
[0006] Obtaining feature matching rules, which include at least two of the following: a sequential matching rule, a hit rate matching rule, and a closeness matching rule; the sequential matching rule is used to indicate that matching is performed sequentially according to the order in which the N feature data are arranged in the query feature sequence; the hit rate matching rule is used to indicate that matching is performed according to the dimension of the quantity ratio between the hit data and the N feature data; and the closeness matching rule is used to indicate that matching is performed according to the dimension of the closeness between the hit data;
[0007] According to the feature matching rules, the target document that matches the query feature sequence is searched from the database; wherein the hit data refers to the data in the target document that matches the feature data in the query feature sequence.
[0008] On the other hand, an embodiment of the present application provides a data processing device, the data processing device comprising:
[0009] An acquisition unit, configured to acquire query data and extract a query feature sequence from the query data, wherein the query feature sequence includes N feature data, where N is a positive integer;
[0010] The acquisition unit is further used to acquire feature matching rules, which include at least two of the following: a sequential matching rule, a hit rate matching rule, and a closeness matching rule; the sequential matching rule is used to indicate that the N feature data are matched sequentially according to the order in which they are arranged in the query feature sequence; the hit rate matching rule is used to indicate that the matching is performed according to the dimension of the quantity ratio between the hit data and the N feature data; and the closeness matching rule is used to indicate that the matching is performed according to the dimension of the closeness between the hit data;
[0011] The processing unit is used to search the database for a target document that matches the query feature sequence according to feature matching rules; wherein the hit data refers to the data in the target document that matches the feature data in the query feature sequence.
[0012] In one implementation, the acquiring unit is specifically configured to:
[0013] Split the query data to obtain N feature data, where the feature data includes feature words or feature characters;
[0014] The N feature data are combined according to their arrangement order in the query data to form a query feature sequence.
[0015] In one implementation, the processing unit is specifically configured to:
[0016] Extracting a first document to be matched from a database;
[0017] Splitting the first document to obtain a data set to be matched, wherein the data set includes multiple data; the data includes words or characters;
[0018] Obtain M hit data from the data set according to the feature matching rules. Hit data refers to the data in the data set that matches the feature data in the query feature sequence, and M is a positive integer;
[0019] Calculate the matching scores between the M hit data and the N feature data in the query feature sequence according to the feature matching rules;
[0020] If the matching score is higher than the threshold, the first document is determined as a target document that matches the query feature sequence.
[0021] In one implementation, the feature matching rule includes a sequential matching rule, and the matching score includes a sequential matching score; the processing unit is specifically configured to:
[0022] According to the matching order between the M hit data and the query feature sequence, the longest common subsequence of the query feature sequence is determined;
[0023] Calculating a first ratio between the number of hit data in the longest common subsequence and the number of N feature data in the query feature sequence;
[0024] The first ratio is determined as the sequence matching score.
[0025] In one implementation, the feature matching rule includes a hit rate matching rule, and the matching score includes a hit rate matching score; the processing unit is specifically configured to:
[0026] Calculating a second ratio between the number of the M hit data and the number of the N feature data in the query feature sequence;
[0027] The second ratio is determined as a hit rate matching score.
[0028] In one implementation, the feature matching rule includes a closeness matching rule, and the matching score includes a closeness matching score; the processing unit is specifically configured to:
[0029] According to the matching order between the M hit data and the query feature sequence, the longest common subsequence of the query feature sequence is determined;
[0030] Obtaining the first hit data at the first arrangement position and the second hit data at the last arrangement position in the longest common subsequence;
[0031] Obtaining a first position of the first hit data in the first document, and a second position of the second hit data in the first document;
[0032] Calculating a length difference between the first position and the second position, and calculating a third ratio between the length difference and the length of the query feature sequence;
[0033] The third ratio is determined as a closeness match score.
[0034] In one implementation, if the feature matching rule is a combination of a sequence matching rule and a hit rate matching rule, the matching score is the product of the sequence matching score and the hit rate matching score;
[0035] If the feature matching rule is a combination of the order matching rule and the closeness matching rule, the matching score is the product of the order matching score and the closeness matching score;
[0036] If the feature matching rule is a combination of the hit rate matching rule and the closeness matching rule, the matching score is the product of the hit rate matching score and the closeness matching score;
[0037] If the feature matching rule is a combination of the sequence matching rule, the hit rate matching rule, and the closeness matching rule, the matching score is the product of the sequence matching score, the hit rate matching score, and the closeness matching score.
[0038] In one implementation, if the feature matching rule is a combination of a sequence matching rule and a hit rate matching rule, the matching score is a weighted sum of the sequence matching score and the hit rate matching score;
[0039] If the feature matching rule is a combination of the order matching rule and the closeness matching rule, the matching score is the weighted sum of the order matching score and the closeness matching score;
[0040] If the feature matching rule is a combination of the hit rate matching rule and the closeness matching rule, the matching score is the weighted sum of the hit rate matching score and the closeness matching score;
[0041] If the feature matching rule is a combination of the order matching rule, the hit rate matching rule, and the closeness matching rule, the matching score is the weighted sum of the order matching score, the hit rate matching score, and the closeness matching score.
[0042] In one implementation, the processing unit is further configured to:
[0043] According to the feature matching rules, P documents that match the query feature sequence are searched from the database. The target document is any one of the P documents, where P is a positive integer.
[0044] In one implementation, the data processing device further includes a display unit, the display unit being specifically configured to:
[0045] Display the document search page of the target application, which includes a search box, search options, and a query result display area;
[0046] When there is an input operation acting on the search box, the query data input into the search box is displayed in the search box;
[0047] When the search option is selected, P documents are displayed in the query result display area.
[0048] In one implementation, each of the P documents corresponds to a matching score, and a higher matching score indicates a higher matching degree between the document and the feature matching rule; the display unit is specifically configured to:
[0049] Arrange and display P documents in the query result display area in descending order of matching scores.
[0050] On the other hand, an embodiment of the present application provides a data processing device, the data processing device comprising:
[0051] a processor adapted to implement a computer program; and
[0052] Computer-readable storage medium, the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the above-mentioned data processing method.
[0053] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is read and executed by a processor of a computer device, the computer device executes the above-mentioned data processing method.
[0054] In another aspect, embodiments of the present application provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned data processing method.
[0055] In an embodiment of the present application, the query feature sequence extracted from the query data may include N feature data, where N is a positive integer; the feature matching rules obtained may include at least two of a sequence matching rule, a hit rate matching rule, and a closeness matching rule; the feature matching rules may perform feature matching on the query data from multiple dimensions, wherein the sequence matching rule is used to indicate that the N feature data are matched sequentially according to the order in which they are arranged in the query feature sequence, the hit rate matching rule is used to indicate that the match is performed according to the dimension of the quantity ratio between the hit data and the N feature data, and the closeness matching rule is used to indicate that the match is performed according to the dimension of the closeness between the hit data, and the hit data refers to the data in the target document that matches the feature data in the query feature sequence. Thus, a target document that matches the query feature sequence can be searched from the database according to the multi-dimensional feature matching rules. Using the multi-dimensional feature matching rules to match the query data can effectively improve the query data matching accuracy and can search the database for target documents that have a high degree of match with the query data. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0057] Figure 1 A basic flow chart of an intelligent search technology provided by an exemplary embodiment of the present application is shown;
[0058] Figure 2A schematic diagram of the architecture of a data processing system provided by an exemplary embodiment of the present application is shown;
[0059] Figure 3a A schematic diagram of a document search page provided by an exemplary embodiment of the present application is shown;
[0060] Figure 3b A schematic diagram of a document search page provided by another exemplary embodiment of the present application is shown;
[0061] Figure 3c A schematic diagram of a document search page provided by another exemplary embodiment of the present application is shown;
[0062] Figure 4 A flow chart of a data processing method provided by an exemplary embodiment of the present application is shown;
[0063] Figure 5 A flow chart of a data processing method provided by another exemplary embodiment of the present application is shown;
[0064] Figure 6 A schematic structural diagram of a data processing device provided by an exemplary embodiment of the present application is shown;
[0065] Figure 7 A structural diagram of a data processing device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION
[0066] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.
[0067] The present application embodiment proposes a data processing solution involving artificial intelligence natural language processing, machine learning and other technologies, wherein:
[0068] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.
[0069] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.
[0070] Natural language processing (NLP) is a key area of research in computer science and artificial intelligence. It studies the theories and methods that enable effective communication between humans and computers using natural language. Natural language processing (NLP) integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language we use in everyday life—and is closely linked to the study of linguistics. Natural language processing technologies typically include text processing, semantic understanding, machine translation, robotic question answering, and knowledge graphs.
[0071] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.
[0072] The embodiments of the present application relate to intelligent search technology. Intelligent search technology refers to technology that can return documents that match the query data (such as a search string, a search term, or a search statement) given by the user. The documents mentioned in the embodiments of the present application may include, but are not limited to: articles, videos, audio, games, etc. Figure 1 The following is a basic flow chart of an intelligent search technology provided by an exemplary embodiment of the present application. Figure 1 Taking the video search application as an example, for the query data "XXX" given by the user, the video search application analyzes the query data and filters out multiple videos matching "XXX" from the database through stages such as intersection recall, coarse sorting recall, and fine sorting, and returns the filtered videos to the user.
[0073] A key point in intelligent search technology lies in the construction of matching rules. The better the matching rules are constructed, the more accurate the match to the query data, and the higher the matching degree between the query data and the searched documents. An embodiment of the present application provides a data processing method that uses feature matching rules to perform multi-dimensional matching on query data, thereby improving the matching accuracy of the query data. The data processing method may specifically include: obtaining query data and extracting a query feature sequence from the query data, wherein the query feature sequence may include N feature data, where N is a positive integer; obtaining feature matching rules, wherein the feature matching rules may include at least two of a sequence matching rule, a hit rate matching rule, and a closeness matching rule; and searching a database for a target document that matches the query feature sequence according to the feature matching rules. The feature matching rules may perform feature matching on the query data from multiple dimensions. The sequence matching rule indicates that matching is performed sequentially according to the order in which the N feature data are arranged in the query feature sequence; the hit rate matching rule indicates that matching is performed according to the dimension of the quantitative ratio between the hit data and the N feature data; and the closeness matching rule indicates that matching is performed according to the dimension of the closeness between the hit data. The hit data refers to the data in the target document that matches the feature data in the query feature sequence. The query data is matched using multi-dimensional feature matching rules, which effectively improves the matching accuracy of the query data and can search the database for target documents with a high degree of match with the query data.
[0074] It should be noted that the data processing solution provided in the embodiment of the present application can be implemented using a feature matching model. The feature matching model can be obtained by training a machine learning model (for example, an artificial neural network model, a generative adversarial network model, etc.) using a sample data set. The sample data set may include multiple sample query data and multiple documents that match each sample query data. The process of extracting a query feature sequence from the query data can be implemented using natural language processing technology in artificial intelligence; the user provides query data described in natural language, and the text in the query data is analyzed and processed using natural language processing technology to obtain N feature data. The query feature sequence includes N feature data, where N is a positive integer.
[0075] In order to better understand the data processing solution provided by the embodiment of the present application, the data processing system involved in the embodiment of the present application will be introduced below in combination with actual application scenarios. Figure 2 FIG. 1 shows an architecture diagram of a data processing system provided by an exemplary embodiment of the present application. Figure 2 As shown, the data processing system 20 may include a terminal 201 and a server 202. The embodiment of the present application does not limit the number of terminals 201 and servers 202. The terminal 201 may include but is not limited to: a smart phone, a tablet computer, a PC (Personal Computer), a desktop computer, a smart speaker, a smart watch, a smart wearable device, etc.; the terminal 201 is often configured with a display device, and the display device may be a display, a display screen, a touch screen, etc., and the touch screen may be a touch screen, a touch panel, etc. The server 202 can provide computing and application service support for the terminal 201. The server 202 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services, which is not limited in the embodiment of the present application. The terminal 201 and the server 202 may be directly or indirectly connected via wired communication or wireless communication, which is not limited in the present application.
[0076] The feature matching model that can implement the data processing solution can be deployed in the terminal 201 or in the server 202. When the feature matching model is deployed in the server 202, the server 202 can provide data processing services to the terminal 201.
[0077] (1) Feature matching model deployed in server 202
[0078] The terminal can support the installation and operation of various applications, including but not limited to instant messaging applications, video service applications, audio service applications, game applications, etc. The target application (for example, a smart search application) is any application installed and running on the terminal. The target application can include multiple service pages, such as a login page, a settings page, etc. The document search page is a service page in the target application.
[0079] When a user on a terminal requests a search, the terminal may display the document search page of the target application. The document search page may include a search box, search options, and a query result display area. When an input operation is performed on the search box (e.g., a user enters query data in the search box), the query data entered in the search box is displayed in the search box. When the search option is selected (e.g., a user clicks the search option), the terminal sends the query data to the server. The server may use a feature matching model to match the query data, searching the server's database for P documents that match the query data according to the feature matching rules, and returning the P documents found to the terminal. Each of the P documents is assigned a matching score. The matching score indicates the degree of match between the document and the query data. A higher matching score indicates a higher degree of match between the document and the feature matching rules. Alternatively, a higher matching score indicates a higher degree of match between the document and the query data. The terminal may randomly display the P documents in the query result display area of the document search page. Alternatively, the terminal may arrange the P documents in descending order of matching scores in the query result display area, where P is a positive integer.
[0080] (2) Feature matching model deployed in terminal 201
[0081] The terminal can support the installation and operation of various applications, including but not limited to instant messaging applications, video service applications, audio service applications, game applications, etc. The target application (for example, a smart search application) is any application installed and running on the terminal. The target application can include multiple service pages, such as a login page, a settings page, etc. The document search page is a service page in the target application.
[0082] When a user has a search request, the terminal can display the document search page of the target application. The document search page may include a search box, search options, and a query result display area. When there is an input operation acting on the search box (for example, the user enters query data in the search box), the query data entered in the search box is displayed in the search box. When the search option is selected (for example, the user clicks the search option), the terminal can use a feature matching model to match the query data and search for P documents matching the query data from the terminal's database according to the feature matching rules; the matching score can indicate the degree of match between the document and the query data. The higher the matching score, the higher the degree of match between the document and the feature matching rules. It can also be understood that the higher the matching score, the higher the degree of match between the document and the query data. The terminal can randomly display P documents in the query result display area of the document search page. The terminal can also arrange and display P documents in the query result display area in descending order of matching scores, where P is a positive integer. In this way, the higher the degree of match between a document and the query data, the higher its ranking position in the query result display area. Users can obtain documents that closely match the query results in a timely manner, which meets the user's query needs and improves the user's query experience.
[0083] From the above content, it can be seen that no matter whether the feature matching model is deployed in the terminal 201 or in the server 202, for the user, when the user enters the query data in the search box of the document search page and clicks the search option, multiple documents matching the query data will be displayed in the query result display area of the document search page, such as multiple videos, multiple articles, etc. Figure 3a A schematic diagram of a document search page provided by an exemplary embodiment of the present application is shown in FIG. Figure 3aAs shown, document search page 30 includes a search box 301, search options 302, and document display area 303. When a user enters a search query, they enter the query "strawberry cake style" in search box 301. Search box 301 then displays the query data. When the user clicks search option 302, six videos matching the query data are displayed in query result display area 303 of document search page 30. Each video is assigned a matching score, with a higher matching score indicating a higher matching degree between the video and the query data. The six videos are then displayed in query result display area 303 in descending order of matching scores. It should be noted that, when the document is a video, the video may include a video title, and the video matching the query data may mean that the video title matches the query data; in the query result display area 303, the video 304 with the video title "Strawberry Cake Styles" is arranged before the video 305 with the video title "One Hundred Styles of Strawberry Cake", indicating that the matching score of video 304 is higher than the matching score of video 305, and the matching degree between video 304 and the query data is higher than the matching degree between video 305 and the query data.
[0084] Documents can also be articles. Figure 3b A schematic diagram of a document search page provided by another exemplary embodiment of the present application is shown. Figure 3b As shown, a user enters the query "strawberry cake style" in search box 301, and the query data entered in search box 301 is displayed in search box 301. When the user clicks search option 302, six articles matching the query data are displayed in query result display area 303 of document search page 30. Each article has a matching score, with a higher matching score indicating a higher matching degree between the article and the query data. The six articles are arranged and displayed in query result display area 303 in descending order of matching scores. It should be noted that, when the document is an article, the article may include an article title and article content, and the article matching the query data may mean that the article title matches the query data, or it may mean that the article content matches the query data; in the query result display area 303, article 306 with the title "Complete Collection of Strawberry Cake Styles" is arranged before article 307 with the title "One Hundred Styles of Strawberry Cakes", indicating that the matching score of article 306 is higher than the matching score of article 307, and the matching degree between article 306 and the query data is higher than the matching degree between article 307 and the query data.
[0085] Documents can also be a combination of articles, videos, etc. Figure 3c A schematic diagram of a document search page provided by another exemplary embodiment of the present application is shown. Figure 3cAs shown, the user enters the query data "Strawberry Cake Styles" in the search box 301, and the query data entered in the search box 301 is displayed in the search box 301. When the user clicks the search option 302, five documents matching the query data are displayed in the query result display area 303 of the document search page 30. The five documents may include videos, articles, etc. For example, the video 304 with the title "Strawberry Cake Styles" and the article 306 with the title "Strawberry Cake Styles" may be included. The document search page 30 also includes a video option 308 and an article option 309. When the video option 308 is selected, the videos are filtered out for display in the query result display area 303. When the article option 309 is selected, the articles are filtered out for display in the query result display area 303.
[0086] Since the capacity of the query result display area 303 is limited, the number of documents that can be displayed in the query result display area 303 is also limited. You can also select the continue search option 310 (e.g. Figure 3a 、 Figure 3b or Figure 3c Alternatively, a sliding operation (e.g., sliding up and down, left and right) may be performed in the query result display area 303 to view more documents matching the query data.
[0087] It should be noted that the calculation method of the matching score mentioned in the embodiments of the present application can be found in the specific description in the following embodiments and will not be repeated here.
[0088] In an embodiment of the present application, when a user enters query data in a document search page on a terminal, P documents matching the query data may be displayed on a document editing page, and the P documents are arranged and displayed in descending order according to their matching scores. Thus, documents with a higher degree of matching to the query data are arranged higher in the query result display area. This allows users to promptly access documents that closely match the query results, satisfying their query needs and improving their query experience.
[0089] It can be understood that the data processing system described in the embodiment of the present application is for the purpose of more clearly illustrating the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.
[0090] Based on the above description, see Figure 4 , Figure 4 A flow chart of a data processing method provided by an exemplary embodiment of the present application is shown. The data processing method can be performed by Figure 2 The data processing method is executed by the terminal 201 in the data processing system 20 shown in FIG. 1 , and may include the following steps S401 to S404:
[0091] S401, obtaining query data.
[0092] The query data may be input into the terminal by a user on the terminal side. The terminal displays a document editing page, which may include a search box, and the user may input the query data through the search box. That is, obtaining the query data may refer to obtaining the query data input into the search box by the user.
[0093] S402: extracting a query feature sequence from the query data.
[0094] After obtaining the query data, a query feature sequence can be extracted from the query data, and the query feature sequence includes N feature data, where N is a positive integer. The method of extracting the query feature sequence from the query data can be: splitting the query data to obtain N feature data, and combining the N feature data according to the arrangement order of the N feature data in the query data to form a query feature sequence. Among them, the feature data may include feature words or feature characters, the feature word may refer to a string composed of multiple feature characters, and the feature character may refer to a single character. The query feature sequence can be a pure feature word sequence, that is, all the N feature data in the query feature sequence are feature words; the query feature sequence can also be a pure feature character sequence, that is, all the N feature data in the query feature sequence are feature characters; the query feature sequence can also be a mixed sequence of feature words and feature characters, that is, part of the feature data in the query feature sequence is feature words, and part of the feature data is feature characters.
[0095] For example, if the query data is “strawberry cake style”, the query data can be split into three feature words, namely “strawberry”, “cake” and “style”. The three feature words are combined according to the order in which these three feature words are arranged in the query data to form a query feature sequence {strawberry, cake, style}. Alternatively, the query data can be split into six feature characters, namely “grass”, “berry”, “egg”, “cake”, “style” and “style”. The six feature characters are combined according to the order in which these six feature characters are arranged in the query data to form a query feature sequence {grass, berry, egg, cake, style, style}. Alternatively, the query data can be split into two feature words and two feature characters, the two feature words are “strawberry” and “style”, and the two feature characters are “egg” and “cake”. The two feature words and the two feature characters are combined according to the order in which they are arranged in the query data to form a query feature sequence {strawberry, egg, cake, style}.
[0096] S403: Acquire feature matching rules.
[0097] S404: Search the database for a target document that matches the query feature sequence according to feature matching rules.
[0098] In step S403, the feature matching rules may include but are not limited to at least two of the following: a sequential matching rule, a hit rate matching rule, and a closeness matching rule, which is not limited in the embodiments of the present application. Among them, the sequential matching rule can be used to indicate that the N feature data are matched in the order in which they are arranged in the query feature sequence; the hit rate matching rule can be used to indicate that the match is performed according to the quantity ratio dimension between the hit data and the N feature data; and the closeness matching rule can be used to indicate that the match is performed according to the closeness dimension between the hit data. The hit data may refer to the data in the target document that matches the feature data in the query feature sequence; the data in the target document that matches the feature data in the query feature sequence may include two situations: one is the data in the target document that is identical to the feature data in the query feature sequence. The other is the data in the target document that is similar to the feature data in the query feature sequence. When the ratio of the number of identical characters between the data in the target document and the feature data to the number of characters in the feature data exceeds the quantity threshold, it can be considered that the data in the target document is similar to the feature data, and the data can be hit data. For example, a certain data in the target document is "strawberry cake", and the feature data is "raspberry cake". The ratio of the number of identical characters in the data and the feature data to the number of characters in the feature data is 0.75, which exceeds the quantity threshold of 0.5. It can be considered that the data in the target document is similar to the feature data, and the data is hit data. The embodiment of the present application is illustrated by taking the first case as an example, that is, the embodiment of the present application is illustrated by taking the hit data being the data in the target document that is identical to the feature data in the query feature sequence as an example. For example, the target document is "I want to make strawberry cake today, do you have a tutorial for making it?", and the query feature sequence is {strawberry, cake, style}, then there are 2 hit data, one is "strawberry" and the other is "cake".
[0099] In step S404, the method of searching for a target document that matches the query feature sequence from the database according to the feature matching rules may be: the database may include multiple documents to be matched, and a first document to be matched is extracted from the database; the first document is split to obtain a data set to be matched, and the data set may include multiple data, and the data may include words or characters; M hit data are obtained from the data set according to the feature matching rules, and the hit data refers to the data in the data set that matches the feature data in the query feature sequence, and M is a positive integer; the matching score between the M hit data and the N feature data in the query feature sequence is calculated according to the feature matching rules; if the matching score is higher than a threshold, the first document is determined as the target document that matches the query feature sequence. Among them, the matching score may include but is not limited to at least two of the following: sequential matching score, hit rate matching score, and closeness matching score, which are not limited in the present embodiment; according to the sequential matching rule, the sequential matching score between M hit data and N feature data in the query feature sequence can be calculated; according to the hit rate matching rule, the hit rate matching score between M hit data and N feature data in the query feature sequence can be calculated; according to the closeness matching rule, the closeness matching score between M hit data and N feature data in the query feature sequence can be calculated. The specific calculation process of the sequential matching score, hit rate matching score, and closeness matching score can be found in Figure 5 Detailed description of step S507 in the illustrated embodiment.
[0100] In an embodiment of the present application, the matching score can be used as a standard to measure whether a document matches the query data. If the matching score of a document exceeds a threshold, the document is considered to match the query data. If the matching score of a document does not exceed the threshold, the document is considered to not match the query data. The matching score of a document can include a combination of at least two of the order matching score, the hit rate matching score, and the closeness matching score. The order matching score between the document and the query data can be calculated according to the order matching rule; the hit rate matching score between the document and the query data can be calculated according to the hit rate matching rule; and the closeness matching score between the document and the query data can be calculated according to the closeness matching rule. The order matching score, the hit rate matching score, and the closeness matching score represent the matching relationship between the document and the query data from different dimensions. In the matching process between the document and the query data, the degree of matching between the hit data in the document and the feature data in the query data in the three dimensions of data arrangement order, quantity ratio, and closeness is comprehensively considered, which can effectively improve the matching accuracy of the query data.
[0101] See Figure 5 , Figure 5A flow chart of a data processing method provided by another exemplary embodiment of the present application is shown. The data processing method can be performed by Figure 2 The data processing method is executed by the terminal 201 in the data processing system 20 shown in FIG. 1 , and may include the following steps S501 to S508:
[0102] S501, obtaining query data.
[0103] S502: Extract a query feature sequence from the query data.
[0104] S503: Acquire feature matching rules.
[0105] The execution process of step S501 in the embodiment of the present application is the same as Figure 4 The execution process of step S401 in the embodiment shown is the same as that of step S502. Figure 4 The execution process of step S402 in the embodiment shown is the same as that of step S503. Figure 4 The execution process of step S403 in the embodiment shown is the same. For the specific execution process, please refer to Figure 4 The description of the illustrated embodiment will not be repeated here for the embodiments of the present application.
[0106] S504: extract a first document to be matched from the database.
[0107] The database may include multiple documents to be matched, and a first document to be matched is extracted from the database. The first document is any document in the database.
[0108] S505: Split the first document to obtain a data set to be matched.
[0109] The first document may include a document title and document content. Splitting the first document may refer to splitting the document title of the first document or splitting the document content of the first document. When the first document is a video, the document title may refer to the video title, and the document content may refer to the video data; when the first document is an article, the document title may refer to the article title, and the document content may refer to the document content.
[0110] The data set to be matched may include multiple data to be matched; the method for splitting the first document to obtain the data set to be matched may be: splitting the first document to obtain multiple data to be matched, and combining the multiple data to be matched according to the arrangement order of the multiple data to be matched in the query data to form the data set to be matched. The data may include words or characters, a word in the data may refer to a string of multiple characters, and a character in the data may refer to a single character. The data set to be matched may be a word set, that is, the multiple data in the data set to be matched are all words; the data set to be matched may also be a character set, that is, the multiple data in the data set to be matched are all characters; the data set to be matched may also be a mixed set of words and characters, that is, some of the data in the data set to be matched are words, and some of the data are characters.
[0111] S506: Obtain M hit data from the data set according to the feature matching rule.
[0112] Hit data may refer to data in the data set to be matched that matches the feature data in the query feature sequence. M hit data may be obtained from the data set according to the hit rate matching rule, where M is a positive integer.
[0113] For example, if the first document is "One hundred styles of cake," the dataset obtained by splitting the first document is {cake, of, one hundred styles, style}. This split dataset can also be optimized by removing meaningless particles such as "of," resulting in an optimized dataset of {cake, one hundred styles, style}. The query data is "strawberry cake style," and the query feature sequence obtained by splitting the query data is {strawberry, cake, style}. According to the hit rate matching rule, two hit data items can be obtained from the dataset: "cake" and "style."
[0114] S507: Calculate the matching scores between the M hit data and the N feature data in the query feature sequence according to the feature matching rules.
[0115] Feature matching rules may include but are not limited to at least two of the following: sequential matching rules, hit rate matching rules, and closeness matching rules, which are not limited in the embodiments of the present application; matching scores may include but are not limited to at least two of the following: sequential matching scores, hit rate matching scores, and closeness matching scores, which are not limited in the embodiments of the present application. According to the sequential matching rules, the sequential matching scores between the M hit data and the N feature data in the query feature sequence can be calculated; according to the hit rate matching rules, the hit rate matching scores between the M hit data and the N feature data in the query feature sequence can be calculated; according to the closeness matching rules, the closeness matching scores between the M hit data and the N feature data in the query feature sequence can be calculated. Among them:
[0116] (1) Calculate the sequential matching score between the M hit data and the N feature data in the query feature sequence according to the sequential matching rule. Specifically, the longest common subsequence of the query feature sequence can be determined according to the matching order between the M hit data and the query feature sequence; calculate a first ratio between the number of hit data in the longest common subsequence and the number of N feature data in the query feature sequence, and determine the first ratio as the sequential matching score. The sequential matching score is calculated as shown in the following formula 1:
[0117] Score_sequence=count(LSC) / count(Query) Formula 1
[0118] As shown in the above formula 1, Score_sequence represents the sequential matching score, LSC represents the longest common subsequence of the query feature sequence, count(LSC) represents the number of hit data in the longest common subsequence, Query represents the query feature sequence, and count(Query) represents the number of N feature data in the query feature sequence.
[0119] The longest common subsequence (LCS) is a classic dynamic programming problem. Given Q sequences, the longest common subsequence can be the common subsequence containing the largest number of data among all the common subsequences of the Q sequences, where Q is an integer greater than or equal to 2. The common subsequence of the Q sequences can be the sequence containing the same data obtained by deleting a certain number of data (for example, 0, 1, 2, or more) from each of the Q sequences without changing the order of the remaining data in the Q sequences. For example, given two sequences, sequence X and sequence Y, sequence X = {1, a, 2, b, 3}, sequence Y = {h, f, a, h, b, r, c}, the common subsequences of sequence X and sequence Y are common subsequence Z1 = {a, b}, common subsequence Z2 = {a}, and common subsequence Z3 = {b}, the common subsequence containing the largest number of data among common subsequences Z1, common subsequence Z2, and common subsequence Z3 is common subsequence Z1, then the longest common subsequence of sequence X and sequence Y is common subsequence Z1.
[0120] For example, if the two hit data are "cake" and "style", the longest common subsequence between the two hit data and the three feature data {strawberry, cake, style} in the query feature sequence is {cake, style}, the number of hit data in the longest common subsequence is 2, the number of feature data in the query feature sequence is 3, the first ratio between the number of hit data in the longest common subsequence and the number of feature data in the query feature sequence is 2 / 3, and the sequential matching score is 2 / 3. If the two hit data are "style" and "cake", the longest common subsequence between the two hit data and the three feature data {strawberry, cake, style} in the query feature sequence is {style} or {cake}, the number of hit data in the longest common subsequence is 1, the number of feature data in the query feature sequence is 3, the first ratio between the number of hit data in the longest common subsequence and the number of feature data in the query feature sequence is 1 / 3, and the sequential matching score is 1 / 3.
[0121] As can be seen from the above, the order of the hits is taken into account when matching the M hits with the query feature sequence using the sequential matching rule. If the M hits match the feature data in the query feature sequence in order, the calculated sequential matching score is high. If the M hits do not match the feature data in the query feature sequence in order, or only partially match them in order, the calculated sequential matching score is low. Using the sequential matching rule improves the matching accuracy of the query data by taking into account the matching order.
[0122] (2) Calculate the hit rate matching score between the M hit data and the N feature data in the query feature sequence according to the hit rate matching rule. Specifically, a second ratio between the number of M hit data and the number of N feature data in the query feature sequence can be calculated, and the second ratio can be determined as the hit rate matching score. The hit rate matching score is calculated as shown in the following formula 2:
[0123] Score_hit=count(Hit) / count(Query) Formula 2
[0124] As shown in the above formula 2, Score_hit represents the hit rate matching score, Hit represents the hit data, count(Hit) represents the number of hit data, Query represents the query feature sequence, and count(Query) represents the number of N feature data in the query feature sequence.
[0125] For example, if the two hit data are "cake" and "style", and the query feature sequence is {strawberry, cake, style}, then the number of hit data is 2, the number of feature data in the query feature sequence is 3, the second ratio between the number of hit data and the number of feature data in the query feature sequence is 2 / 3, and the hit rate matching score is 2 / 3.
[0126] As can be seen from the above, the number of hits is taken into account when matching M hits with the query feature sequence using the hit rate matching rule. A high number of hits results in a high hit rate matching score; a low number of hits results in a low hit rate matching score. Using the hit rate matching rule improves the matching accuracy of query data by taking into account the hit rate.
[0127] (3) Calculate the closeness matching score between the M hit data and the N feature data in the query feature sequence according to the closeness matching rule. Specifically, according to the matching order between the M hit data and the query feature sequence, determine the longest common subsequence of the query feature sequence; obtain the first hit data located at the first arrangement position and the second hit data located at the last arrangement position in the longest common subsequence; obtain the first position of the first hit data in the first document, and the second position of the second hit data in the first document; calculate the length difference between the first position and the second position, and calculate the third ratio between the length difference and the length of the query feature sequence, and determine the third ratio as the closeness matching score; wherein, the length of the query feature sequence may refer to the sum of the lengths of the N feature data included in the query feature sequence, and the length of the feature data may be equal to the number of characters contained in the feature data, for example, if the feature data is "strawberry", the length of the feature data is 2. The calculation method of the closeness matching score is shown in the following formula 3:
[0128] Score_span=(pos_end-pos_first) / len(Query) Formula 3
[0129] As shown in Formula 3 above, Score_span represents the closeness matching score, pos_end represents the second position of the second hit data at the last arranged position in the longest common subsequence in the first document, pos_first represents the first position of the first hit data at the first arranged position in the longest common subsequence in the first document, (pos_end-pos_first) represents the length difference between the first position and the second position, Query represents the query feature sequence, and len(Query) represents the length of the query feature sequence.
[0130] For example, the first document is "One Hundred Styles of Cake", and the query feature sequence is {strawberry, cake, style}. According to the hit rate matching rule, 2 hit data can be obtained, namely "cake" and "style"; the longest common subsequence between the 2 hit data and the 3 feature data in the query feature sequence is {cake, style}; the first hit data located at the first arrangement position in the longest common subsequence is "cake", and the second hit data located at the last arrangement position in the longest common subsequence is "style". The length difference between the first position of the first hit data in the first document and the second position of the second hit data in the first document is 4, the length of the query feature sequence is 6, and the third ratio between the length difference and the length of the query feature sequence is 4 / 6, so the closeness matching score is 4 / 6.
[0131] As can be seen from the above, in the process of matching M hit data with the query feature sequence using the closeness matching rule, the closeness between the hit data is taken into account. If the span between the first hit data and the second hit data in the first document is smaller, it indicates that the first hit data and the second hit data are closer, and the matching degree between the query data and the first document is higher; if the span between the first hit data and the second hit data in the first document is larger, it indicates that the first hit data and the second hit data are sparser, and the matching degree between the query data and the first document is lower. The closeness matching rule is used to improve the matching accuracy of the query data from the closeness dimension between the hit data.
[0132] Feature matching rules may include but are not limited to at least two of the following: sequential matching rules, hit rate matching rules, and closeness matching rules, which are not limited in this embodiment of the present application; matching scores may include but are not limited to at least two of the following: sequential matching scores, hit rate matching scores, and closeness matching scores, which are not limited in this embodiment of the present application. The specific calculation methods of matching scores may include the following situations:
[0133] (1) If the feature matching rule is a combination of the sequential matching rule and the hit rate matching rule, the matching score may be the product of the sequential matching score and the hit rate matching score, or the matching score may be the weighted sum of the sequential matching score and the hit rate matching score.
[0134] (2) If the feature matching rule is a combination of a sequential matching rule and a closeness matching rule, the matching score may be the product of the sequential matching score and the inverse of the closeness matching score, or the matching score may be the weighted sum of the sequential matching score and the inverse of the closeness matching score.
[0135] (3) If the feature matching rule is a combination of a hit rate matching rule and a closeness matching rule, the matching score may be the product of the hit rate matching score and the inverse of the closeness matching score, or the matching score may be the weighted sum of the hit rate matching score and the inverse of the closeness matching score.
[0136] (4) If the feature matching rule is a combination of the sequential matching rule, the hit rate matching rule, and the closeness matching rule, the matching score can be the product of the sequential matching score, the hit rate matching score, and the inverse of the closeness matching score, or the matching score can be the weighted sum of the sequential matching score, the hit rate matching score, and the inverse of the closeness matching score.
[0137] For example, the matching score is the product of the order matching score, the hit rate matching score, and the reciprocal of the closeness matching score. The matching score is calculated as shown in the following formula 4:
[0138] Score=Score_sequence×Score_hit×(1 / Score_span) Formula 4
[0139] As shown in the above formula 4, Score represents the matching score, Score_sequence represents the sequence matching score, Score_hit represents the hit rate matching score, and (1 / Score_span) represents the inverse of the closeness score.
[0140] For example, the matching score is the weighted sum of the order matching score, the hit rate matching score, and the reciprocal of the closeness matching score. The matching score is calculated as shown in the following formula 5:
[0141] Score=a×Score_sequence+b×Score_hit+c×(1 / Score_span) Formula 5
[0142] As shown in Formula 5 above, Score represents the matching score; a represents the weight of the sequence matching score, Score_sequence represents the sequence matching score; b represents the weight of the hit rate matching score, Score_hit represents the hit rate matching score; c represents the weight of the closeness matching score, and (1 / Score_span) represents the inverse of the closeness matching score. a, b, and c can be the same or different. Different weights can be used to adjust the feature matching rules. If the weight of the sequence matching score is relatively large, it indicates that the matching process pays more attention to the matching order between the hit data and the query feature sequence; if the weight of the hit rate matching score is relatively large, it indicates that the matching process pays more attention to the ratio of the number of feature data in the hit data and the query feature sequence; if the weight of the closeness matching score is relatively large, it indicates that the matching process pays more attention to the closeness of the hit data in the document, further improving the matching accuracy of the query data.
[0143] For example, a calculation method for the matching score between the first document “One Hundred Styles of Cake” and the query feature sequence {strawberry, cake, style} can be seen in Table 1.
[0144] Table 1
[0145]
[0146] S508: If the matching score is higher than the threshold, the first document is determined as a target document that matches the query feature sequence.
[0147] The target application can be run in the terminal, and the query data can be input by the user through the document search page of the target application. The document search page of the target application is displayed in the terminal, and the document search page includes a search box, search options and a query result display area; when there is an input operation acting on the search box, the query data input in the search box is displayed in the search box; when the search option is selected, P documents matching the query feature sequence can be searched from the database according to the feature matching rules, and P documents can be displayed in the query result display area. The target document is any one of the P documents, and P is a positive integer. Each of the P documents can correspond to a matching score. The higher the matching score, the higher the matching degree of the document and the feature matching rule, that is, the higher the matching degree of the document and the query feature sequence; the P documents can be arranged and displayed in the query result display area in descending order of the matching scores. The various display methods of the document search page can be found above. Figures 3a to 3c The description is not repeated here.
[0148] In an embodiment of the present application, the matching score of the first document may include a combination of at least two of a sequence matching score, a hit rate matching score, and a closeness matching score. According to the sequence matching rule, the sequence matching score between the M hit data of the first document and the N feature data in the query feature sequence can be calculated; according to the hit rate matching rule, the hit rate matching score between the M hit data of the first document and the N feature data in the query feature sequence can be calculated; according to the closeness matching rule, the closeness matching score between the M hit data of the first document and the N feature data in the query feature sequence can be calculated. The matching score can be the product of at least two of the sequence matching score, the hit rate matching score, and the inverse of the closeness matching score, or the weighted sum of at least two of the sequence matching score, the hit rate matching score, and the inverse of the closeness matching score. In the process of weighted fusion of the sequence matching score, the hit rate matching score, and the inverse of the closeness matching score, different weights can be set. By using different weights, the attention of the matching rules of different dimensions in the feature matching rules can be adjusted, further improving the matching accuracy of the query data.
[0149] See Figure 6 , Figure 6 A schematic diagram of the structure of a data processing device provided by another exemplary embodiment of the present application is shown. The data processing device 60 can be set in a data processing device, which can be Figure 2 The data processing system shown in FIG201 is a terminal. The data processing device 60 can be used to perform Figure 4 or Figure 5 For the corresponding steps in the data processing method shown, the data processing device 60 may include the following units:
[0150] An acquisition unit 601 is configured to acquire query data and extract a query feature sequence from the query data, wherein the query feature sequence includes N feature data, where N is a positive integer;
[0151] The acquisition unit 601 is further configured to acquire a feature matching rule, which includes at least two of the following: a sequential matching rule, a hit rate matching rule, and a closeness matching rule; the sequential matching rule is used to indicate that matching is performed sequentially according to the order in which the N feature data are arranged in the query feature sequence; the hit rate matching rule is used to indicate that matching is performed according to the dimension of the quantity ratio between the hit data and the N feature data; and the closeness matching rule is used to indicate that matching is performed according to the dimension of the closeness between the hit data.
[0152] The processing unit 602 is configured to search the database for a target document that matches the query feature sequence according to feature matching rules; wherein the hit data refers to data in the target document that matches the feature data in the query feature sequence.
[0153] In one implementation, the acquiring unit 601 is specifically configured to:
[0154] Split the query data to obtain N feature data, where the feature data includes feature words or feature characters;
[0155] The N feature data are combined according to their arrangement order in the query data to form a query feature sequence.
[0156] In one implementation, the processing unit 602 is specifically configured to:
[0157] Extracting a first document to be matched from a database;
[0158] Splitting the first document to obtain a data set to be matched, wherein the data set includes multiple data; the data includes words or characters;
[0159] Obtain M hit data from the data set according to the feature matching rules. Hit data refers to the data in the data set that matches the feature data in the query feature sequence, and M is a positive integer;
[0160] Calculate the matching scores between the M hit data and the N feature data in the query feature sequence according to the feature matching rules;
[0161] If the matching score is higher than the threshold, the first document is determined as a target document that matches the query feature sequence.
[0162] In one implementation, the feature matching rule includes a sequential matching rule, and the matching score includes a sequential matching score; the processing unit 602 is specifically configured to:
[0163] According to the matching order between the M hit data and the query feature sequence, the longest common subsequence of the query feature sequence is determined;
[0164] Calculating a first ratio between the number of hit data in the longest common subsequence and the number of N feature data in the query feature sequence;
[0165] The first ratio is determined as the sequence matching score.
[0166] In one implementation, the feature matching rule includes a hit rate matching rule, and the matching score includes a hit rate matching score; the processing unit 602 is specifically configured to:
[0167] Calculating a second ratio between the number of the M hit data and the number of the N feature data in the query feature sequence;
[0168] The second ratio is determined as a hit rate matching score.
[0169] In one implementation, the feature matching rule includes a closeness matching rule, and the matching score includes a closeness matching score; the processing unit 602 is specifically configured to:
[0170] According to the matching order between the M hit data and the query feature sequence, the longest common subsequence of the query feature sequence is determined;
[0171] Obtaining the first hit data at the first arrangement position and the second hit data at the last arrangement position in the longest common subsequence;
[0172] Obtaining a first position of the first hit data in the first document, and a second position of the second hit data in the first document;
[0173] Calculating a length difference between the first position and the second position, and calculating a third ratio between the length difference and the length of the query feature sequence;
[0174] The third ratio is determined as a closeness match score.
[0175] In one implementation, if the feature matching rule is a combination of a sequence matching rule and a hit rate matching rule, the matching score is the product of the sequence matching score and the hit rate matching score;
[0176] If the feature matching rule is a combination of the order matching rule and the closeness matching rule, the matching score is the product of the order matching score and the closeness matching score;
[0177] If the feature matching rule is a combination of the hit rate matching rule and the closeness matching rule, the matching score is the product of the hit rate matching score and the closeness matching score;
[0178] If the feature matching rule is a combination of the sequence matching rule, the hit rate matching rule, and the closeness matching rule, the matching score is the product of the sequence matching score, the hit rate matching score, and the closeness matching score.
[0179] In one implementation, if the feature matching rule is a combination of a sequence matching rule and a hit rate matching rule, the matching score is a weighted sum of the sequence matching score and the hit rate matching score;
[0180] If the feature matching rule is a combination of the order matching rule and the closeness matching rule, the matching score is the weighted sum of the order matching score and the closeness matching score;
[0181] If the feature matching rule is a combination of the hit rate matching rule and the closeness matching rule, the matching score is the weighted sum of the hit rate matching score and the closeness matching score;
[0182] If the feature matching rule is a combination of the order matching rule, the hit rate matching rule, and the closeness matching rule, the matching score is the weighted sum of the order matching score, the hit rate matching score, and the closeness matching score.
[0183] In one implementation, the processing unit 602 is further configured to:
[0184] According to the feature matching rules, P documents that match the query feature sequence are searched from the database. The target document is any one of the P documents, where P is a positive integer.
[0185] In one implementation, the data processing device further includes a display unit 603, and the display unit 603 is specifically configured to:
[0186] Display the document search page of the target application, which includes a search box, search options, and a query result display area;
[0187] When there is an input operation acting on the search box, the query data input into the search box is displayed in the search box;
[0188] When the search option is selected, P documents are displayed in the query result display area.
[0189] In one implementation, each of the P documents corresponds to a matching score, and a higher matching score indicates a higher matching degree between the document and the feature matching rule; the display unit 603 is specifically configured to:
[0190] Arrange and display P documents in the query result display area in descending order of matching scores.
[0191] According to one embodiment of the present application, Figure 6 The various units in the data processing device 60 shown can be individually or all combined into one or several other units to form a whole, or one (or some) of the units can be further divided into multiple functionally smaller units to form a whole, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the function of one unit can also be realized by multiple units, or the functions of multiple units can be realized by one unit. In other embodiments of the present application, the data processing device 60 may also include other units. In actual applications, these functions can also be implemented with the assistance of other units and can be implemented by the collaboration of multiple units. According to another embodiment of the present application, it is possible to execute the following operations by running on a general-purpose computing device of a general-purpose computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM). Figure 4 or Figure 5A computer program (including program code) for each step involved in the corresponding method shown in Figure 6 The data processing device 60 shown in the figure can be used to implement the data processing method of the embodiment of the present application. The computer program can be recorded on a computer-readable storage medium, for example, and loaded into the above-mentioned computing device through the computer-readable storage medium and run therein.
[0192] In an embodiment of the present application, the query feature sequence extracted from the query data may include N feature data, where N is a positive integer; the feature matching rules obtained may include at least two of a sequence matching rule, a hit rate matching rule, and a closeness matching rule; the feature matching rules may perform feature matching on the query data from multiple dimensions, wherein the sequence matching rule is used to indicate that the N feature data are matched sequentially according to the order in which they are arranged in the query feature sequence, the hit rate matching rule is used to indicate that the match is performed according to the dimension of the quantity ratio between the hit data and the N feature data, and the closeness matching rule is used to indicate that the match is performed according to the dimension of the closeness between the hit data, and the hit data refers to the data in the target document that matches the feature data in the query feature sequence. Thus, a target document that matches the query feature sequence can be searched from the database according to the multi-dimensional feature matching rules. Using the multi-dimensional feature matching rules to match the query data can effectively improve the query data matching accuracy and can search the database for target documents that have a high degree of match with the query data.
[0193] See Figure 7 , Figure 7 A structural schematic diagram of a data processing device provided by an exemplary embodiment of the present application is shown, and the data processing device 70 includes at least a processor 701, a computer-readable storage medium 702 and a communication interface 703. The processor 701, the computer-readable storage medium 702 and the communication interface 703 can be connected via a bus or other means. The communication interface 703 can be used to receive or send data. The computer-readable storage medium 702 can be stored in a memory, and the computer-readable storage medium 702 is used to store computer programs, and the computer programs include computer instructions. The processor 701 is used to execute computer instructions. The processor 701 (or CPU (Central Processing Unit)) is the computing core and control core of the data processing device 70, which is suitable for implementing one or more computer instructions, and is specifically suitable for loading and executing one or more computer instructions to implement the corresponding method flow or corresponding function.
[0194] The embodiment of the present application also provides a computer-readable storage medium (Memory), which is a memory device in the data processing device 70 for storing programs and data. It is understandable that the computer-readable storage medium 702 here can include both the built-in storage medium in the data processing device 70 and, of course, the extended storage medium supported by the data processing device 70. The computer-readable storage medium provides a storage space that stores the operating system of the data processing device 70. In addition, one or more computer instructions suitable for being loaded and executed by the processor 701 are also stored in the storage space. These computer instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium 702 here can be a high-speed RAM memory or a non-volatile memory (Non-Volatile Memory), such as at least one disk storage; optionally, it can also be at least one computer-readable storage medium located away from the aforementioned processor 701.
[0195] The data processing device 70 may be Figure 2 The terminal 201 in the data processing system 20 shown in FIG. 1 stores a computer program in the computer-readable storage medium 702. The computer program includes one or more computer instructions. The processor 701 loads and executes the one or more computer instructions to implement Figure 4 or Figure 5 Corresponding steps in the method embodiment shown; in a specific implementation, the computer instructions in the computer-readable storage medium 702 are loaded by the processor 701 and execute the following steps:
[0196] Obtain query data and extract a query feature sequence from the query data, where the query feature sequence includes N feature data, where N is a positive integer;
[0197] Obtaining feature matching rules, which include at least two of the following: a sequential matching rule, a hit rate matching rule, and a closeness matching rule; the sequential matching rule is used to indicate that matching is performed sequentially according to the order in which the N feature data are arranged in the query feature sequence; the hit rate matching rule is used to indicate that matching is performed according to the dimension of the quantity ratio between the hit data and the N feature data; and the closeness matching rule is used to indicate that matching is performed according to the dimension of the closeness between the hit data;
[0198] According to the feature matching rules, the target document that matches the query feature sequence is searched from the database; wherein the hit data refers to the data in the target document that matches the feature data in the query feature sequence.
[0199] In one implementation, when the computer instructions in the computer-readable storage medium 702 are loaded by the processor 701, the following steps are specifically performed:
[0200] Split the query data to obtain N feature data, where the feature data includes feature words or feature characters;
[0201] The N feature data are combined according to their arrangement order in the query data to form a query feature sequence.
[0202] In one implementation, when the computer instructions in the computer-readable storage medium 702 are loaded by the processor 701, the following steps are specifically performed:
[0203] Extracting a first document to be matched from a database;
[0204] Splitting the first document to obtain a data set to be matched, wherein the data set includes multiple data; the data includes words or characters;
[0205] Obtain M hit data from the data set according to the feature matching rules. Hit data refers to the data in the data set that matches the feature data in the query feature sequence, and M is a positive integer;
[0206] Calculate the matching scores between the M hit data and the N feature data in the query feature sequence according to the feature matching rules;
[0207] If the matching score is higher than the threshold, the first document is determined as a target document that matches the query feature sequence.
[0208] In one implementation, the feature matching rule includes a sequential matching rule, and the matching score includes a sequential matching score. When the computer instructions in the computer-readable storage medium 702 are loaded by the processor 701, the following steps are specifically executed:
[0209] According to the matching order between the M hit data and the query feature sequence, the longest common subsequence of the query feature sequence is determined;
[0210] Calculating a first ratio between the number of hit data in the longest common subsequence and the number of N feature data in the query feature sequence;
[0211] The first ratio is determined as the sequence matching score.
[0212] In one implementation, the feature matching rule includes a hit rate matching rule, and the matching score includes a hit rate matching score. When the computer instructions in the computer-readable storage medium 702 are loaded by the processor 701, the following steps are specifically executed:
[0213] Calculating a second ratio between the number of the M hit data and the number of the N feature data in the query feature sequence;
[0214] The second ratio is determined as a hit rate matching score.
[0215] In one implementation, the feature matching rule includes a closeness matching rule, and the matching score includes a closeness matching score; when the computer instructions in the computer-readable storage medium 702 are loaded by the processor 701, the following steps are specifically executed:
[0216] According to the matching order between the M hit data and the query feature sequence, the longest common subsequence of the query feature sequence is determined;
[0217] Obtaining the first hit data at the first arrangement position and the second hit data at the last arrangement position in the longest common subsequence;
[0218] Obtaining a first position of the first hit data in the first document, and a second position of the second hit data in the first document;
[0219] Calculating a length difference between the first position and the second position, and calculating a third ratio between the length difference and the length of the query feature sequence;
[0220] The third ratio is determined as a closeness match score.
[0221] In one implementation, if the feature matching rule is a combination of a sequence matching rule and a hit rate matching rule, the matching score is the product of the sequence matching score and the hit rate matching score;
[0222] If the feature matching rule is a combination of the order matching rule and the closeness matching rule, the matching score is the product of the order matching score and the closeness matching score;
[0223] If the feature matching rule is a combination of the hit rate matching rule and the closeness matching rule, the matching score is the product of the hit rate matching score and the closeness matching score;
[0224] If the feature matching rule is a combination of the sequence matching rule, the hit rate matching rule, and the closeness matching rule, the matching score is the product of the sequence matching score, the hit rate matching score, and the closeness matching score.
[0225] In one implementation, if the feature matching rule is a combination of a sequence matching rule and a hit rate matching rule, the matching score is a weighted sum of the sequence matching score and the hit rate matching score;
[0226] If the feature matching rule is a combination of the order matching rule and the closeness matching rule, the matching score is the weighted sum of the order matching score and the closeness matching score;
[0227] If the feature matching rule is a combination of the hit rate matching rule and the closeness matching rule, the matching score is the weighted sum of the hit rate matching score and the closeness matching score;
[0228] If the feature matching rule is a combination of the order matching rule, the hit rate matching rule, and the closeness matching rule, the matching score is the weighted sum of the order matching score, the hit rate matching score, and the closeness matching score.
[0229] In one implementation, the computer instructions in the computer-readable storage medium 702, when loaded by the processor 701, further perform the following steps:
[0230] According to the feature matching rules, P documents that match the query feature sequence are searched from the database. The target document is any one of the P documents, where P is a positive integer.
[0231] In one implementation, when the computer instructions in the computer-readable storage medium 702 are loaded by the processor 701, the following steps are specifically performed:
[0232] Display the document search page of the target application, which includes a search box, search options, and a query result display area;
[0233] When there is an input operation acting on the search box, the query data input into the search box is displayed in the search box;
[0234] When the search option is selected, P documents are displayed in the query result display area.
[0235] In one implementation, each of the P documents corresponds to a matching score, where a higher matching score indicates a higher matching degree between the document and the feature matching rule. When the computer instructions in the computer-readable storage medium 702 are loaded by the processor 701, the following steps are specifically executed:
[0236] Arrange and display P documents in the query result display area in descending order of matching scores.
[0237] In an embodiment of the present application, the query feature sequence extracted from the query data may include N feature data, where N is a positive integer; the feature matching rules obtained may include at least two of a sequence matching rule, a hit rate matching rule, and a closeness matching rule; the feature matching rules may perform feature matching on the query data from multiple dimensions, wherein the sequence matching rule is used to indicate that the N feature data are matched sequentially according to the order in which they are arranged in the query feature sequence, the hit rate matching rule is used to indicate that the match is performed according to the dimension of the quantity ratio between the hit data and the N feature data, and the closeness matching rule is used to indicate that the match is performed according to the dimension of the closeness between the hit data, and the hit data refers to the data in the target document that matches the feature data in the query feature sequence. Thus, a target document that matches the query feature sequence can be searched from the database according to the multi-dimensional feature matching rules. Using the multi-dimensional feature matching rules to match the query data can effectively improve the query data matching accuracy and can search the database for target documents that have a high degree of match with the query data.
[0238] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the data processing methods provided in the various optional embodiments described above.
[0239] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.
Claims
1. A data processing method, characterized in that: The method comprises: Acquire query data, and extract a query feature sequence from the query data, wherein the query feature sequence includes N feature data, where N is a positive integer; Acquire feature matching rules, where the feature matching rules include at least two of the following: a sequence matching rule, a hit rate matching rule, and a closeness matching rule; Calculating a matching score between the M hit data and the N feature data in the query feature sequence according to the feature matching rule, and determining the first document as a target document that matches the query feature sequence if the matching score is higher than a threshold; the M hit data refers to data in the first document in the database that matches the feature data in the query feature sequence; The matching score includes the closeness matching score corresponding to the closeness matching rule, and the process of determining the closeness matching score includes: determining the longest common subsequence of the query feature sequence according to the matching order between the M hit data and the query feature sequence; obtaining the first hit data located at the first arrangement position and the second hit data located at the last arrangement position in the longest common subsequence; obtaining the first position of the first hit data in the first document and the second position of the second hit data in the first document; calculating the length difference between the first position and the second position, and determining the third ratio between the length difference and the length of the query feature sequence as the closeness matching score; a common subsequence refers to a sequence containing the same data obtained by deleting a certain number of data from the sequence composed of the M hit data and the query feature sequence without changing the arrangement order of the remaining data in the sequence, and the longest common subsequence is the common subsequence containing the largest number of data.
2. The method according to claim 1, wherein The extracting the query feature sequence from the query data includes: Splitting the query data to obtain the N feature data, where the feature data includes feature words or feature characters; The N feature data are combined according to the arrangement order of the N feature data in the query data to form the query feature sequence.
3. The method according to claim 1, wherein The feature matching rule includes the sequential matching rule, and the matching score includes a sequential matching score; The calculating, according to the feature matching rule, the matching scores between the M hit data and the N feature data in the query feature sequence includes: Determine the longest common subsequence of the query feature sequence according to the matching order between the M hit data and the query feature sequence; Calculating a first ratio between the number of hit data in the longest common subsequence and the number of N feature data in the query feature sequence; The first ratio is determined as the sequence matching score.
4. The method according to claim 1, wherein The feature matching rule includes the hit rate matching rule, and the matching score includes the hit rate matching score; The calculating, according to the feature matching rule, the matching scores between the M hit data and the N feature data in the query feature sequence includes: Calculating a second ratio between the number of the M hit data and the number of the N feature data in the query feature sequence; The second ratio is determined as the hit rate matching score.
5. The method according to claim 1, wherein: If the feature matching rule is a combination of the sequence matching rule and the hit rate matching rule, the matching score is the product of the sequence matching score and the hit rate matching score; If the feature matching rule is a combination of the sequence matching rule and the closeness matching rule, the matching score is the product of the sequence matching score and the closeness matching score; If the feature matching rule is a combination of the hit rate matching rule and the closeness matching rule, the matching score is the product of the hit rate matching score and the closeness matching score; If the feature matching rule is a combination of the sequence matching rule, the hit rate matching rule, and the closeness matching rule, the matching score is the product of the sequence matching score, the hit rate matching score, and the closeness matching score.
6. The method according to claim 1, wherein: If the feature matching rule is a combination of the sequence matching rule and the hit rate matching rule, the matching score is a weighted sum of the sequence matching score and the hit rate matching score; If the feature matching rule is a combination of the sequence matching rule and the closeness matching rule, the matching score is a weighted sum of the sequence matching score and the closeness matching score; If the feature matching rule is a combination of the hit rate matching rule and the closeness matching rule, the matching score is a weighted sum of the hit rate matching score and the closeness matching score; If the feature matching rule is a combination of the sequence matching rule, the hit rate matching rule and the closeness matching rule, the matching score is a weighted sum of the sequence matching score, the hit rate matching score and the closeness matching score.
7. The method according to claim 1, wherein The method further comprises: According to the feature matching rule, P documents matching the query feature sequence are searched from the database, the target document is any one of the P documents, and P is a positive integer.
8. The method according to claim 7, wherein The method further comprises: Displaying a document search page of the target application, wherein the document search page includes a search box, search options, and a query result display area; When there is an input operation acting on the search box, displaying the query data input into the search box in the search box; When the search option is selected, the P documents are displayed in the query result display area.
9. The method according to claim 8, wherein Each document in the P documents corresponds to a matching score, and a higher matching score indicates a higher matching degree between the document and the feature matching rule; The displaying of the P documents in the query result display area includes: The P documents are arranged and displayed in the query result display area in descending order of the matching scores.
10. A data processing device, characterized in that: The data processing device includes: an acquisition unit, configured to acquire query data and extract a query feature sequence from the query data, wherein the query feature sequence includes N feature data, where N is a positive integer; The acquisition unit is further configured to acquire feature matching rules, wherein the feature matching rules include at least two of the following: a sequence matching rule, a hit rate matching rule, and a closeness matching rule; a processing unit, configured to calculate, according to the feature matching rule, a matching score between the M hit data and the N feature data in the query feature sequence, and determine the first document as a target document matching the query feature sequence if the matching score is higher than a threshold; the M hit data being data in the first document in the database that matches the feature data in the query feature sequence; The matching score includes the closeness matching score corresponding to the closeness matching rule, and the process of determining the closeness matching score includes: determining the longest common subsequence of the query feature sequence according to the matching order between the M hit data and the query feature sequence; obtaining the first hit data located at the first arrangement position and the second hit data located at the last arrangement position in the longest common subsequence; obtaining the first position of the first hit data in the first document and the second position of the second hit data in the first document; calculating the length difference between the first position and the second position, and determining the third ratio between the length difference and the length of the query feature sequence as the closeness matching score; a common subsequence refers to a sequence containing the same data obtained by deleting a certain number of data from the sequence composed of the M hit data and the query feature sequence without changing the arrangement order of the remaining data in the sequence, and the longest common subsequence is the common subsequence containing the largest number of data.
11. A data processing device, characterized in that: The data processing device comprises: a processor adapted to implement a computer program; and A computer-readable storage medium storing a computer program, wherein the computer program is suitable for being loaded by the processor and executing the data processing method according to any one of claims 1 to 9.
12. A computer-readable storage medium, characterized in that The computer-readable storage medium comprises a computer program, and the computer program is suitable for being loaded by a processor and executing the data processing method according to any one of claims 1 to 9.
13. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the data processing method according to any one of claims 1 to 9.
Citation Information
Patent Citations
Search ranking method, apparatus, computer device, and storage medium
CN109086394A
Retrieved text recognition method and device and storage medium
CN111291214A
Document query method and device
CN111310477A