Generation method and device of range type search term, storage medium and electronic equipment
By combining the three-level tags in the video library and using large language model optimization, the problem of the inability to automatically generate range-based search terms in the existing technology is solved, and more accurate and efficient search results are achieved, improving user experience.
Patent Information
- Application Number
- CN202411344476.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-25
- Publication Date
- 2025-02-14
AI Technical Summary
The lack of automated generation of range-based search terms in existing search systems has led to the inability to fully meet users' complex search needs and weaken the user experience.
By obtaining the three-level tags in the video library, the initial range-based search terms are generated in combination, and the large language model is used to perform grammatical detection and optimization of these words, the words with grammatical problems are deleted, and the target range-based search terms are generated.
It realizes automated generation and optimization of scoped search terms collections, reduces dependence on manual construction and maintenance of vocabulary, reduces labor costs, and improves the accuracy and user experience of search results.
Smart Images
Figure CN119441536A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computers, and in particular to a method, device, storage medium and electronic device for generating range search terms. Background Art
[0002] In existing video websites, users usually search for videos they want to watch through search query portals. Users only need to enter a few keywords, and the system will display a list of recommended words related to these keywords. For example, when a user enters "genius" as a keyword, the system may recommend related recommended words such as [Genius Gunner, Genius Spectacle Dog, Genius Basic Law]. Due to the large number of users, the search module needs to build a complete vocabulary to meet the personalized search needs of users. However, there are a large number of users with range search intentions, and their search needs are more complex. At present, the range search vocabulary in the search system lacks an automatic generation method, and only relies on a vocabulary consisting of thousands of recommended words manually screened to support online search. This limited vocabulary is insufficient in processing range search intentions. When users enter related keywords, it is difficult for the system to recall appropriate recommended words, resulting in the search module being unable to fully meet the user's range search needs. This limitation weakens the user experience and fails to provide users with accurate search results. Therefore, it is particularly important to improve and expand the range search vocabulary, especially to generate more relevant recommended words through automated means to better meet the search needs of users. Summary of the invention
[0003] The present application provides a method, device, storage medium and electronic device for generating a range search term to solve the technical problem that a range search term cannot be generated by automated means.
[0004] In a first aspect, the present application provides a method for generating a range search term, comprising: obtaining all three-level tags for each video in a video library to obtain a tag set; combining all three-level tags in the above tag set to construct an initial range search term set; inputting each range search term in the above initial range search term set into a target large language model, identifying grammatical problems of each range search term in the above initial range search term set, and obtaining a recognition result of each range search term in the above initial range search term set; deleting a target range search term in the above initial range search term set to obtain a target range search term set, wherein the recognition result of the above target range search term is that there is a grammatical problem.
[0005] In a second aspect, the present application provides a device for generating range search terms, including: an acquisition module, used to acquire all three-level tags for each video in a video library to obtain a tag set; a first construction module, used to combine all three-level tags in the above tag set to construct an initial range search term set; an identification module, used to input each range search term in the above initial range search term set into a target large language model, identify grammatical problems of each range search term in the above initial range search term set, and obtain a recognition result of each range search term in the above initial range search term set; a deletion module, used to delete the target range search term in the above initial range search term set to obtain a target range search term set, wherein the recognition result of the above target range search term is that there is a grammatical problem.
[0006] As an optional example, the above-mentioned construction module includes: a determination unit, used to determine a combination rule, wherein the above-mentioned combination rule is that each range search word in the above-mentioned initial range search word set includes at least two third-level tags; a combination unit, used to combine all the third-level tags in the above-mentioned tag set according to the above-mentioned combination rule to obtain the above-mentioned initial range search word set.
[0007] As an optional example, the above-mentioned device also includes: a first processing module, which is used to, after constructing an initial range search word set, use each range search word in the above-mentioned initial range search word set as a current range search word, and perform the following operations on the above-mentioned current range search word: determine whether there is a range search word identical to the above-mentioned current range search word in the above-mentioned initial range search word set; if there is a range search word identical to the above-mentioned current range search word in the above-mentioned initial range search word set, delete all range search words identical to the above-mentioned current range search word in the above-mentioned initial range search word set except the above-mentioned current range search word.
[0008] As an optional example, the above-mentioned device also includes: a second construction module, which is used to, when there is a scope search term identical to the above-mentioned current scope search term in the above-mentioned initial scope search term set, delete all scope search terms identical to the above-mentioned current scope search term in the above-mentioned initial scope search term set except the above-mentioned current scope search term, and then construct a training data set and a verification data set according to the above-mentioned initial scope search term set; a second processing module, which is used to input the above-mentioned training data set into the initial large language model to obtain a result data set of the above-mentioned training data set; a first calculation module, which is used to calculate the difference between the above-mentioned result data set and the above-mentioned verification data set using a loss function to obtain a loss value; and a second calculation module, which is used to adjust the model parameters of the above-mentioned initial large language model according to the above-mentioned loss value, and recalculate the above-mentioned loss value until the recalculated loss value is less than the target threshold value to obtain the above-mentioned target large language model.
[0009] As an optional example, the above-mentioned recognition module includes: a processing unit, used to take each range search term in the above-mentioned initial range search term set as a current range search term, and perform the following operations on the above-mentioned current range search term: searching for a target video corresponding to the above-mentioned current range search term in the above-mentioned video library; when the above-mentioned target video does not exist in the above-mentioned video library, determining that the grammatical problem of the above-mentioned current range search term is that there is a grammatical problem; when the above-mentioned target video exists in the above-mentioned video library, determining that the grammatical problem of the above-mentioned current range search term is that there is no grammatical problem.
[0010] As an optional example, the above-mentioned processing unit is used to split the above-mentioned current range search term to obtain all current three-level tags; obtain a video set corresponding to each of the above-mentioned current three-level tags from the above-mentioned video library; and determine a video existing in each of the above-mentioned video sets as a target video corresponding to the above-mentioned current range search term.
[0011] In a third aspect, the present application provides a storage medium storing a computer program, wherein the computer program, when executed by a processor, executes the above-mentioned method for generating range search terms.
[0012] In a fourth aspect, the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-mentioned method for generating range search terms through the computer program.
[0013] In an embodiment of the present application, the method adopts the steps of obtaining all the three-level tags of each video in the video library to obtain a tag set; combining all the three-level tags in the above tag set to construct an initial range search word set; inputting each range search word in the above initial range search word set into the target large language model, identifying the grammatical problem of each range search word in the above initial range search word set, and obtaining the recognition result of each range search word in the above initial range search word set; deleting the target range search word in the above initial range search word set to obtain the target range search word set, wherein the recognition result of the above target range search word is a method with grammatical problems. In the above method, the range search word set is generated by automatic combination of three-level tags, and the initial range search word set is subjected to grammatical detection and optimization by the large language model, and finally a more grammatically accurate range search word set is generated, thereby realizing the purpose of automatically generating and optimizing the range search word set, reducing the dependence on manually constructing and maintaining the word library, and reducing the labor cost, thereby solving the technical problem that the range search word cannot be generated by automated means. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0016] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.
[0017] Figure 1 is a flowchart of an optional method for generating a range-based search term according to an embodiment of the present application;
[0018] Figure 2 is a schematic structural diagram of an optional range-based search term generation device according to an embodiment of the present application;
[0019] Figure 3 is a schematic diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0021] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.
[0022] According to a first aspect of an embodiment of the present application, a method for generating a range search term is provided. Optionally, as follows: Figure 1 As shown, the above method includes:
[0023] S102, obtaining all three-level tags of each video in the video library to obtain a tag set;
[0024] S104, combining all the third-level tags in the tag set to construct an initial range search word set;
[0025] S106, inputting each scoped search word in the initial scoped search word set into the target large language model, identifying a grammatical problem of each scoped search word in the initial scoped search word set, and obtaining a recognition result of each scoped search word in the initial scoped search word set;
[0026] S108, deleting the target range search words in the initial range search word set to obtain a target range search word set, wherein the recognition result of the target range search words is that there is a grammatical problem.
[0027] Optionally, in this embodiment, a large language model (LLM) refers to a large-scale natural language processing (NLP) model trained by a deep learning method. Such models usually have hundreds of millions or even billions of parameters, can perform a variety of NLP tasks, and can capture and generate complex structures and contexts in natural languages.
[0028] Optionally, in this embodiment, when a video website stores PPC (Professional Produced Content, a long video in a film and television website), it will associate the long video with a tag, build a knowledge graph, and store the data in a database. PPC tags cover multiple dimensions such as starring stars, types, plots, styles, and themes, and relevant information can be obtained by parsing knowledge graph data. In the tag system, secondary tags are manually constructed by editors and operators, and are further abstracted or classified from tertiary tags. For example, the secondary tag "theme" may correspond to multiple tertiary tags, such as "crime", "action", "police and bandits", etc. The data of the tertiary tags is stored in a specific video library and can be obtained by querying the video library to obtain a tag set. All the tertiary tags in the tag set are combined to construct an initial range search word set, and these combined words cover the range search intent that the user may enter. After constructing the initial range search word set, these range search words are input into the target large language model to identify the grammatical problems of each range search word and obtain the corresponding recognition results. If a range search term is identified as having a grammatical problem, it will be deleted from the initial range search term set, and finally a grammatically correct and widely covered target range search term set will be generated. Finally, the target range search term set will be applied to the search system.
[0029] Optionally, in this embodiment, by automatically constructing a range search word set, the user's search intent can be matched more accurately, especially when the user performs a complex or multi-dimensional search, the system can recommend more relevant query results and improve the user experience. Using a three-level tag combination to generate a range search word set can significantly expand the coverage of the search word library, thereby increasing the system's responsiveness when the user enters complex search conditions and reducing the problem of missing recommended words due to insufficient word library. The syntax detection and optimization of the initial range search word set by the large language model ensures that the final generated search word set is more grammatically accurate. This not only improves the effectiveness of the search, but also reduces invalid searches caused by grammatical errors by users. The process of automatically generating and optimizing the range search word set reduces the reliance on manually building and maintaining the word library, reduces labor costs, and can update and expand the word library more timely, maintain the real-time and accuracy of the system, and thus solves the technical problem of being unable to generate range search words through automated means.
[0030] As an optional example, all third-level tags in the tag set are combined to construct an initial range search term set including:
[0031] Determine a combination rule, wherein the combination rule is that each range search word in the initial range search word set includes at least two third-level tags;
[0032] According to the combination rule, all the third-level tags in the tag set are combined to obtain an initial range search word set.
[0033] Optionally, in this embodiment, when constructing the initial range search term set, the combination rule needs to be clarified first. The combination rule specifies how to combine the three-level tags to form a range search term with practical search significance. In this rule, it can be specified that each range search term must contain at least two different three-level tags. This means that the search term needs to involve multiple dimensions or attributes at the same time so that users can express their search intentions through more specific queries. For example, if there are two three-level tags "crime" and "action" in the tag set, after following the combination rule, a range search term such as "crime action movie" can be generated. The rule can also be set to include at least two three-level tags under the second-level tags, such as two second-level tags "era" and "subject matter", and then according to the three-level tags, they are arranged and combined, and range search terms such as "medieval religious movies" and "Ming Dynasty costume suspense TV series" can be constructed. After determining the combination rule, the next step is to systematically combine all the three-level tags. This process can be completed by an automated algorithm to ensure that as many range search terms that meet the rules as possible are generated in the tag set. For example, if there are multiple third-level tags in the tag set, such as "love", "suspense", "ancient costume", etc., the system will generate possible search word combinations based on the combination rules, such as "Tang Dynasty love suspense movie", "ancient costume love drama", etc. The initial range search word set generated by this process will cover the user's possible diverse search needs.
[0034] Optionally, in this embodiment, more specific range search terms generated automatically can help users find content they are interested in more quickly, reduce frustration and invalid queries during the search process, and improve the overall user experience. Generating search terms through predefined combination rules ensures the structural logic and grammatical correctness of the search terms, avoids grammatical errors that may occur during user input, and improves the effectiveness of the search. The automated generation and management of search term sets reduces the need for manual intervention, reduces maintenance costs, and can respond more quickly to content changes and user needs.
[0035] As an optional example, after constructing the initial range search word set, the method further includes:
[0036] Take each range search term in the initial range search term set as the current range search term, and perform the following operations on the current range search term:
[0037] Determine whether there is a range search term that is the same as the current range search term in the initial range search term set;
[0038] When there is a scope search term identical to the current scope search term in the initial scope search term set, all scope search terms identical to the current scope search term in the initial scope search term set except the current scope search term are deleted.
[0039] Optionally, in this embodiment, in the process of constructing and optimizing the range search word set, it is first necessary to check whether there are repeated range search words. The initial range search word set is generated by combining multiple third-level tags, so the same search word may be generated multiple times. For example, if different combination orders or different generation processes cause the search word "love suspense movie" to be generated multiple times, such repeated entries not only occupy resources, but also may reduce search efficiency. In order to avoid this situation, the system needs to determine in real time whether the initial range search word set already has the same entry as the currently generated range search word when constructing the set. If the same entry exists, it means that the search word has been recorded and does not need to be retained again. If it is detected that there are other entries that are the same as the current range search word in the initial range search word set, then these repeated entries need to be deleted and only one unique search word is retained. For example, if the search word "love suspense movie" already exists in the set, then if this entry appears again during the generation process, the system will delete the newly generated duplicates and only retain the original search word. This deduplication process ensures the simplicity and efficiency of the search word set and avoids unnecessary data redundancy.
[0040] As an optional example, in the case where there is a scope search term identical to the current scope search term in the initial scope search term set, after deleting all scope search terms identical to the current scope search term in the initial scope search term set except the current scope search term, the method further includes:
[0041] Based on the initial range search word set, build a training data set and a validation data set.
[0042] Input the training data set into the initial large language model to obtain a result data set of the training data set;
[0043] Use the loss function to calculate the difference between the result dataset and the validation dataset to get the loss value;
[0044] The model parameters of the initial large language model are adjusted according to the loss value, and the loss value is recalculated until the recalculated loss value is less than the target threshold, thereby obtaining the target large language model.
[0045] Optionally, in this embodiment, before using the target large language model to perform initial range search term set optimization, the target large language model needs to be trained to improve the recognition accuracy of the large language model.
[0046] Through the PPC tag, you can know the video list of long videos corresponding to each tag. The range search term set is constructed according to the combination rules. In essence, it is a combination of tags. If the video lists corresponding to two tags do not intersect, it means that the range search term has no corresponding PPC. Because it is not possible to recommend a range search term that has no resources to undertake to the user, such a range search term should be filtered out. For example: the range search term in the range search term set is "Tang Dynasty science fiction movies". Although this is grammatically legal, it has no PPC resources to undertake, so it should be filtered out. Therefore, this situation is set as a grammatical problem with the range search term.
[0047] According to the initial range search term set, two data sets need to be constructed first: a training data set and a validation data set. The training data set is used to train the large language model to help the model learn the pattern and structure of the search terms. The validation data set is used to evaluate the performance of the model and verify the accuracy and generalization ability of the model when processing unseen data. The two data sets are usually divided according to a certain ratio to ensure that the model can obtain sufficient data support during the training and validation process. The training data set is labeled and input into the initial large language model. The model will output the result of whether each range search term has a grammatical problem by learning the patterns and relationships in the training data, and generate a result data set. This result data set reflects the model's ability to process and understand the input data in the current training state. The loss function is used to evaluate the difference between the result data set and the validation data set. The loss function is a function that measures the gap between the model prediction and the actual result. By calculating the loss value, the prediction error of the model can be quantified and provide a basis for model optimization. According to the calculated loss value, the model parameters of the large language model are adjusted through SFT. The goal of adjusting the model parameters is to reduce the loss value and make the model's prediction results more accurate. After adjusting the parameters, the model will recalculate the loss value and check whether the new loss value is lower than the previous one. This process will be repeated until the loss value is lower than the set target threshold or the training cycle reaches a certain number of times. At this time, the model is considered to have been fully optimized and meets the standards of the target large language model.
[0048] SFT (Supervised Fine-Tuning) refers to fine-tuning a model using labeled data based on a pre-trained model to adapt it to a specific task. Supervised fine-tuning is the process of further training based on a pre-trained model. In the field of NLP, models are usually first pre-trained on large-scale unlabeled text data to learn general language representations. After that, the model is fine-tuned using a labeled dataset for a specific task to perform better on the specific task.
[0049] As an optional example, each scoped search word in the initial scoped search word set is input into the target large language model, and a grammatical problem of each scoped search word in the initial scoped search word set is identified, and the recognition result of each scoped search word in the initial scoped search word set is obtained, including:
[0050] Take each range search term in the initial range search term set as the current range search term, and perform the following operations on the current range search term:
[0051] Search the video library for the target video corresponding to the current range search term;
[0052] When the target video does not exist in the video library, determining that a grammatical problem of the current range search term exists;
[0053] When the target video exists in the video library, it is determined that there is no grammatical problem with the current range search term.
[0054] Optionally, in this embodiment, each range search term in the initial range search term set is sequentially input into the target large language model, and the target large language model has been trained to accurately identify and judge grammatical problems in the range search terms. The model performs grammatical analysis on each input range search term and gives the recognition result of the range search term. The recognition result will indicate whether there is a grammatical problem in the range search term. Taking a certain range search term as an example, first, the range search term is split into each independent three-level label. For example, the range search term "ancient costume martial arts movie" can be split into two three-level labels: "ancient costume" and "martial arts". The video set corresponding to each three-level label is obtained from the video library. Each three-level label will correspond to a group of videos in the video library, and these videos are related to the label. For example, the label "ancient costume" may correspond to a group of videos with ancient costume themes, while "martial arts" corresponds to another group of videos with martial arts themes. The video set corresponding to each three-level label is subjected to an intersection operation, that is, to find videos that exist in all sets. Only when the same video exists in each video collection can it be determined that the grammatical problem of the range search term is not a grammatical problem, and the search term can effectively retrieve videos related to the user's intention. If the target video that meets all the three-level label conditions is not found in the video library, that is, the intersection of all video collections is empty, it is determined that the grammatical problem of the range search term is a grammatical problem.
[0055] As an optional example, searching for a target video corresponding to the current range search term in the video library includes:
[0056] Split the current range search term to obtain all current third-level tags;
[0057] Obtain a video set corresponding to each current third-level tag from the video library;
[0058] The videos existing in each video set are determined as target videos corresponding to the current range search term.
[0059] Optionally, in this embodiment, when searching the video library for videos corresponding to the range search term, the range search term is split into independent three-level tags. For example, the range search term "ancient costume martial arts movies" can be split into two three-level tags: "ancient costume" and "martial arts". The video set corresponding to each three-level tag is obtained from the video library, and the video set corresponding to each three-level tag is intersected, that is, the videos that exist in all sets are searched. Only when the same video exists in each video set can it be determined that the grammatical problem of the range search term is that there is no grammatical problem, and the search term can effectively retrieve videos related to the user's intention. If the target video that meets all the three-level tag conditions is not found in the video library, that is, it is empty in the intersection of all video sets, it is determined that the grammatical problem of the range search term is that there is a grammatical problem.
[0060] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.
[0061] According to another aspect of the embodiment of the present application, a device for generating a range search term is also provided, such as Figure 2 As shown, including:
[0062] The acquisition module 202 is used to acquire all the third-level tags of each video in the video library to obtain a tag set;
[0063] The first construction module 204 is used to combine all the third-level tags in the tag set to construct an initial range search word set;
[0064] The recognition module 206 is used to input each scoped search word in the initial scoped search word set into the target large language model, recognize the grammatical problem of each scoped search word in the initial scoped search word set, and obtain the recognition result of each scoped search word in the initial scoped search word set;
[0065] The deletion module 208 is used to delete the target range search words in the initial range search word set to obtain the target range search word set, wherein the recognition result of the target range search words is that there is a grammatical problem.
[0066] Optionally, in this embodiment, a large language model (LLM) refers to a large-scale natural language processing (NLP) model trained by a deep learning method. Such models usually have hundreds of millions or even billions of parameters, can perform a variety of NLP tasks, and can capture and generate complex structures and contexts in natural languages.
[0067] Optionally, in this embodiment, when a video website stores PPC (Professional Produced Content, a long video in a film and television website), it will associate the long video with a tag, build a knowledge graph, and store the data in a database. PPC tags cover multiple dimensions such as starring stars, types, plots, styles, and themes, and relevant information can be obtained by parsing knowledge graph data. In the tag system, secondary tags are manually constructed by editors and operators, and are further abstracted or classified from tertiary tags. For example, the secondary tag "theme" may correspond to multiple tertiary tags, such as "crime", "action", "police and bandits", etc. The data of the tertiary tags is stored in a specific video library and can be obtained by querying the video library to obtain a tag set. All the tertiary tags in the tag set are combined to construct an initial range search word set, and these combined words cover the range search intent that the user may enter. After constructing the initial range search word set, these range search words are input into the target large language model to identify the grammatical problems of each range search word and obtain the corresponding recognition results. If a range search term is identified as having a grammatical problem, it will be deleted from the initial range search term set, and finally a grammatically correct and widely covered target range search term set will be generated. Finally, the target range search term set will be applied to the search system.
[0068] Optionally, in this embodiment, by automatically constructing a range search word set, the user's search intent can be matched more accurately, especially when the user performs a complex or multi-dimensional search, the system can recommend more relevant query results and improve the user experience. Using a three-level tag combination to generate a range search word set can significantly expand the coverage of the search word library, thereby increasing the system's responsiveness when the user enters complex search conditions and reducing the problem of missing recommended words due to insufficient word library. The syntax detection and optimization of the initial range search word set by the large language model ensures that the final generated search word set is more grammatically accurate. This not only improves the effectiveness of the search, but also reduces invalid searches caused by grammatical errors by users. The process of automatically generating and optimizing the range search word set reduces the reliance on manually building and maintaining the word library, reduces labor costs, and can update and expand the word library more timely, maintain the real-time and accuracy of the system, and thus solves the technical problem of being unable to generate range search words through automated means.
[0069] As an optional example, the building blocks include:
[0070] A determination unit, configured to determine a combination rule, wherein the combination rule is that each range search term in the initial range search term set includes at least two third-level tags;
[0071] The combining unit is used to combine all the third-level tags in the tag set according to the combination rule to obtain an initial range search word set.
[0072] Optionally, in this embodiment, when constructing the initial range search term set, the combination rule needs to be clarified first. The combination rule specifies how to combine the three-level tags to form a range search term with practical search significance. In this rule, it can be specified that each range search term must contain at least two different three-level tags. This means that the search term needs to involve multiple dimensions or attributes at the same time so that users can express their search intentions through more specific queries. For example, if there are two three-level tags "crime" and "action" in the tag set, after following the combination rule, a range search term such as "crime action movie" can be generated. The rule can also be set to include at least two three-level tags under the second-level tags, such as two second-level tags "era" and "subject matter", and then according to the three-level tags, they are arranged and combined, and range search terms such as "medieval religious movies" and "Ming Dynasty costume suspense TV series" can be constructed. After determining the combination rule, the next step is to systematically combine all the three-level tags. This process can be completed by an automated algorithm to ensure that as many range search terms that meet the rules as possible are generated in the tag set. For example, if there are multiple third-level tags in the tag set, such as "love", "suspense", "ancient costume", etc., the system will generate possible search word combinations based on the combination rules, such as "Tang Dynasty love suspense movie", "ancient costume love drama", etc. The initial range search word set generated by this process will cover the user's possible diverse search needs.
[0073] Optionally, in this embodiment, more specific range search terms generated automatically can help users find content they are interested in more quickly, reduce frustration and invalid queries during the search process, and improve the overall user experience. Generating search terms through predefined combination rules ensures the structural logic and grammatical correctness of the search terms, avoids grammatical errors that may occur during user input, and improves the effectiveness of the search. The automated generation and management of search term sets reduces the need for manual intervention, reduces maintenance costs, and can respond more quickly to content changes and user needs.
[0074] As an optional example, the above device further includes:
[0075] The first processing module is used to, after constructing an initial range search word set, use each range search word in the initial range search word set as a current range search word, and perform the following operations on the current range search word:
[0076] Determine whether there is a range search term that is the same as the current range search term in the initial range search term set;
[0077] When there is a scope search term identical to the current scope search term in the initial scope search term set, all scope search terms identical to the current scope search term in the initial scope search term set except the current scope search term are deleted.
[0078] Optionally, in this embodiment, in the process of constructing and optimizing the range search word set, it is first necessary to check whether there are repeated range search words. The initial range search word set is generated by combining multiple third-level tags, so the same search word may be generated multiple times. For example, if different combination orders or different generation processes cause the search word "love suspense movie" to be generated multiple times, such repeated entries not only occupy resources, but also may reduce search efficiency. In order to avoid this situation, the system needs to determine in real time whether the initial range search word set already has the same entry as the currently generated range search word when constructing the set. If the same entry exists, it means that the search word has been recorded and does not need to be retained again. If it is detected that there are other entries that are the same as the current range search word in the initial range search word set, then these repeated entries need to be deleted and only one unique search word is retained. For example, if the search word "love suspense movie" already exists in the set, then if this entry appears again during the generation process, the system will delete the newly generated duplicates and only retain the original search word. This deduplication process ensures the simplicity and efficiency of the search word set and avoids unnecessary data redundancy.
[0079] As an optional example, the above device further includes:
[0080] A second construction module is used for, when there is a range search term identical to the current range search term in the initial range search term set, deleting all range search terms identical to the current range search term in the initial range search term set except the current range search term, and then constructing a training data set and a validation data set according to the initial range search term set;
[0081] A second processing module is used to input the training data set into the initial large language model to obtain a result data set of the training data set;
[0082] A first calculation module is used to calculate the difference between the result data set and the verification data set using a loss function to obtain a loss value;
[0083] The second calculation module is used to adjust the model parameters of the initial large language model according to the loss value, and recalculate the loss value until the recalculated loss value is less than the target threshold, so as to obtain the target large language model.
[0084] Optionally, in this embodiment, before using the target large language model to perform initial range search term set optimization, the target large language model needs to be trained to improve the recognition accuracy of the large language model.
[0085] Through the PPC tag, you can know the video list of long videos corresponding to each tag. The range search term set is constructed according to the combination rules. In essence, it is a combination of tags. If the video lists corresponding to two tags do not intersect, it means that the range search term has no corresponding PPC. Because it is not possible to recommend a range search term that has no resources to undertake to the user, such a range search term should be filtered out. For example: the range search term in the range search term set is "Tang Dynasty science fiction movies". Although this is grammatically legal, it has no PPC resources to undertake, so it should be filtered out. Therefore, this situation is set as a grammatical problem with the range search term.
[0086] According to the initial range search term set, two data sets need to be constructed first: a training data set and a validation data set. The training data set is used to train the large language model to help the model learn the pattern and structure of the search terms. The validation data set is used to evaluate the performance of the model and verify the accuracy and generalization ability of the model when processing unseen data. The two data sets are usually divided according to a certain ratio to ensure that the model can obtain sufficient data support during the training and validation process. The training data set is labeled and input into the initial large language model. The model will output the result of whether each range search term has a grammatical problem by learning the patterns and relationships in the training data, and generate a result data set. This result data set reflects the model's ability to process and understand the input data in the current training state. The loss function is used to evaluate the difference between the result data set and the validation data set. The loss function is a function that measures the gap between the model prediction and the actual result. By calculating the loss value, the prediction error of the model can be quantified and provide a basis for model optimization. According to the calculated loss value, the model parameters of the large language model are adjusted through SFT. The goal of adjusting the model parameters is to reduce the loss value and make the model's prediction results more accurate. After adjusting the parameters, the model will recalculate the loss value and check whether the new loss value is lower than the previous one. This process will be repeated until the loss value is lower than the set target threshold or the training cycle reaches a certain number of times. At this time, the model is considered to have been fully optimized and meets the standards of the target large language model.
[0087] SFT (Supervised Fine-tuning) refers to fine-tuning a model using labeled data based on a pre-trained model to adapt it to a specific task. Supervised fine-tuning is the process of further training based on a pre-trained model. In the field of NLP, models are usually first pre-trained on large-scale unlabeled text data to learn general language representations. After that, the model is fine-tuned using a labeled dataset for a specific task to perform better on the specific task.
[0088] As an optional example, the recognition module includes: a processing unit, configured to use each range search term in the initial range search term set as a current range search term, and perform the following operations on the current range search term:
[0089] Search the video library for the target video corresponding to the current range search term;
[0090] When the target video does not exist in the video library, determining that a grammatical problem of the current range search term exists;
[0091] When the target video exists in the video library, it is determined that there is no grammatical problem with the current range search term.
[0092] Optionally, in this embodiment, each range search term in the initial range search term set is sequentially input into the target large language model, and the target large language model has been trained to accurately identify and judge grammatical problems in the range search terms. The model performs grammatical analysis on each input range search term and gives the recognition result of the range search term. The recognition result will indicate whether there is a grammatical problem in the range search term. Taking a certain range search term as an example, first, the range search term is split into each independent three-level label. For example, the range search term "ancient costume martial arts movie" can be split into two three-level labels: "ancient costume" and "martial arts". The video set corresponding to each three-level label is obtained from the video library. Each three-level label will correspond to a group of videos in the video library, and these videos are related to the label. For example, the label "ancient costume" may correspond to a group of videos with ancient costume themes, while "martial arts" corresponds to another group of videos with martial arts themes. The video set corresponding to each three-level label is subjected to an intersection operation, that is, to find videos that exist in all sets. Only when the same video exists in each video collection can it be determined that the grammatical problem of the range search term is not a grammatical problem, and the search term can effectively retrieve videos related to the user's intention. If the target video that meets all the three-level label conditions is not found in the video library, that is, the intersection of all video collections is empty, it is determined that the grammatical problem of the range search term is a grammatical problem.
[0093] As an optional example, the processing unit is used to split the current range search term to obtain all current third-level tags;
[0094] Obtain a video set corresponding to each current third-level tag from the video library;
[0095] The videos existing in each video set are determined as target videos corresponding to the current range search term.
[0096] Optionally, in this embodiment, when searching the video library for videos corresponding to the range search term, the range search term is split into independent three-level tags. For example, the range search term "ancient costume martial arts movies" can be split into two three-level tags: "ancient costume" and "martial arts". The video set corresponding to each three-level tag is obtained from the video library, and the video set corresponding to each three-level tag is intersected, that is, the videos that exist in all sets are searched. Only when the same video exists in each video set can it be determined that the grammatical problem of the range search term is that there is no grammatical problem, and the search term can effectively retrieve videos related to the user's intention. If the target video that meets all the three-level tag conditions is not found in the video library, that is, it is empty in the intersection of all video sets, it is determined that the grammatical problem of the range search term is that there is a grammatical problem.
[0097] For other examples of this embodiment, please refer to the above examples and will not be repeated here.
[0098] Figure 3 is a schematic diagram of an optional electronic device according to an embodiment of the present application, such as Figure 3 As shown, it includes a processor 302, a communication interface 304, a memory 306 and a communication bus 308, wherein the processor 302, the communication interface 304 and the memory 306 communicate with each other through the communication bus 308, wherein,
[0099] Memory 306, used for storing computer programs;
[0100] The processor 302 is used to implement the following steps when executing the computer program stored in the memory 306:
[0101] Get all the three-level tags of each video in the video library to get the tag set;
[0102] Combine all the third-level tags in the tag set to construct an initial range search term set;
[0103] Inputting each scoped search word in the initial scoped search word set into the target large language model, identifying a grammatical problem of each scoped search word in the initial scoped search word set, and obtaining a recognition result of each scoped search word in the initial scoped search word set;
[0104] The target range search words in the initial range search word set are deleted to obtain a target range search word set, wherein the recognition result of the target range search words is that there is a grammatical problem.
[0105] Optionally, in this embodiment, the communication bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 The communication interface is used for communication between the above electronic device and other devices.
[0106] The memory may include RAM, or may include non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0107] As an example, the memory 306 may include, but is not limited to, the acquisition module 202, the first construction module 204, the identification module 206, and the deletion module 208 in the device for generating the range-based search term. In addition, other module units in the device for generating the range-based search term may also be included but are not limited to, which will not be described in detail in this example.
[0108] The above-mentioned processor can be a general-purpose processor, which can include but not be limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0109] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.
[0110] It can be understood by those skilled in the art that Figure 3 The structure shown is for illustration only, and the device for implementing the above-mentioned range search term generation method may be a terminal device, which may be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, and other terminal devices. Figure 3 The structure of the electronic device is not limited. For example, the electronic device may also include Figure 3 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 3 Different configurations shown.
[0111] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.
[0112] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program executes the steps in the above-mentioned method for generating range search terms when executed by a processor.
[0113] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.
[0114] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0115] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.
[0116] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0117] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0118] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0119] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0120] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A method for generating a range search term, characterized in that: include: Get all the three-level tags of each video in the video library to get the tag set; Combining all the third-level tags in the tag set to construct an initial range search word set; Inputting each scoped search word in the initial scoped search word set into the target large language model, identifying a grammatical problem of each scoped search word in the initial scoped search word set, and obtaining a recognition result of each scoped search word in the initial scoped search word set; The target range search words in the initial range search word set are deleted to obtain a target range search word set, wherein the recognition result of the target range search words is that there is a grammatical problem.
2. The method according to claim 1, characterized in that The step of combining all the third-level tags in the tag set to construct an initial range search word set includes: Determine a combination rule, wherein the combination rule is that each range search word in the initial range search word set includes at least two third-level tags; According to the combination rule, all the third-level tags in the tag set are combined to obtain the initial range search word set.
3. The method according to claim 1, characterized in that: After constructing the initial range search word set, the method further includes: Each range search term in the initial range search term set is used as a current range search term, and the following operations are performed on the current range search term: Determining whether there is a range search term identical to the current range search term in the initial range search term set; In the case where there is a scope search term identical to the current scope search term in the initial scope search term set, all scope search terms identical to the current scope search term in the initial scope search term set except the current scope search term are deleted.
4. The method according to claim 3, characterized in that In the case that there is a scoped search term identical to the current scoped search term in the initial scoped search term set, after deleting all scoped search terms identical to the current scoped search term in the initial scoped search term set except the current scoped search term, the method further includes: According to the initial range search word set, a training data set and a validation data set are constructed, Inputting the training data set into an initial large language model to obtain a result data set of the training data set; Using a loss function to calculate the difference between the result data set and the verification data set, to obtain a loss value; The model parameters of the initial large language model are adjusted according to the loss value, and the loss value is recalculated until the recalculated loss value is less than a target threshold, thereby obtaining the target large language model.
5. The method according to claim 1, characterized in that The step of inputting each scoped search word in the initial scoped search word set into the target large language model, identifying a grammatical problem of each scoped search word in the initial scoped search word set, and obtaining a recognition result of each scoped search word in the initial scoped search word set includes: Each range search term in the initial range search term set is used as a current range search term, and the following operations are performed on the current range search term: Searching the video library for a target video corresponding to the current range search term; In a case where the target video does not exist in the video library, determining that a grammatical problem of the current range search term is a grammatical problem; In a case where the target video exists in the video library, it is determined that there is no grammatical problem with the current range search term.
6. The method according to claim 5, characterized in that The step of searching the video library for a target video corresponding to the current range search term comprises: Split the current range search term to obtain all current third-level tags; Acquire a video set corresponding to each of the current third-level tags from the video library; A video existing in each of the video sets is determined as a target video corresponding to the current range search term.
7. A device for generating a range search term, characterized in that: include: The acquisition module is used to obtain all the three-level tags of each video in the video library to obtain a tag set; A first construction module is used to combine all the third-level tags in the tag set to construct an initial range search word set; a recognition module, configured to input each scoped search word in the initial scoped search word set into a target large language model, recognize a grammatical problem of each scoped search word in the initial scoped search word set, and obtain a recognition result of each scoped search word in the initial scoped search word set; The deleting module is used to delete the target range search words in the initial range search word set to obtain a target range search word set, wherein the recognition result of the target range search words is that there is a grammatical problem.
8. The device according to claim 7, characterized in that The building blocks include: A determination unit, configured to determine a combination rule, wherein the combination rule is that each range search term in the initial range search term set includes at least two third-level tags; The combining unit is used to combine all the third-level tags in the tag set according to the combining rule to obtain the initial range search word set.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is executed.
10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 6 through the computer program.