Data determination method and device, electronic equipment and storage medium
By receiving target topic words and target category tags, expanding topic words, and matching the expanded topic words and target category tags from the preset data-label library, the problem that the model is not universal in traditional data mining methods is solved, and a high-versatility and reusability data determination method is realized.
Patent Information
- Application Number
- CN202311466390.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-06
- Publication Date
- 2025-05-06
AI Technical Summary
In traditional data mining methods, the high coupling between business and model results in the mining model not being universal and difficult to adapt to different business needs.
By receiving target topic words and target category tags, the subject word augmentation is performed, and the subject word and target category tags are matched from the preset data-label library to determine the data. This method uses preset general category labels for data annotation and library construction to ensure data universality and reusability.
It realizes high universality of data determination methods, can adapt to different business needs, improve data recall and query efficiency, and has good reusability and scalability.
Smart Images
Figure CN119938728A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present disclosure relate to the field of computer technology, and in particular, to a data determination method, device, electronic device, and storage medium. Background Art
[0002] In traditional data mining methods, it is necessary to build a corresponding data mining model for each business requirement. The high degree of coupling between business and model makes the mining model not universal. Summary of the invention
[0003] The embodiments of the present disclosure provide a data determination method, device, electronic device, and storage medium, which can achieve highly universal data determination.
[0004] In a first aspect, an embodiment of the present disclosure provides a data determination method, including:
[0005] Receiving a target subject word and a target category label; wherein the target category label is included in a preset general category label;
[0006] Expanding the target keyword to obtain an expanded keyword;
[0007] From the preset data-label library, target data matching the expanded keyword and the target category label is determined; wherein the label corresponding to the data in the preset data-label belongs to the general category label.
[0008] In a second aspect, the present disclosure also provides a data determination device, including:
[0009] A receiving module, used for receiving a target subject word and a target category label; wherein the target category label is included in a preset general category label;
[0010] An expansion module, used for expanding the target keyword to obtain an expanded keyword;
[0011] A matching module is used to determine target data that matches the expanded keyword and the target category label from a preset data-label library; wherein the label corresponding to the data in the preset data-label belongs to the general category label.
[0012] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:
[0013] one or more processors;
[0014] a storage device for storing one or more programs,
[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the data determination method as described in any one of the embodiments of the present disclosure.
[0016] In a fourth aspect, an embodiment of the present disclosure further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to execute the data determination method as described in any one of the embodiments of the present disclosure.
[0017] The technical solution of the embodiment of the present disclosure receives a target subject word and a target category label; wherein the target category label is included in a preset general category label; the target subject word is expanded to obtain an expanded subject word; and target data matching the expanded subject word and the target category label is determined from a preset data-label library; wherein the label corresponding to the data in the preset data-label belongs to the general category label. By presetting a general category label that is common to any subject word, the reuse of category labels can be achieved. By pre-labeling the source data according to the general category label in advance to construct a data-label library, it is possible to determine the data that hits the target subject word and the target category label from the data-label, which has good versatility. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.
[0019] Figure 1 A flowchart of a data determination method provided by an embodiment of the present disclosure;
[0020] Figure 2 A block diagram of a data determination method provided by an embodiment of the present disclosure;
[0021] Figure 3 A schematic diagram of a flow chart of determining an expanded keyword in a data determination method provided in an embodiment of the present disclosure;
[0022] Figure 4 A schematic diagram of a block diagram for determining an expanded keyword in a data determination method provided by an embodiment of the present disclosure;
[0023] Figure 5 A schematic diagram of a process for constructing a preset data-label library in a data determination method provided in an embodiment of the present disclosure;
[0024] Figure 6 A flowchart of a data determination method provided by an embodiment of the present disclosure;
[0025] Figure 7 A schematic diagram of the structure of a data determination device provided by an embodiment of the present disclosure;
[0026] Figure 8 A schematic diagram of the structure of a data determination device provided by an embodiment of the present disclosure;
[0027] Fig. 9 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION
[0028] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0029] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.
[0030] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.
[0031] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0032] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0033] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0034] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.
[0035] Figure 1 The present invention provides a flow chart of a data determination method provided by an embodiment of the present invention, which is applicable to the case of determining target data from a database. The method can be executed by a data determination device, which can be implemented in the form of software and / or hardware, and can be configured in an electronic device, such as a computer.
[0036] like Figure 1 As shown, the data determination method provided in this embodiment may include:
[0037] S110 , receiving a target subject word and a target category label; wherein the target category label is included in a general category label that is common to any subject word.
[0038] In the embodiment of the present disclosure, data determination can be understood as querying data related to the target object from a large amount of data. Among them, the target keyword can include the name of the target object. Among them, the general category label can divide the dimensions related to any keyword. Accordingly, the target category label can represent the target dimension related to the target object.
[0039] Among them, the general category label can be updated based on the update operation, for example, it can include at least one of the following: based on the registration operation, a new general category label can be added; based on the cancellation operation, an existing general category label can be deleted; based on the fusion operation, at least one single-dimensional general category label can be fused to obtain a multi-dimensional general category label, etc. By continuously updating and maintaining the general category label, it can be determined that the task has better universality for different data.
[0040] The data determination device may include a platform layer, which may provide a user interface for inputting target keywords and target category labels. Accordingly, the data determination device may receive the target keywords and target category labels in response to the user input operation in the user interface.
[0041] In addition, in some optional implementations, receiving target keywords and target category labels may include: processing the data to be processed through a language model to obtain a processing result; if there are missing items in the processing result, generating feedback data based on the missing items in the processing result; wherein the missing items include at least one of the target keywords and the target category label; and prompting the feedback data.
[0042] Among them, the language model may include an existing model that can process natural language; the data to be processed may include text data and / or voice data, etc. Exemplarily, the data to be processed may be the first-level comment data publicly released by the user and the multi-level comment data of the public reply to the comment. The language model can be constructed in advance based on sample data, sample keywords and sample category labels, wherein the sample category table labels are also included in the general category labels. Accordingly, based on the constructed language model, the target keywords and target category labels contained in the data to be processed can be identified. Among them, the processing result can be considered as the result of identifying the data to be processed based on the constructed language model.
[0043] There may be a situation where at least one of the target subject words and the target category label is missing in the processing result. Then, the data determination device can generate corresponding feedback data according to the missing items. Here, the feedback data may include text data and / or voice data, for example. By prompting the feedback data, the user can be prompted to supplement the missing items. Then, the supplemented data to be processed can be processed again according to the language model, and the above steps are repeated until the target subject words and the target category label are determined, thereby realizing intelligent input and improving user experience.
[0044] In these optional implementations, intelligent input of target subject words and target category labels based on language models can be achieved. In addition, in some cases, such as when the environmental noise is large, the target subject words and target category labels cannot be identified based on the data to be processed. At this time, the user interface for manual input can be switched back to achieve manual input of target subject words and target category labels, which can ensure smooth reception of target subject words and target category labels.
[0045] S120, expanding the target keyword to obtain an expanded keyword.
[0046] In the disclosed embodiment, the expanded subject words may include the target subject words. The target subject words may be expanded based on the expansion algorithm of existing synonyms and / or synonyms; the expanded subject words may be determined based on the expansion results and the target subject words. Since the same target object can be described in multiple ways, the recall rate of data related to the target object can be improved by expanding the target subject words with synonyms.
[0047] S130. Determine target data that matches the expanded keyword and the target category label from the preset data-label library; wherein the label corresponding to the data in the preset data-label belongs to a general category label.
[0048] In the disclosed embodiment, the preset data-label library may include a large number of data pairs and corresponding labels. Among them, each data can be labeled in advance according to the general category label based on the existing classification model to obtain the label corresponding to each data for building the preset data-label library. In order to ensure the query performance of the data, the preset data-label library can be considered as an offline database, such as a Hive database, a distributed file system (Hadoop Distributed File System, HDFS) database, etc.
[0049] After obtaining the expanded subject words, the data determination device can index the target data containing the expanded subject words and at least one label belonging to the target category label from the preset data-label library. Exemplarily, taking the Hive database as an example, the data determination device can generate a structured query language (SQL) according to the expanded subject words and the target category label, and convert the SQL into an indexing task for execution; so as to index the target data containing the expanded subject words and at least one label belonging to the target category label from the data-label library; finally, the hit target data can be exported.
[0050] For example, Figure 2 A block diagram of a data determination method provided by an embodiment of the present disclosure. Figure 2 , the data determination method may include:
[0051] Receive target keywords and target category labels by receiving manual input from the user through a user interface, and / or process the data to be processed through a language model to determine the target keywords and target category labels; expand the target keywords to obtain expanded keywords; generate query SQL based on the expanded keywords and target category labels; query from a preset data-label library based on the SQL, and output the query results (i.e., target data).
[0052] See again Figure 2 The process of determining the target subject words and target category labels through a language model may include: inputting the data to be processed into the language model to obtain the processing result of the language model; judging whether there are missing items in the processing result; if so, generating feedback data through the language model for prompting; if not, it is considered that the target subject words and target category labels are determined, and subsequent operations can be continued.
[0053] In the disclosed embodiment, a target subject word and a target category label are received; wherein the target category label is included in a preset general category label; the target subject word is expanded to obtain an expanded subject word; and target data matching the expanded subject word and the target category label is determined from a preset data-label library; wherein the label corresponding to the data in the preset data-label belongs to a general category label. By presetting a general category label that is common to any subject word, the reuse of category labels can be achieved. By pre-labeling the source data according to the general category label in advance to construct a data-label library, it is possible to determine the data that hits the target subject word and the target category label from the data-label, which has good versatility.
[0054] The various optional schemes in the data determination method provided in the embodiment of the present disclosure and the above embodiment can be combined. The data determination method provided in this embodiment describes the expansion process of the target subject word in detail. By expanding the data of the hit subject word according to the attribute value of at least one first preset dimension attribute, it is possible to extract an expanded subject word similar to the target subject word from the expanded data, thereby achieving the expansion of the target subject word.
[0055] Figure 3 The following is a flow chart of determining an expanded keyword in a data determination method provided by an embodiment of the present disclosure. Figure 3 As shown, in the data determination method provided in this embodiment, the target keyword is expanded to obtain the expanded keyword, which may include:
[0056] S310: Determine candidate data matching the target keyword from a preset data-tag library.
[0057] Among them, the candidate data containing the target keyword can be first queried from the preset data-label library. It is understandable that since the same target object has multiple description methods, the candidate data determined based on the target keyword can usually only cover a part of the data.
[0058] S320: Determine a first attribute value of a first preset dimensional attribute of the candidate data.
[0059] Each piece of data in the preset data-tag library may contain attributes of multiple dimensions. For example, when the data is a comment under an article, the data may at least contain attributes of dimensions such as the publisher, the published content, and the subject being commented on (ie, the article).
[0060] Among them, for different types of data, at least one first dimension attribute can be set in advance from the attributes contained in the data. Furthermore, after obtaining the candidate data, at least one first preset dimension attribute of the candidate data can be determined according to the type of the candidate data; and then the first attribute value of the candidate data in the first preset dimension attribute can be obtained.
[0061] S330: Determine, from the preset data-label library, the extended data consistent with the first attribute value.
[0062] If the first preset dimension attribute of other data in the preset data-label library has the same first attribute value as the first dimension data of the candidate data, the corresponding other data can be used as the expanded data. For example, when the data is a comment under an article, the data can be expanded according to the subject (i.e., the article) associated with the comment, so that other comments under the same article can be used as the expanded data.
[0063] Since other data with the same first attribute value have a high probability of using other keywords to describe the same object as the target keyword, other data with the same first attribute value can be used as extended data to extract other keywords.
[0064] S340: Determine the expanded keywords related to the target keyword based on the candidate data and the expanded data.
[0065] Among them, based on the expansion algorithm of existing synonyms and / or synonyms, other subject words similar to the target subject word can be selected from the candidate data and the expansion data to achieve the expansion of the target subject word. For example, the word similar to the target subject word can be directly selected from the candidate data and the expansion data by global keyword extraction. This expansion method can better select the head similar words with higher word frequency.
[0066] Figure 4 A schematic diagram of a block diagram for determining expanded keywords in a data determination method provided in an embodiment of the present disclosure. Figure 4 The expansion method in can well select medium and long tail words that are similar to the target keyword and have low word frequency. Figure 4 In some optional implementations, determining the expanded keywords related to the target keyword based on the candidate data and the expanded data may include:
[0067] A word co-occurrence graph is constructed based on the candidate data and the expanded data; a sub-graph is constructed based on the nodes of the target subject word in the word co-occurrence graph; and the expanded subject words related to the target subject word are determined based on the words of each node in the sub-graph.
[0068] Figure 4 In the method, after obtaining candidate data that hits the target keyword, data expansion can be performed according to the first attribute value of at least one first preset dimension attribute to obtain expanded data; the candidate data and the expanded data can be segmented, and a weighted undirected word co-occurrence graph of the word dimension can be constructed. In the word co-occurrence graph, each node can represent a word, and the weight of the connection between the nodes can represent the number of co-occurrences of the two words corresponding to the nodes.
[0069] After determining the word co-occurrence graph, a sub-graph containing the word co-occurrence of the target subject word can be extracted from the word co-occurrence graph, and the target subject word in the sub-graph can be located near the core of the graph. Furthermore, related words with a high number of co-occurrences with the target subject word can be determined from the sub-graph, and other words with a co-occurrence number greater than a preset threshold with the related words can be used as similar words of the target subject word, that is, as expanded subject words.
[0070] For example, the target keyword is "diarrhea", and related words with a higher co-occurrence probability include "stomach ache", "antidiarrhea", etc. Other words that co-occur with related words more than the preset threshold include "diarrhea". At this time, "diarrhea" can be used as an expanded keyword for "diarrhea".
[0071] See again Figure 4 In some implementations, the expanded keywords and the original target keywords can be merged into new target keywords, and the keyword expansion can be repeated until a preset cutoff condition is reached (for example, the expanded keywords reach a preset number, etc.) to obtain a relatively complete expanded keyword.
[0072] In these optional implementations, since different keywords of the same target object co-occur many times with other identical words, the keyword expansion can be carried out through this co-occurrence relationship, and medium and long-tail words with low frequency that are similar to the target keyword can be well selected.
[0073] The technical solution of the embodiment of the present disclosure describes in detail the expansion process of the target subject word. By expanding the data of the hit target subject word according to the attribute value of at least one first preset dimension attribute, it is possible to extract expanded subject words similar to the target subject word from the expanded data, and the expansion of the target subject word can be achieved. The data determination method provided by the embodiment of the present disclosure and the data determination method provided by the above embodiment belong to the same public concept, and the technical details not described in detail in this embodiment can be referred to the above embodiment, and the same technical features have the same beneficial effects in this embodiment and the above embodiment.
[0074] The various optional schemes in the data determination method provided in the embodiment of the present disclosure and the above embodiment can be combined. The data determination method provided in this embodiment describes in detail the construction process of the preset data-label library. By constructing a real-time database, the incremental data in the previous time interval in the real-time database is processed at every preset time interval and stored in the preset data-label library, and the update and maintenance of the preset data-label library can be realized on the basis of supporting data determination.
[0075] Figure 5 The following is a flow chart of constructing a preset data-label library in a data determination method provided in an embodiment of the present disclosure. Figure 5As shown, in the data determination method provided in this embodiment, the process of constructing the preset data-label library may include:
[0076] S510: Classify the source data according to the general category labels to obtain the category labels of the source data.
[0077] In this embodiment, each source data may be classified according to a general category label based on an existing classification model to obtain a category label corresponding to each source data.
[0078] S520: Store the source data and the corresponding category labels in a real-time database.
[0079] Exemplarily, the source data and the category labels corresponding to the source data may be stored in a real-time database, wherein the real-time database may include an existing online database, such as a Redis database.
[0080] S530: Process the incremental data in the real-time database within the previous time interval at every preset time interval.
[0081] In this embodiment, the preset time interval can be set according to the actual application scenario, for example, it can be set to one hour, two hours, etc. Among them, the incremental data within this time interval can be processed (for example, filtered, fused, etc.) at each preset time interval, so that the data stored in the preset data-label library meets the business needs.
[0082] S540: Store the processed incremental data in a preset data-label library.
[0083] The technical solution of the embodiment of the present disclosure describes in detail the construction process of the preset data-label library. By constructing a real-time database, the incremental data in the previous time interval in the real-time database is processed at preset time intervals and stored in the preset data-label library, and the update and maintenance of the preset data-label library can be realized on the basis of supporting data determination. The data determination method provided in the embodiment of the present disclosure and the data determination method provided in the above embodiment belong to the same public concept, and the technical details not described in detail in this embodiment can be referred to the above embodiment, and the same technical features have the same beneficial effects in this embodiment and the above embodiment.
[0084] The various optional solutions in the data determination method provided in the embodiment of the present disclosure and the above embodiment can be combined. The data determination method provided in this embodiment optimizes the determination range of the target data, can lock the query range of the preset data-label library based on the second attribute word of the second preset dimension attribute, and then can determine the target data that hits the target subject word and the target category label within the query range, which can improve the data determination efficiency.
[0085] Figure 6 The following is a flow chart of a data determination method provided by an embodiment of the present disclosure. Figure 6 As shown, the data determination method provided in this embodiment includes:
[0086] S610, receiving a target subject word, a target category label, and a second attribute word of a second preset dimension attribute; wherein the target category label is included in a general category label that is common to any subject word.
[0087] In this embodiment, the second attribute word of the second preset dimensional attribute may be received in the manner of receiving the target subject word and the target category label, wherein the second preset dimensional attribute may represent the dimensional attribute used for locking the data determination range.
[0088] S621. Expand the target keyword to obtain an expanded keyword.
[0089] S622. Determine preliminary screening data consistent with the second attribute value from the preset data-label library.
[0090] There is no strict time limit for step S621 and step S622. If the second preset dimension attribute of the data in the preset data-label library is the second attribute value, the data can be determined as the preliminary screening data, thereby narrowing the scope of the target data.
[0091] S630: Determine target data that matches the expanded keywords and target category labels from the primary screening data; wherein the labels corresponding to the data in the preset data-label belong to general category labels.
[0092] After the initial screening data is determined, label data corresponding to the initial screening data can be further obtained based on the preset data-label library. Furthermore, the data determination device can determine the target data containing the expanded keyword and at least one label belonging to the target category label from the initial screening data and the label data corresponding to the initial screening data.
[0093] In addition, in some implementations, when the second attribute word of the second preset dimensional attribute is not received, the data determination device can also determine the second attribute word of the second preset dimensional attribute based on the preset correspondence between the subject word and the second attribute value to obtain preliminary screening data with a smaller range, so as to improve data determination efficiency.
[0094] The technical solution of the disclosed embodiment optimizes the determination scope of the target data, can lock the query scope of the preset data-label library based on the second attribute word of the second preset dimension attribute, and then can determine the target data that hits the target subject word and the target category label within the query scope, which can improve the data determination efficiency. The data determination method provided by the disclosed embodiment and the data determination method provided by the above embodiment belong to the same disclosed concept, and the technical details not described in detail in this embodiment can be referred to the above embodiment, and the same technical features have the same beneficial effects in this embodiment and the above embodiment.
[0095] Figure 7 The data determination device provided in this embodiment is applicable to the case of determining target data from a database.
[0096] like Figure 7 As shown, the data determination device provided by the embodiment of the present disclosure may include:
[0097] The receiving module 710 is used to receive a target keyword and a target category label; wherein the target category label is included in a preset general category label;
[0098] An expansion module 720 is used to expand the target keyword to obtain an expanded keyword;
[0099] The matching module 730 is used to determine the target data that matches the expanded subject word and the target category label from the preset data-label library; wherein the label corresponding to the data in the preset data-label belongs to a general category label.
[0100] In some optional implementations, the receiving module may be used to:
[0101] The data to be processed is processed through the language model to obtain the processing result;
[0102] In the case where there are missing items in the processing results, feedback data is generated according to the missing items in the processing results; wherein the missing items include at least one of the target subject words and the target category labels;
[0103] Prompt for feedback data.
[0104] In some optional implementations, the expansion module may be used to:
[0105] Determine candidate data matching the target keyword from the preset data-tag library;
[0106] Determine a first attribute value of a first preset dimension attribute of the candidate data;
[0107] Determine, from the preset data-label library, the extended data consistent with the first attribute value;
[0108] Based on the candidate data and the expanded data, the expanded keywords related to the target keyword are determined.
[0109] In some optional implementations, the expansion module may be used to:
[0110] Construct a word co-occurrence graph based on candidate data and augmented data;
[0111] Construct a subgraph based on the nodes of the target topic words in the word co-occurrence graph;
[0112] According to the words of each node in the subgraph, the expanded keywords related to the target keyword are determined.
[0113] In some optional implementations, the data determination device further includes:
[0114] A building block for building a preset data-label library based on the following process:
[0115] Classify the source data according to the general category labels to obtain the category labels of the source data;
[0116] Store the source data and corresponding category labels in a real-time database;
[0117] At every preset time interval, the incremental data in the real-time database within the previous time interval is processed;
[0118] The processed incremental data is stored in the preset data-label library.
[0119] In some optional implementations, the receiving module may also be used to receive a second attribute word of a second preset dimension attribute;
[0120] Correspondingly, the matching module can also be used to determine the preliminary screening data consistent with the second attribute value from the preset data-label library; and determine the target data matching the expanded subject word and the target category label from the preliminary screening data.
[0121] In some optional implementations, the general category label is updated based on an update operation.
[0122] The data determination device provided in the embodiments of the present disclosure can execute the data determination method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.
[0123] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized.
[0124] For example, Figure 8 This is a schematic diagram of the structure of a data determination device provided by an embodiment of the present disclosure. Figure 8 , the data determination device may include a platform layer, a feature center layer and an infrastructure layer. The platform layer may support interaction with users, for example, the platform layer may provide users with a user interface for inputting target keywords and target category labels. The feature center layer may be used to process related services of category labels, for example, general category labels may be updated, etc.; the category labels of the feature center layer may include single-dimensional general category labels, and multi-dimensional general category labels. The infrastructure layer may provide various algorithmic capabilities required in the data determination method, for example, it may provide the processing capability of a language model to process the input data to be processed to extract the target keywords; it may also provide an expansion algorithm to expand the target keywords; it may also provide the processing capability of a classification model to classify each data according to the general category label, and construct a preset data-label library, etc.
[0125] based on Figure 8 The process of implementing data determination by the device may include: constructing a real-time database and a preset data-label library through a construction module of the infrastructure layer; receiving target keywords and target category labels through a receiving module in the platform layer; wherein the target category labels are included in preset general category labels; expanding the target keywords through an expansion module of the infrastructure layer; and determining target data matching the expanded keywords and target category labels from the preset data-label library through a matching module of the infrastructure layer.
[0126] In addition, the specific names of the functional units are only for the convenience of distinguishing each other and are not used to limit the protection scope of the embodiments of the present disclosure.
[0127] Reference below Fig. 9 , which shows an electronic device (eg, Fig. 9 The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0128] like Fig. 9As shown, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 to a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.
[0129] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although Fig. 9 The electronic device 900 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.
[0130] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the data determination method of the embodiment of the present disclosure are executed.
[0131] The electronic device provided by the embodiment of the present disclosure and the data determination method provided by the above embodiment belong to the same disclosed concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.
[0132] An embodiment of the present disclosure provides a computer storage medium on which a computer program is stored. When the program is executed by a processor, the data determination method provided by the above embodiment is implemented.
[0133] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or a flash memory (FLASH), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0134] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0135] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
[0136] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:
[0137] Receive a target subject word and a target category label; wherein the target category label is included in a general category label common to any subject word; expand the target subject word to obtain an expanded subject word; determine target data that matches the expanded subject word and the target category label from a preset data-label library; wherein the label corresponding to the data in the preset data-label belongs to a general category label.
[0138] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0139] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0140] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the names of the units and modules do not, in some cases, limit the units and modules themselves.
[0141] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Parts (ASSP), System on Chip (SOC), Complex Programmable Logic Device (CPLD), etc.
[0142] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0143] According to one or more embodiments of the present disclosure, a data determination method is provided, the method comprising:
[0144] Receive a target subject word and a target category label; wherein the target category label is included in a general category label common to any subject word;
[0145] Expanding the target keyword to obtain an expanded keyword;
[0146] From the preset data-label library, determine the target data that matches the expanded keyword and the target category label; wherein the label corresponding to the data in the preset data-label belongs to the general category label.
[0147] According to one or more embodiments of the present disclosure, a data determination method is provided, further comprising:
[0148] In some optional implementations, receiving a target topic word and a target category label includes:
[0149] The data to be processed is processed through the language model to obtain the processing result;
[0150] In the case where there are missing items in the processing result, generating feedback data according to the missing items in the processing result; wherein the missing items include at least one of the target subject word and the target category label;
[0151] Prompt the feedback data.
[0152] According to one or more embodiments of the present disclosure, a data determination method is provided, further comprising:
[0153] In some optional implementations, the expanding the target keyword to obtain the expanded keyword includes:
[0154] Determining candidate data matching the target keyword from the preset data-tag library;
[0155] Determine a first attribute value of a first preset dimension attribute of the candidate data;
[0156] Determining, from the preset data-label library, extended data consistent with the first attribute value;
[0157] An expanded keyword related to the target keyword is determined according to the candidate data and the expanded data.
[0158] According to one or more embodiments of the present disclosure, a data determination method is provided, further comprising:
[0159] In some optional implementations, determining, based on the candidate data and the expanded data, an expanded keyword related to the target keyword includes:
[0160] Constructing a word co-occurrence graph according to the candidate data and the expanded data;
[0161] Constructing a subgraph according to the nodes of the target topic words in the word co-occurrence graph;
[0162] According to the words of each node in the subgraph, an expanded keyword related to the target keyword is determined.
[0163] According to one or more embodiments of the present disclosure, a data determination method is provided, further comprising:
[0164] In some optional implementations, the process of constructing the preset data-label library includes:
[0165] Classify the source data according to the general category label to obtain the category label of the source data;
[0166] Storing the source data and corresponding category labels in a real-time database;
[0167] At every preset time interval, processing the incremental data in the real-time database within a previous time interval;
[0168] The processed incremental data is stored in the preset data-label library.
[0169] According to one or more embodiments of the present disclosure, a data determination method is provided, further comprising:
[0170] In some optional implementations, receiving a second attribute word of a second preset dimension attribute;
[0171] Determining preliminary screening data consistent with the second attribute value from the preset data-label library;
[0172] From the primary screening data, target data matching the expanded keyword and the target category label is determined.
[0173] According to one or more embodiments of the present disclosure, a data determination method is provided, further comprising:
[0174] In some optional implementations, the general category tag is updated based on an update operation.
[0175] According to one or more embodiments of the present disclosure, a data determination device is provided, the device comprising:
[0176] A receiving module, used for receiving a target subject word and a target category label; wherein the target category label is included in a preset general category label;
[0177] An expansion module, used for expanding the target keyword to obtain an expanded keyword;
[0178] A matching module is used to determine target data that matches the expanded keyword and the target category label from a preset data-label library; wherein the label corresponding to the data in the preset data-label belongs to the general category label.
[0179] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.
[0180] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.
[0181] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.
Claims
1. A data determination method, characterized in that: include: Receiving target topic words and target category labels; The target category label is included in the general category label common to any subject word; Expanding the target keyword to obtain an expanded keyword; From the preset data-label library, target data matching the expanded keyword and the target category label is determined; wherein the label corresponding to the data in the preset data-label belongs to the general category label.
2. The method according to claim 1, characterized in that The receiving of target subject words and target category labels includes: The data to be processed is processed through the language model to obtain the processing result; In the case where there are missing items in the processing result, generating feedback data according to the missing items in the processing result; wherein the missing items include at least one of the target subject word and the target category label; Prompt the feedback data.
3. The method according to claim 1, characterized in that The step of expanding the target keyword to obtain an expanded keyword includes: Determining candidate data matching the target keyword from the preset data-tag library; Determine a first attribute value of a first preset dimension attribute of the candidate data; Determining, from the preset data-label library, extended data consistent with the first attribute value; An expanded keyword related to the target keyword is determined according to the candidate data and the expanded data.
4. The method according to claim 3, characterized in that The step of determining, based on the candidate data and the expanded data, an expanded keyword related to the target keyword comprises: Constructing a word co-occurrence graph according to the candidate data and the expanded data; Constructing a subgraph according to the nodes of the target topic words in the word co-occurrence graph; According to the words of each node in the subgraph, an expanded keyword related to the target keyword is determined.
5. The method according to claim 1, characterized in that The process of constructing the preset data-label library includes: Classify the source data according to the general category label to obtain the category label of the source data; Storing the source data and corresponding category labels in a real-time database; At every preset time interval, processing the incremental data in the real-time database within a previous time interval; The processed incremental data is stored in the preset data-label library.
6. The method according to claim 1, characterized in that Also includes: receiving a second attribute word of a second preset dimension attribute; Determining preliminary screening data consistent with the second attribute value from the preset data-label library; From the primary screening data, target data matching the expanded keyword and the target category label is determined.
7. The method according to any one of claims 1 to 6, characterized in that: The general category label is updated based on an update operation.
8. A data determination device, characterized in that: include: A receiving module, used for receiving target subject words and target category labels; The target category label is included in the preset general category label; An expansion module, used for expanding the target keyword to obtain an expanded keyword; A matching module is used to determine target data that matches the expanded keyword and the target category label from a preset data-label library; wherein the label corresponding to the data in the preset data-label belongs to the general category label.
9. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the data determination method according to any one of claims 1 to 7.
10. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to perform the data determination method according to any one of claims 1 to 7 when executed by a computer processor.