Determination method and device of reasoning data, equipment and storage medium

Abstract

The invention provides an inference data determination method and device, equipment and a storage medium, and relates to the technical field of data processing, in particular to the technical fields of artificial intelligence, big data, large models and the like. According to the specific implementation scheme, a first webpage set and a second webpage set are obtained, the first webpage set comprises at least one first webpage with the overall webpage data quality meeting the webpage quality requirement, and the second webpage set comprises at least one second webpage with the overall webpage data quality meeting the webpage quality requirement; the second webpage set comprises at least one second webpage of which the data quality of at least part of webpage contents meets the data requirement; and at least utilizing the first analysis data of the first webpage and the second analysis data of the second webpage to obtain a multimedia reasoning database for model training.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to technical fields such as artificial intelligence, big data, and big models. Background Art

[0002] In large-scale multimodal applications, high-quality multimodal reasoning data is severely scarce, which limits the reasoning ability of large models. Although the multimodal data in existing web pages is rich in content, the data quality of existing web pages is uneven, so it is impossible to achieve efficient use of web page data. For example, the cleaning, processing and mining of web page data are the core difficulties at present. Summary of the invention

[0003] The present disclosure provides a method, apparatus, device and storage medium for determining inference data.

[0004] According to one aspect of the present disclosure, a method for determining inference data is provided, comprising:

[0005] Acquire a first web page set and a second web page set, wherein the first web page set includes at least one first web page whose overall data quality meets the page quality requirement, and the second web page set includes at least one second web page whose data quality of at least part of the web page content meets the data requirement;

[0006] A multimedia reasoning database for model training is obtained by at least using the first parsed data of the first web page and the second parsed data of the second web page.

[0007] According to another aspect of the present disclosure, there is provided a device for determining inference data, comprising:

[0008] A data preprocessing unit, configured to obtain a first web page set and a second web page set, wherein the first web page set includes at least one first web page whose overall data quality meets the page quality requirement, and the second web page set includes at least one second web page whose data quality of at least part of the web page content meets the data requirement;

[0009] The database construction unit is used to obtain a multimedia reasoning database for model training by at least using the first parsed data of the first web page and the second parsed data of the second web page.

[0010] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0011] at least one processor; and

[0012] a memory communicatively connected to the at least one processor; wherein,

[0013] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0014] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0015] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0016] The disclosed solution can utilize different screening requirements (such as page quality requirements and data requirements) to determine the required web pages (such as the first web page and the second web page), and then utilize the first parsed data of the first web page and the second parsed data of the second web page to construct inference data (such as a multimedia inference database) that can be used for model training. In this way, high-quality inference data under multimodality can be efficiently mined from massive web pages, thereby improving the richness of the inference data and laying the foundation for improving the reasoning ability of the model in the subsequent model training process.

[0017] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0019] Figure 1 This is a schematic flow chart of a method for determining inference data according to an embodiment of the present application. Figure 1 ;

[0020] Figure 2 This is a schematic flow chart of a method for determining inference data according to an embodiment of the present application. Figure 2 ;

[0021] Figure 3 This is a schematic flow chart of a method for determining inference data according to an embodiment of the present application. Figure 3 ;

[0022] Figure 4 is a flowchart of a method for determining inference data in a specific example according to an embodiment of the present application;

[0023] Figure 5is a schematic diagram of the structure of a device for determining inference data according to an embodiment of the present application;

[0024] Figure 6 The present invention is a block diagram of an electronic device for implementing the method for determining inference data according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0026] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this article refer to multiple similar technical terms and distinguish them. They do not mean to limit the order or to limit them to only two. For example, the first feature and the second feature refer to two types / two features. The first feature can be one or more, and the second feature can also be one or more.

[0027] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the specific embodiments below. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present disclosure.

[0028] The disclosed solution proposes a method for determining inference data to obtain high-quality inference data.

[0029] Figure 1 This is a schematic flow chart of a method for determining inference data according to an embodiment of the present application. Figure 1 The method may be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices.

[0030] Further, the method includes at least part of the following contents. Figure 1 As shown, including:

[0031] Step S101: Acquire a first web page set and a second web page set.

[0032] Here, the first web page set includes at least one first web page whose overall data quality meets the page quality requirements, and the second web page set includes at least one second web page whose data quality of at least part of the web page content meets the data requirements.

[0033] For example, in one example, the overall data quality of the first web pages included in the first web page set meets the page quality requirements, and the overall data quality of the second web pages included in the second web page set does not meet the page quality requirements, but the data quality of some page contents meets the data requirements.

[0034] Step S102: obtaining a multimedia inference database for model training by at least using the first parsed data of the first web page and the second parsed data of the second web page.

[0035] That is, in this example, the first parsed data of each first web page in the first web page set and the second parsed data of each second web page in the second web page set can be used to obtain a multimedia reasoning database that can be used for model training.

[0036] In this way, the disclosed solution can utilize different screening requirements (such as page quality requirements and data requirements) to determine the required web pages (such as the first web page and the second web page), and then utilize the first parsed data of the first web page and the second parsed data of the second web page to construct inference data that can be used for model training (such as a multimedia inference database). In this way, high-quality inference data under multimodality can be efficiently mined from massive web pages, thereby improving the richness of the inference data and laying the foundation for improving the reasoning ability of the model in the subsequent model training process.

[0037] Furthermore, since the web pages involved in the disclosed solution (such as the first web page and the second web page) can be web pages in any field and expressed in any language, the disclosed solution can obtain reasoning data in different languages ​​(such as Chinese, English, etc.) and different modalities from web pages in different fields. At this time, if the obtained reasoning data is used for model training, the subsequently trained model can have the reasoning ability to adapt to various languages ​​and various fields. Therefore, the disclosed solution has strong applicability and practicality.

[0038] It should be noted that, in a specific example, the disclosed solution can also construct a multimedia reasoning database for model training using the first parsed data of at least one first web page when a first web page set is obtained, or construct a multimedia reasoning database for model training using the second parsed data of at least one second web page when a second web page set is obtained. The disclosed solution does not impose any specific restrictions on this.

[0039] Furthermore, in one example, the multimedia reasoning database may contain reasoning data in multiple modes (e.g., data formats such as images, text, and / or voice), and in this case, it may also be specifically referred to as a multimodal reasoning database. For example, the multimedia database includes but is not limited to at least one of the following: text data, image data, audio data, video data, etc., wherein the text data may also be text data in different languages, such as Chinese text data, English text data, etc. The disclosed solution does not limit this.

[0040] Figure 2 This is a schematic flow chart of a method for determining inference data according to an embodiment of the present application. Figure 2 The method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices. It can be understood that the above Figure 1 The relevant contents of the method shown can also be applied to this example, and the relevant contents will not be described in detail in this example.

[0041] Further, the method includes at least part of the following contents. Figure 2 As shown, including:

[0042] Step S201: obtaining target parsing data of each preset web page among a plurality of preset web pages.

[0043] Step S202: Based on the target parsed data of the preset web page, the overall data quality of the preset web page is evaluated to obtain a page evaluation result of the preset web page.

[0044] Step S203: taking a preset web page whose page evaluation result meets the page quality requirement as a first web page to obtain a first web page set.

[0045] Here, the first parsed data of the first webpage is the target parsed data. Further, the page evaluation result may be specifically a page evaluation score, and at this time, the page quality requirement may be specifically that the page evaluation score is greater than or equal to a first threshold.

[0046] Step S204: Acquire a second web page set.

[0047] Here, the second web page set includes at least one second web page whose data quality of at least a part of the web page content meets the data requirement.

[0048] It is understandable that in actual applications, the step of obtaining the second web page set (ie, step S204) can be interchanged with any step of obtaining the first web page set (ie, steps S201 to S203), or both can be performed simultaneously, and the disclosed solution does not limit this.

[0049] Step S205: obtaining a multimedia inference database for model training by at least using the first parsed data of the first web page and the second parsed data of the second web page.

[0050] That is to say, in this example, first, according to the obtained target parsed data of the preset web page, the overall data quality of the preset web page is evaluated to obtain the page evaluation result of the preset web page (such as the page evaluation score), so as to obtain the page evaluation result of each preset web page; secondly, from all the preset web pages, the preset web page whose page evaluation result meets the page quality requirement is selected as the first web page. For example, in one example, the preset web page whose page evaluation score is greater than or equal to the first threshold is selected as the first web page to obtain the first web page set. At this time, the first parsed data of the first web page in the first web page set can be specifically the target parsed data; finally, the first parsed data of at least one first web page and the obtained second parsed data of the second web page are used to construct a multimedia reasoning database.

[0051] In this way, the disclosed scheme provides a method for obtaining a first set of web pages, which can quickly and efficiently obtain the first web page (i.e., the first set of web pages) that meets the page quality requirements. In this way, it provides data support for the subsequent rapid construction of a multimedia reasoning database, and further lays the foundation for subsequent training to obtain a reasoning model with stronger reasoning capabilities.

[0052] Further, in a specific example, the following method can be used to quickly obtain the page evaluation result. Specifically, the above-mentioned target parsing data based on the preset web page is used to evaluate the overall data quality of the preset web page to obtain the page evaluation result of the preset web page (for example, step S202), which specifically includes:

[0053] The target parsed data of the preset web page is input into the data quality model to obtain the page evaluation result of the preset web page.

[0054] Here, the data quality model is used to predict the data quality of the entire web page and output an evaluation result.

[0055] For example, in one example, the data quality model can output a page evaluation score. At this time, the first web page that meets the page quality requirements can be quickly screened based on the page evaluation score output by the data quality model.

[0056] Alternatively, in one example, the data quality model may be specifically a classification model. In this case, the output evaluation results may be specifically classification results. For example, classification results of category 1 or category 2 may be obtained, where category 1 may indicate that the page evaluation results meet the page quality requirements, and category 2 may indicate that the page evaluation results do not meet the page quality requirements.

[0057] It should be noted that the disclosed solution does not impose any specific restrictions on the data quality model, and the model can be selected or trained based on actual needs.

[0058] In this way, the disclosed scheme can utilize the data quality model to evaluate the overall data quality of each preset web page, and thus can quickly and accurately obtain the page evaluation results of each preset web page, thereby providing strong support for obtaining the first web page and the second web page, and thus laying the foundation for the subsequent construction of a high-quality multimedia reasoning database.

[0059] Further, in a specific example, the target parsing data of the preset webpage can be obtained in the following manner; specifically, the above-mentioned obtaining the target parsing data of each preset webpage in the plurality of preset webpages (for example, step S201) specifically includes:

[0060] Step S201 - 1 : Acquire initial webpage data of each preset webpage among the plurality of preset webpages.

[0061] For example, in one example, the initial page data may specifically be HyperText Markup Language (HTML) data.

[0062] Step S201 - 2 : parsing the initial web page data of the preset web pages based on the parsing method matching the initial web page data to obtain target parsed data of each preset web page.

[0063] For instance, in one example, the initial web page data may contain a specified tag (for example, "table", "audio", "code" or "video", etc.). At this time, the parsing method corresponding to the initial web page data can be determined based on the specified tag. For example, if the initial web page data contains the specified tag "table", the parsing method corresponding to the specified tag "table" is used to parse the initial web page data. For example, the data where the specified tag "table" is located can be extracted to obtain the target parsed data of the preset web page.

[0064] Alternatively, in another example, the initial web page data may not include the specified tag. In this case, the initial web page data may be parsed based on a general parsing method to obtain target parsed data.

[0065] In this way, the disclosed scheme provides a refined scheme for obtaining target parsed data, which can provide data support for subsequent scientific and reasonable evaluation of the overall data quality of preset web pages, thereby laying the foundation for building a high-quality multimedia reasoning database.

[0066] Figure 3 This is a schematic flow chart of a method for determining inference data according to an embodiment of the present application. Figure 3 The method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices. It can be understood that the above Figure 1 and Figure 2 The relevant contents of the method shown can also be applied to this example, and the relevant contents will not be described in detail in this example.

[0067] Further, the method includes at least part of the following contents. Figure 3 As shown, including:

[0068] Step S301: obtaining target parsing data of each preset web page among a plurality of preset web pages.

[0069] Step S302: Based on the target parsed data of the preset web page, the overall data quality of the preset web page is evaluated to obtain a page evaluation result of the preset web page.

[0070] It should be noted that the preset web page parsed by the special parsing method does not need to be evaluated for the overall data quality of the web page. The preset web page can be directly used as the third web page to obtain a third web page set. At this time, the third parsed data of the third web page is the target parsed data of the preset web page.

[0071] Step S303: determine whether the page evaluation result of the preset web page meets the page quality requirement, if so, proceed to step S304, otherwise, proceed to step S305.

[0072] Step S304: taking a preset web page whose page evaluation result meets the page quality requirement as a first web page to obtain a first web page set.

[0073] Here, the first parsed data of the first webpage is the target parsed data of the preset webpage.

[0074] Step S305: Determine whether there is target data that meets the data requirements in the target parsed data of the preset web page whose page evaluation result does not meet the page quality requirement. If so, proceed to step S306; otherwise, proceed to step S307.

[0075] For example, in one example, it can be specifically determined whether the contents such as paragraphs, image links, or text fragments in the target parsed data of a preset web page meet the data requirements.

[0076] Step S306: when it is determined that there is target data that meets the data requirement, the preset web page containing the target data is used as the second web page to obtain a second web page set.

[0077] That is, in this example, when it is determined that the page evaluation result of the preset webpage does not meet the page quality requirements, it can be further determined whether there is target data that meets the data requirements in the target parsed data of the preset webpage, such as determining whether there are paragraphs, image links, text fragments, etc. that meet the data requirements. If so, the preset webpage with the target data that meets the data requirements can be used as the second webpage.

[0078] Further, in this example, the second parsed data of the second webpage may specifically be target data that meets the data requirement in the target parsed data of the second webpage.

[0079] Step S307: Abandon the preset web page.

[0080] In this way, by using the above steps S302 to S307, the evaluation and classification of all the preset web pages can be completed, the first web page set and the second web page set are obtained, and the process proceeds to step S308.

[0081] Step S308: Obtain a multimedia inference database for model training by at least using the first parsed data of the first web page and the second parsed data of the second web page.

[0082] For example, in one example, first, according to the obtained target parsing data of the preset web page, the overall data quality of the preset web page is evaluated to obtain the page evaluation score of each preset web page; secondly, it is determined whether the page evaluation score of the preset web page is greater than or equal to the first threshold value. If the page evaluation score of the preset web page is greater than or equal to the first threshold value, the preset web page is used as the first web page to obtain the first web page set; otherwise, that is, when the page evaluation score of the preset web page is less than the first threshold value, it is determined whether there is target data that meets the data requirements in the target parsing data of the preset web page. If so, the preset web page is used as the second web page to obtain the second web page set, otherwise the preset web page is discarded. At this point, the evaluation and classification of all preset web pages are completed; finally, the multimedia reasoning database can be constructed using the first parsing data of at least one first web page and the second parsing data of at least one second web page.

[0083] In this way, the disclosed scheme provides a specific scheme that can evaluate and classify multiple preset web pages, so that a first web page set and a second web page set can be efficiently obtained, and then the first parsed data of the first web page and the second parsed data of the second web page can be used to quickly construct a multimedia reasoning database. In this way, high-quality reasoning data can be effectively obtained, thereby laying the foundation for improving the reasoning ability of the model in the subsequent model training process.

[0084] Further, in a specific example, the multimedia inference database can be obtained in the following manner; specifically, the above-mentioned method of obtaining the multimedia inference database for model training (for example, step S102, step S205, step S308) by at least using the first parsed data of the first webpage and the second parsed data of the second webpage, specifically includes:

[0085] Step S308 - 1 : Acquire multimedia data corresponding to first parsed data of a first webpage, and acquire multimedia data corresponding to second parsed data of a second webpage, to obtain a multimedia data set.

[0086] For example, in one example, multimedia data is downloaded from the parsed data of each web page to obtain multimedia data corresponding to each web page, thereby obtaining a multimedia data set.

[0087] Step S308-2: label each multimedia data in the multimedia data set to obtain multimedia data with labeling information.

[0088] Here, the multimedia data with labeled information may also be referred to as inference data.

[0089] Step S308-3: obtaining a multimedia reasoning database for model training based at least on the multimedia data with annotation information.

[0090] It should be pointed out that, in one example, multimedia data with annotation information can constitute a multimedia reasoning database; while multimedia data that is not annotated or fails to be annotated can constitute general data. Alternatively, the annotation information includes a preset label and a score (for example, the probability of the data corresponding to the preset label). In this case, multimedia data with a score greater than or equal to the preset score can be used to construct a multimedia reasoning database; and multimedia data with a score less than the preset score can be classified into a general database. In this way, the disclosed solution can construct a multimedia reasoning database for model training on the one hand, and a general database on the other hand.

[0091] In this way, the disclosed scheme can annotate the obtained multimedia data, and then use the multimedia data with annotated information to construct a multimedia reasoning database. In this way, the quality and diversity of the multimedia reasoning data are improved, laying the foundation for improving the reasoning ability of the model in the subsequent model training process.

[0092] Further, in a specific example, the labeling information of each multimedia data may be obtained in the following manner; specifically, the labeling of each multimedia data in the multimedia data set described above to obtain multimedia data with labeling information (for example, step S308-2) specifically includes at least one of the following:

[0093] Labeling method 1: input the image data in the multimedia data into the image reasoning labeling model to obtain the labeling information of the image data in the multimedia data.

[0094] Annotation method 2: Input the text data in the multimedia data into the text reasoning annotation model to obtain the annotation information of the text data in the multimedia data.

[0095] In this way, the disclosed solution can use different annotation models according to different data types in the multimedia data to quickly annotate the corresponding data in the multimedia data, so as to quickly obtain multimedia data with annotation information, thus laying the foundation for the subsequent construction of a high-quality multimedia reasoning database.

[0096] In a specific example of the disclosed solution, after constructing the multimedia database, it also includes obtaining training samples for training the above-mentioned annotation model (such as the image reasoning annotation model or the text reasoning annotation model) (for example, step S309), which may specifically include:

[0097] Step S309 - 1 : Based on obtaining the first web page and / or the second web page of the multimedia reasoning database, obtain a plurality of initial website sites.

[0098] For example, in one example, a plurality of initial websites in the form of "XXX.com" may be obtained based on the first web page and / or the second web page used to construct the multimedia reasoning database.

[0099] Here, it should be noted that the website described in this example may refer to a website, including a set of web pages provided by the same domain and maintained by a single organization. In other words, a website may include multiple web pages, which may be interconnected via hyperlinks.

[0100] Step S309 - 2 : Based on the multiple initial websites, determine at least one target page that meets the page requirement.

[0101] That is to say, in this example, at least one target page that meets the page requirements can be determined based on the multiple web pages contained in the initial website. For example, in one example, the web pages that can be clicked and opened in the initial website can be annotated by the large model, and the web pages annotated with information of "inference" are used as target pages (also called target web pages). In this way, multiple target pages can be obtained.

[0102] It should be noted that the large model can mark pages that contain sample data that can be used for model training as "inference" pages.

[0103] Step S309 - 3 : using the at least one target page, obtaining image samples for training the image reasoning annotation model and / or text samples for training the text reasoning annotation model.

[0104] That is, in one example, first, based on the first web page and / or the second web page of the multimedia reasoning database, multiple initial websites are obtained by statistics; second, for each initial website, if the web pages that can be clicked and opened under the initial website can be annotated by the large model, and the web pages with the annotation information of "reasoning" are used as the target page, at least one target page can be obtained; finally, at least one of the following samples is obtained from each target page: image sample, text sample. In this way, data support is provided for the subsequent improvement of the annotation accuracy of the reasoning annotation model.

[0105] In this way, the disclosed solution provides a method for efficiently obtaining training samples for training image reasoning annotation models and / or text reasoning annotation models, thereby providing data support for subsequent improvement of the annotation quality of the reasoning annotation models, and further laying the foundation for improving the richness and data quality of reasoning data in multimedia reasoning databases.

[0106] Furthermore, in a specific example, after obtaining the image sample and / or the text sample, the method further includes:

[0107] (1) Training the model used in labeling method 1 (i.e., the image reasoning labeling model). Specifically, at least using the image samples obtained for training the image reasoning labeling model, the image reasoning labeling model is trained to obtain the image reasoning labeling model after training.

[0108] And / or, (2) training the model (text inference annotation model) used in annotation method 2, specifically, performing model training on the text inference annotation model using at least the text samples obtained for training the text inference annotation model to obtain the text inference annotation model after training.

[0109] For example, in one example, the trained image reasoning annotation model and text reasoning annotation model can further annotate unlabeled multimedia data, such as multimedia data in a general database, to further enrich the data volume of the multimedia reasoning database.

[0110] In this way, the disclosed solution can use the obtained image samples to train the image reasoning annotation model, and / or use the obtained text samples to train the text reasoning annotation model. In this way, the annotation accuracy of the reasoning annotation model is effectively improved, laying the foundation for the subsequent improvement of the data volume, data quality and data richness of the constructed multimedia reasoning database.

[0111] Further, in a specific example, at least one target page may be obtained in the following manner; specifically, the above-mentioned determination of at least one target page that meets the page requirements based on the multiple initial websites (e.g., step S309-2) specifically includes:

[0112] Step S309-2-1: Determine at least one target network site that meets preset site requirements from the multiple initial network sites.

[0113] For example, in one example, an initial website whose number of web pages that can be clicked and opened exceeds the second threshold value can be used as the target website. Alternatively, in another example, an initial website whose number of web pages that can be clicked and opened exceeds the second threshold value and whose number of web pages marked as "inference" among the clicked and opened web pages exceeds the third threshold value can be used as the target website. In other words, an initial website that meets the above two conditions can be used as the target website.

[0114] It should be noted that the above is only an exemplary description. In actual applications, the preset site requirements may be determined based on actual scenario needs, and the present disclosure does not limit this.

[0115] Step S309-2-2: Select at least one target page that meets the page requirement from the at least one target website.

[0116] For instance, in one example, when the number of web pages that can be clicked to open under the initial website exceeds 100, and the number of web pages that are "inference" web pages exceeds 10%, the initial website can be used as the target website to obtain at least one target website; and then 30% of the web pages (hereinafter referred to as sampled web pages) are sampled from each target website, and the large model is used to determine whether each sampled web page is an "inference" web page, and the sampled web pages that are "inference" web pages are used as target pages to obtain at least one target page.

[0117] In this way, the disclosed scheme provides a refined scheme for obtaining the target page, which is simple, efficient, and highly interpretable. This lays the foundation for the subsequent efficient acquisition of samples that can be used to train image reasoning annotation models and text reasoning annotation models, and further lays the foundation for improving the annotation accuracy of image reasoning annotation models and text reasoning annotation models, and for improving the data volume, data quality, and data richness of the constructed multimedia reasoning database.

[0118] The following is a detailed description of the disclosed scheme with reference to specific examples; the disclosed scheme provides a scheme for mining inference data. Specifically, the disclosed scheme can determine the mining direction on reasoning, and can define an inference label system according to actual needs. For example, in one example, the inference label system may include explanations, steps, authoritative arguments, mathematics, analogy arguments, etc. In this way, the multimedia data obtained by subsequent mining can be annotated based on the defined inference label system to obtain the required inference data.

[0119] Furthermore, the disclosed scheme can also produce target domain data (corresponding to the above multimedia reasoning database) and general data (compared to multimedia reasoning data, the database composed of general data can be called a general database) through processes such as data analysis, data cleaning, and model annotation for web pages; and, the disclosed scheme can also further improve the data volume, data quality, and data richness of the produced target domain data through model training, correction, and recycling. For example, the disclosed scheme can also obtain seed data (corresponding to the target page mentioned above) for training or optimizing the multimodal reasoning annotation model to obtain a text reasoning annotation model and an image reasoning annotation model, thereby improving the quantity, data quality, and richness of the reasoning data.

[0120] Specifically, Figure 4 As shown, the steps of the disclosed solution include:

[0121] Step S401: Determine whether the HTML data of the webpage (corresponding to the initial webpage data above) contains a specified tag (such as table, audio, code, video, etc.) to determine a parsing method that matches the HTML data.

[0122] Here, in one example, if it is determined that the HTML data of the web page contains a specified tag, the HTML data of the web page can be parsed using a preset special parsing method to obtain target parsed data and directly enter step S404; otherwise, a general parsing method is used to parse the HTML data of the web page to obtain target parsed data and enter step S402.

[0123] Step S402: Input the target parsed data of each web page into the data quality model to classify the target parsed data and give a quality classification label. For example, if the quality classification label of the web page indicates that it meets the high quality requirements, the web page is used as the first web page to obtain the first web page set. At this time, the parsed data of the first web page can be specifically the target parsed data. Otherwise (that is, the quality classification label of the web page indicates that it does not meet the high quality requirements), continue to determine whether the target parsed data of the web page has target data that meets the data requirements (such as legal image links, legal text links). If there is target data, the web page is used as the second web page. At this time, the target parsed data of the second web page is filtered, and the filtered target parsed data, for example, the target data that meets the data requirements is directly used as the parsed data of the second web page. Otherwise (that is, there is no target data), the preset web page is discarded, such as being put into a trash can.

[0124] Step S403: Download multimedia data (also called multimedia material) for the parsed data of the first web page and the parsed data of the second web page to obtain multimedia data (such as pictures, videos, audios, etc.) corresponding to each web page, and then obtain a multimedia corpus (corresponding to the above multimedia data set).

[0125] Step S404: Combine the image reasoning annotation model and the text reasoning annotation model to annotate the multimedia data in the multimedia corpus to obtain multimedia data with annotation information to build a multimedia reasoning library (that is, corresponding to the above multimedia reasoning database). In addition, a multimedia general library (that is, corresponding to the above general database) is also obtained.

[0126] Here, in one example, the image data in the multimedia data is annotated using a picture reasoning annotation model, and annotation information of the image data (e.g., including preset reasoning labels and scores) is obtained, and the text data in the multimedia data is annotated using a text reasoning annotation model, and annotation information of the text data (e.g., including preset reasoning labels and scores) is obtained; further, a multimedia reasoning library is constructed using image data and text data with scores greater than or equal to a preset score, and a multimedia general library is constructed using image data and text data with scores less than a preset score.

[0127] Furthermore, it is also possible to construct a text reasoning library using only text data with scores greater than or equal to a preset score, and to construct a text general library using text data with scores less than a preset score. In this way, multiple databases can be constructed based on the structure.

[0128] Step S405: Based on the first web page and the second web page of the multimedia reasoning library, multiple initial network sites are obtained by counting, and when it is determined that the total number of web pages under the initial network site exceeds 100 and more than 10% of them are determined to be reasoning web pages, the initial network site is used as a suspected reasoning site (corresponding to the above target network site) to obtain at least one suspected reasoning site.

[0129] Step S406: Sample 30% of the web pages for each suspected inference site (i.e., corresponding to the sampled web pages above), use the large model to annotate these web pages, and use the web pages labeled "inference" to obtain text samples for training the text inference annotation model, and obtain image samples for training the image inference annotation model.

[0130] Step S407: Using the obtained text samples, the text reasoning annotation model in step S404 is trained, and using the obtained image samples, the image reasoning annotation model in step S404 is trained, thereby further enhancing the diversity of the model.

[0131] Step S408: loop through steps S404 to S407 to obtain a multimedia reasoning library consisting of a large amount of high-quality multimodal reasoning data.

[0132] In this way, the disclosed solution can combine the text reasoning annotation model and the image reasoning annotation model to mine high-quality reasoning corpus under multimodality. Compared with the existing solutions, it enriches the data diversity and lays the foundation for improving the reasoning ability of the model in the subsequent model training process.

[0133] The disclosed solution also provides a device for determining inference data, such as Figure 5 As shown, including:

[0134] The data preprocessing unit 501 is used to obtain a first web page set and a second web page set, wherein the first web page set includes at least one first web page whose overall data quality meets the page quality requirement, and the second web page set includes at least one second web page whose data quality of at least part of the web page content meets the data requirement;

[0135] The database construction unit 502 is used to obtain a multimedia reasoning database for model training by at least using the first parsed data of the first web page and the second parsed data of the second web page.

[0136] In a specific example of the disclosed solution, the data preprocessing unit is further used to:

[0137] Obtaining target parsed data of each preset web page among a plurality of preset web pages;

[0138] Based on the target parsed data of the preset web page, the overall data quality of the preset web page is evaluated to obtain a page evaluation result of the preset web page;

[0139] A preset web page whose page evaluation result meets the page quality requirement is used as a first web page to obtain the first web page set; wherein the first parsed data of the first web page is the target parsed data.

[0140] In a specific example of the disclosed solution, the data preprocessing unit is specifically used to:

[0141] The target parsed data of the preset web page is input into the data quality model to obtain the page evaluation result of the preset web page, wherein the data quality model is used to predict the data quality of the entire web page and output the evaluation result.

[0142] In a specific example of the disclosed solution, the data preprocessing unit is specifically used to:

[0143] Acquiring initial webpage data of each preset webpage among the plurality of preset webpages;

[0144] Based on the parsing method matching the initial web page data, the initial web page data of the preset web page is parsed to obtain target parsed data of each preset web page.

[0145] In a specific example of the disclosed solution, the data preprocessing unit is further used to:

[0146] Determining whether there is target data that meets the data requirement in the target parsed data of the preset web page whose page evaluation result does not meet the page quality requirement;

[0147] When it is determined that there is target data that meets the data requirements, the preset web page containing the target data is used as the second web page to obtain the second web page set; wherein the second parsed data of the second web page is: the target data that meets the data requirements in the target parsed data of the second web page.

[0148] In a specific example of the disclosed solution, the database construction unit is specifically used to:

[0149] Acquire multimedia data corresponding to first parsed data of the first webpage, and acquire multimedia data corresponding to second parsed data of the second webpage, so as to obtain a multimedia data set;

[0150] Marking each multimedia data in the multimedia data set to obtain multimedia data with marking information;

[0151] At least based on the multimedia data with labeled information, a multimedia reasoning database for model training is obtained.

[0152] In a specific example of the disclosed solution, the database construction unit specifically performs at least one of the following:

[0153] Inputting image data in the multimedia data into the image reasoning annotation model to obtain annotation information of the image data in the multimedia data;

[0154] The text data in the multimedia data is input into the text reasoning annotation model to obtain the annotation information of the text data in the multimedia data.

[0155] In a specific example of the disclosed solution, the system further includes: a site processing unit; wherein the site processing unit is used to:

[0156] Based on obtaining the first web page and / or the second web page of the multimedia reasoning database, obtaining a plurality of initial network sites;

[0157] Based on the multiple initial websites, determining at least one target page that meets the page requirements;

[0158] Using the at least one target page, an image sample for training the image reasoning annotation model and / or a text sample for training the text reasoning annotation model are obtained.

[0159] In a specific example of the disclosed solution, the system further includes: a training unit; wherein the training unit is used to:

[0160] Performing model training on the image reasoning and annotation model using at least the image samples obtained for training the image reasoning and annotation model, so as to obtain the image reasoning and annotation model after training;

[0161] and / or,

[0162] The text reasoning annotation model is at least trained by using the obtained text samples for training the text reasoning annotation model to obtain the text reasoning annotation model after training.

[0163] In a specific example of the disclosed solution, the site processing unit is specifically used to:

[0164] Determining at least one target network site that meets preset site requirements from the plurality of initial network sites;

[0165] At least one target page that meets the page requirement is selected from the at least one target website.

[0166] For the description of the specific functions and examples of each unit of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0167] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0168] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0169] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0170] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0171] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0172] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the determination method of reasoning data. For example, in some embodiments, the determination method of reasoning data may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the determination method of reasoning data described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the determination method of reasoning data in any other appropriate manner (e.g., by means of firmware).

[0173] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0174] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0175] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0176] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0177] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0178] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0179] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0180] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for determining inference data, comprising: Acquire a first web page set and a second web page set, wherein the first web page set includes at least one first web page whose overall data quality meets the page quality requirement, and the second web page set includes at least one second web page whose data quality of at least part of the web page content meets the data requirement; A multimedia reasoning database for model training is obtained by at least using the first parsed data of the first web page and the second parsed data of the second web page.

2. The method according to claim 1, further comprising: Obtaining target parsed data of each preset web page among a plurality of preset web pages; Based on the target parsed data of the preset web page, the overall data quality of the preset web page is evaluated to obtain a page evaluation result of the preset web page; A preset web page whose page evaluation result meets the page quality requirement is used as a first web page to obtain the first web page set; wherein the first parsed data of the first web page is the target parsed data.

3. The method according to claim 2, wherein: The target parsing data of the preset web page is used to evaluate the overall data quality of the preset web page to obtain the page evaluation result of the preset web page, including: The target parsed data of the preset web page is input into the data quality model to obtain the page evaluation result of the preset web page, wherein the data quality model is used to predict the data quality of the entire web page and output the evaluation result.

4. The method according to claim 2, wherein: The step of obtaining target parsed data of each preset web page among the plurality of preset web pages includes: Acquiring initial webpage data of each preset webpage among the plurality of preset webpages; Based on the parsing method matching the initial web page data, the initial web page data of the preset web page is parsed to obtain target parsed data of each preset web page.

5. The method according to any one of claims 2 to 4, further comprising: Determining whether there is target data that meets the data requirement in the target parsed data of the preset web page whose page evaluation result does not meet the page quality requirement; When it is determined that there is target data that meets the data requirements, the preset web page containing the target data is used as the second web page to obtain the second web page set; wherein the second parsed data of the second web page is: the target data that meets the data requirements in the target parsed data of the second web page.

6. The method according to any one of claims 2 to 5, wherein: The method of obtaining a multimedia inference database for model training by at least using the first parsed data of the first webpage and the second parsed data of the second webpage includes: Acquire multimedia data corresponding to first parsed data of the first webpage, and acquire multimedia data corresponding to second parsed data of the second webpage, so as to obtain a multimedia data set; Marking each multimedia data in the multimedia data set to obtain multimedia data with marking information; At least based on the multimedia data with labeled information, a multimedia reasoning database for model training is obtained.

7. The method according to claim 6, wherein: The step of labeling each multimedia data in the multimedia data set to obtain multimedia data with labeling information includes at least one of the following: Inputting image data in the multimedia data into the image reasoning annotation model to obtain annotation information of the image data in the multimedia data; The text data in the multimedia data is input into the text reasoning annotation model to obtain the annotation information of the text data in the multimedia data.

8. The method according to claim 7, further comprising: Based on obtaining the first web page and / or the second web page of the multimedia reasoning database, obtaining a plurality of initial network sites; Based on the multiple initial websites, determining at least one target page that meets the page requirements; Using the at least one target page, an image sample for training the image reasoning annotation model and / or a text sample for training the text reasoning annotation model are obtained.

9. The method according to claim 8, further comprising: Performing model training on the image reasoning and annotation model using at least the image samples obtained for training the image reasoning and annotation model, so as to obtain the image reasoning and annotation model after training; and / or, The text reasoning annotation model is at least trained by using the obtained text samples for training the text reasoning annotation model to obtain the text reasoning annotation model after training.

10. The method according to claim 8, wherein: The step of determining at least one target page that meets the page requirements based on the multiple initial websites includes: Determining at least one target network site that meets preset site requirements from the plurality of initial network sites; At least one target page that meets the page requirement is selected from the at least one target website.

11. A device for determining inference data, comprising: A data preprocessing unit, configured to obtain a first web page set and a second web page set, wherein the first web page set includes at least one first web page whose overall data quality meets the page quality requirement, and the second web page set includes at least one second web page whose data quality of at least part of the web page content meets the data requirement; The database construction unit is used to obtain a multimedia reasoning database for model training by at least using the first parsed data of the first web page and the second parsed data of the second web page.

12. The device according to claim 11, wherein The data preprocessing unit is further used for: Obtaining target parsed data of each preset web page among a plurality of preset web pages; Based on the target parsed data of the preset web page, the overall data quality of the preset web page is evaluated to obtain a page evaluation result of the preset web page; A preset web page whose page evaluation result meets the page quality requirement is used as a first web page to obtain the first web page set; wherein the first parsed data of the first web page is the target parsed data.

13. The device according to claim 12, wherein: The data preprocessing unit is specifically used for: The target parsed data of the preset web page is input into the data quality model to obtain the page evaluation result of the preset web page, wherein the data quality model is used to predict the data quality of the entire web page and output the evaluation result.

14. The device according to claim 12, wherein: The data preprocessing unit is specifically used for: Acquiring initial webpage data of each preset webpage among the plurality of preset webpages; Based on the parsing method matching the initial web page data, the initial web page data of the preset web page is parsed to obtain target parsed data of each preset web page.

15. The device according to any one of claims 12 to 14, wherein the data preprocessing unit is further configured to: Determining whether there is target data that meets the data requirement in the target parsed data of the preset web page whose page evaluation result does not meet the page quality requirement; When it is determined that there is target data that meets the data requirement, a preset web page containing the target data is used as a second web page to obtain the second web page set; wherein, The second parsed data of the second web page is: the target data that meets the data requirement in the target parsed data of the second web page.

16. The device according to any one of claims 12 to 15, wherein: The database construction unit is specifically used for: Acquire multimedia data corresponding to first parsed data of the first webpage, and acquire multimedia data corresponding to second parsed data of the second webpage, so as to obtain a multimedia data set; Marking each multimedia data in the multimedia data set to obtain multimedia data with marking information; At least based on the multimedia data with labeled information, a multimedia reasoning database for model training is obtained.

17. The device according to claim 16, wherein: The database construction unit specifically performs at least one of the following: Inputting image data in the multimedia data into the image reasoning annotation model to obtain annotation information of the image data in the multimedia data; The text data in the multimedia data is input into the text reasoning annotation model to obtain the annotation information of the text data in the multimedia data.

18. The apparatus according to claim 17, further comprising: A site processing unit; wherein the site processing unit is used to: Based on obtaining the first web page and / or the second web page of the multimedia reasoning database, obtaining a plurality of initial network sites; Based on the multiple initial websites, determining at least one target page that meets the page requirements; Using the at least one target page, an image sample for training the image reasoning annotation model and / or a text sample for training the text reasoning annotation model are obtained.

19. The apparatus according to claim 18, further comprising: Training unit; wherein the training unit is used to: Performing model training on the image reasoning and annotation model using at least the image samples obtained for training the image reasoning and annotation model, so as to obtain the image reasoning and annotation model after training; and / or, The text reasoning annotation model is at least trained by using the obtained text samples for training the text reasoning annotation model to obtain the text reasoning annotation model after training.

20. The device according to claim 18, wherein The site processing unit is specifically used for: Determining at least one target network site that meets preset site requirements from the plurality of initial network sites; At least one target page that meets the page requirement is selected from the at least one target website.

21. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10.

23. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images