Information collection method and device, computer device and storage medium

Abstract

This application relates to an information acquisition method, apparatus, computer device, and storage medium. The method includes: acquiring configuration information of an information acquisition program, the configuration information including data source information and acquisition rule information corresponding to target information to be acquired; the target information being information to be filled into a resource interaction file; the data source information indicating the address of the target webpage where the target information is located; and the acquisition rule information indicating the scheduling rules and network proxy rules of the information acquisition program; invoking the information acquisition program to acquire raw data from the target webpage based on the data source information and acquisition rule information; identifying candidate information associated with the resource interaction file from the raw data based on the data format of the raw data; and determining the target information from the candidate information based on the information entries set in the information template corresponding to the resource interaction file. This method can improve the efficiency of information acquisition from resource interaction files.

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to an information acquisition method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the development of information technology, more and more information is being published through web pages. For resource exchanges involving engineering, goods, or services, the initiator typically submits a request and publishes relevant information on a web page, allowing the provider to retrieve this information and generate the necessary resource exchange documents.

[0003] In related technologies, the provider of resource interaction usually needs to manually retrieve relevant information about the resource interaction file from the webpage where the resource interaction initiator publishes the information, and then generate the resource interaction file based on the retrieved information.

[0004] However, manually obtaining relevant information about resource interaction files requires identifying target information related to the resource interaction file to be generated from a massive amount of information on the webpage where the resource interaction initiator publishes the information. This is time-consuming and labor-intensive, resulting in low efficiency in collecting information from resource interaction files. Summary of the Invention

[0005] Therefore, it is necessary to provide an information collection method, apparatus, computer equipment, and storage medium that can improve the efficiency of information collection from resource exchange files, addressing the aforementioned technical problems.

[0006] Firstly, this application provides an information collection method. The method includes:

[0007] The configuration information of the information collection program is obtained. The configuration information includes the data source information and collection rule information corresponding to the target information to be collected. The target information is the information to be filled in the resource interaction file. The data source information is used to indicate the address of the target webpage where the target information is located. The collection rule information is used to indicate the scheduling rules and network proxy rules of the information collection program.

[0008] Based on the data source information and the collection rule information, the information collection program is invoked to collect the raw data from the target webpage;

[0009] Based on the data format of the original data, candidate information associated with the resource interaction file is identified from the original data;

[0010] The target information is determined from the candidate information based on the information entries set in the information template corresponding to the resource interaction file.

[0011] In one embodiment, identifying candidate information associated with the resource interaction file from the original data according to the data format of the original data includes:

[0012] If the original data is in text format, the original data is parsed by calling the information collection program to obtain the parsed data;

[0013] Based on the characteristics of the resource interaction file, data mining is performed on the parsed data to determine the hidden information corresponding to the parsed data;

[0014] The candidate information is determined based on the hidden information corresponding to the parsed data.

[0015] In one embodiment, the configuration information further includes mining configuration information, and determining the candidate information based on the hidden information corresponding to the parsed data includes:

[0016] Based on the mining configuration information, the semantics of the hidden information are corrected and duplicate hidden information is removed to generate the candidate information.

[0017] In one embodiment, identifying candidate information associated with the resource interaction file from the original data according to the data format of the original data includes:

[0018] If the original data is in image format or portable file format, the original data is input into the optical character recognition model, and the recognition result output by the optical character recognition model is obtained. The optical character recognition model is used to recognize the original data containing the features of the resource interaction file.

[0019] Based on the recognition results, the candidate information is determined.

[0020] In one embodiment, before inputting the raw data into the optical character recognition model and obtaining the recognition result output by the optical character recognition model, the method further includes:

[0021] Acquire training sample data corresponding to the optical character recognition model. The training sample data includes historical image format data and historical portable file format data collected by the information collection program. The training sample data carries a type label, which is used to indicate whether the training sample data contains the features of the resource interaction file.

[0022] The optical character recognition model is trained using the training sample data, and the model parameters of the optical character recognition model are optimized based on the training results.

[0023] In one embodiment, after determining the target information from the candidate information based on the information entries set in the resource interaction file information template, the method further includes:

[0024] Generate a form file corresponding to the target information based on the information entries corresponding to the target information;

[0025] Receive an information acquisition request sent by a terminal device, wherein the information acquisition request is used to request the acquisition of the target information;

[0026] The target information is sent to the terminal device.

[0027] Secondly, this application also provides an information collection device. The device includes:

[0028] The acquisition module acquires the configuration information of the information collection program. The configuration information includes the data source information and collection rule information corresponding to the target information to be collected. The target information is the information to be filled in the resource interaction file. The data source information is used to indicate the address of the target webpage where the target information is located. The collection rule information is used to indicate the scheduling rules and network proxy rules of the information collection program.

[0029] The data collection module is used to call the information collection program to collect raw data from the target webpage based on the data source information and the data collection rule information.

[0030] The determination module is used to identify candidate information associated with the resource interaction file from the original data according to the data format of the original data; and to determine the target information from the candidate information according to the information entries set in the information template corresponding to the resource interaction file.

[0031] In one embodiment, the determining module is specifically configured to, if the original data is in text format, parse the original data by calling the information collection program to obtain parsed data; perform data mining on the parsed data based on the characteristics of the resource interaction file to determine the hidden information corresponding to the parsed data; and determine the candidate information based on the hidden information corresponding to the parsed data.

[0032] In one embodiment, the configuration information further includes mining configuration information. The determining module is specifically used to correct the semantics of the hidden information and remove duplicate hidden information based on the mining configuration information to generate the candidate information.

[0033] In one embodiment, the determining module is specifically used to input the original data into an optical character recognition model if the original data is in image format or portable file format, and obtain the recognition result output by the optical character recognition model. The optical character recognition model is used to recognize the original data containing the features of the resource interaction file; and to determine the candidate information based on the recognition result.

[0034] In one embodiment, the determining module is further configured to acquire training sample data corresponding to the optical character recognition model, the training sample data including historical image format data and historical portable file format data collected by the information acquisition program, the training sample data carrying type labels, the type labels being used to indicate whether the training sample data contains the features of the resource interaction file; using the training sample data to train the optical character recognition model, and optimizing the model parameters of the optical character recognition model based on the training results.

[0035] In one embodiment, the information collection device further includes:

[0036] The distribution module is used to generate a form file corresponding to the target information based on the information entries corresponding to the target information; receive an information acquisition request sent by a terminal device, the information acquisition request being used to request the acquisition of the target information; and send the target information to the terminal device.

[0037] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the aforementioned information acquisition method.

[0038] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, the computer program being executed by a processor using the aforementioned information acquisition method.

[0039] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that is executed by a processor using the aforementioned information acquisition method.

[0040] The aforementioned information collection method, apparatus, computer equipment, and storage medium first acquire the configuration information of the information collection program. This configuration information includes data source information and collection rule information corresponding to the target information to be collected. The target information is the information to be filled into the resource interaction file. The data source information indicates the address of the target webpage where the target information is located, and the collection rule information indicates the scheduling rules and network proxy rules of the information collection program. Secondly, based on the data source information and collection rule information, the information collection program is invoked to collect raw data from the target webpage. Thirdly, based on the data format of the raw data, candidate information associated with the resource interaction file is identified from the raw data. Finally, based on the information entries set in the information template corresponding to the resource interaction file, the target information is determined from the candidate information. In this way, the information collection program can be automatically invoked to automatically collect raw data from the target webpage according to the configuration information, and the candidate information associated with the resource interaction file can be automatically identified from the raw data, and the required target information can be filtered from the candidate information. This achieves automatic collection and filtering of target information on the target webpage, thereby improving the efficiency of information collection from resource interaction files. Attached Figure Description

[0041] Figure 1 An application environment diagram of an information collection method provided in this application embodiment;

[0042] Figure 2 A flowchart illustrating an information collection method provided in an embodiment of this application;

[0043] Figure 3 A flowchart illustrating another information collection method provided in an embodiment of this application;

[0044] Figure 4 A flowchart illustrating another information collection method provided in an embodiment of this application;

[0045] Figure 5 A structural block diagram of an information acquisition device provided in an embodiment of this application;

[0046] Figure 6 This is an internal structural diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] The information collection method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on another network server.

[0049] Server 104 can obtain the configuration information of the information collection program at preset time intervals, and thus determine the target webpage to be collected and the scheduling rules and network proxy rules of the information collection program based on the configuration information. Next, server 104 calls the information collection program to collect raw data from the target webpage and identifies candidate information associated with resource interaction files from the raw data. Then, the server determines the target information from the candidate information according to the information entries set in the information template corresponding to the resource interaction file. Finally, when terminal device 102 initiates an information acquisition request to server 104, server 104 can send the target information to terminal device 102.

[0050] The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle systems. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0051] In one embodiment, such as Figure 2 As shown, an information collection method is provided, which can be applied to... Figure 1 The following explanation uses the servers in the example, including S201-S204:

[0052] S201. Obtain the configuration information of the information collection program.

[0053] In this application, the server can obtain the configuration information of the information collection program before collecting the target information to be filled in the resource interaction file.

[0054] The configuration information includes the data source information and collection rule information corresponding to the target information to be collected, and the target information is the information to be filled in the resource interaction file.

[0055] Data source information indicates the address of the target webpage where the target information is located, while collection rule information indicates the scheduling rules and network proxy rules of the information collection program.

[0056] It should be understood that the embodiments of this application do not limit the resource interaction documents mentioned above, and can be any document related to resource interaction in engineering, goods, or service trade. For example, it can include tender documents, bid documents, etc. Correspondingly, the target information mentioned above can be specific information in the tender document, such as the quantity information of the resource transaction, the initiator information of the resource transaction, etc.

[0057] In some embodiments, the aforementioned data source information is used to indicate the address of the target webpage where the target information is located, and the collection rule information is used to indicate the scheduling rules and network proxy rules of the information collection program. For example, the aforementioned address can be the Internet Protocol (IP) address, Uniform Resource Locator (URL) address, etc., of the target webpage. The aforementioned scheduling rules are used to indicate the information collection program to be invoked, and the aforementioned network proxy rules are used to indicate the proxy configuration required to schedule the information collection program.

[0058] It should be understood that the embodiments of this application do not limit the information collection program described above, and may include web crawler programs, robotic process automation (RPA) programs, etc. For example, in the embodiments of this application, the core file of a web crawler program can be used to collect web page data.

[0059] It should be understood that the embodiments of this application do not limit how the server obtains the configuration information of the information collection program. In some embodiments, the server can start the information collection program at preset time intervals. When the start of the information collection program is detected, the server can obtain the configuration information of the information collection program. The above time interval can be specifically set according to the actual situation. For example, it can be one day, half a day, three days, etc.

[0060] In other embodiments, when the server receives a request from a user to obtain target information, it can start an information collection program and obtain the configuration information of the information collection program after the program is started.

[0061] It should be understood that the information collection program involved in this application embodiment can not only collect data from web pages, but also parse and mine text-formatted data from web pages. Correspondingly, the configuration information of the aforementioned information collection program can include not only data source information and collection rule information, but also parsing configuration information and mining configuration information. The mining configuration information includes anomaly retry information, semantic correction information, deduplication mechanism information, etc.

[0062] S202. Based on the data source information and collection rule information, call the information collection program to collect the raw data from the target webpage.

[0063] In this step, after the server obtains the configuration information of the information collection program, it can call the information collection program to collect the raw data from the target webpage based on the data source information and collection rule information in the configuration information.

[0064] The aforementioned raw data may be data collected from the target webpage that has not been parsed or identified. It should be understood that this application embodiment does not limit the data format of the raw data. For example, the raw data may include text format data, image format data, Portable Document Format (PDF) data, etc.

[0065] It should be understood that the embodiments of this application do not limit how the information collection program is invoked to collect raw data from the target webpage. In some embodiments, the server may determine the information collection program to be scheduled based on the scheduling rules of the information collection program indicated by the collection rule information, determine the network proxy of the information collection program based on the network proxy rules of the information collection program indicated by the collection rule information, and determine the target webpage to be collected based on the data source information. Subsequently, the server invokes the information collection program to collect raw data from the target webpage.

[0066] S203. Based on the data format of the original data, identify candidate information associated with the resource interaction file from the original data.

[0067] In this step, after the server collects the raw data from the target webpage using the information collection program, it can identify candidate information associated with the resource interaction file from the raw data based on the data format of the raw data.

[0068] In some embodiments, if the original data is in text format, the server can parse the original data by calling an information acquisition program to obtain parsed data. Subsequently, the server can perform data mining on the parsed data based on the characteristics of the resource interaction file to determine the hidden information corresponding to the parsed data. Finally, the server can determine candidate information based on the hidden information corresponding to the parsed data.

[0069] The features of the aforementioned resource interaction files may include specific words, object names, numbers, etc., corresponding to the resource interaction files.

[0070] It should be understood that the above configuration information may also include parsing configuration information. Accordingly, through the parsing configuration information, the information collection program can automatically parse the raw data. For example, the above parsing configuration file can indicate the Hyper Text Markup Language (HTML) data to be parsed in the raw data and the parser for parsing HTML data. Subsequently, the server parses the HTML data by calling the HTML data parser in the information collection program to obtain the parsed data.

[0071] It should be understood that the embodiments of this application do not limit how data mining is performed, and it can be performed using machine learning models of artificial intelligence. In some embodiments, the configuration information also includes mining configuration information. After the server performs data mining on the parsed data to determine the hidden information corresponding to the parsed data, the semantics of the hidden information can be corrected and duplicate hidden information can be removed according to the mining configuration information to generate candidate information.

[0072] The configuration information mined includes anomaly retry information, semantic correction information, and deduplication mechanism information. This hidden information refers to information related to resource interaction files contained in the original data.

[0073] It should be noted that the above-mentioned abnormal retry information is used to perform data mining again when no hidden information is found or the found hidden information is obviously unrelated to the resource interaction file. The above-mentioned semantic correction information is used to correct typos and semantic order in the found hidden information. The above-mentioned deduplication mechanism information is used to delete duplicate information in the hidden information.

[0074] It should be understood that the above information collection program may include a data mining model, and the server can directly call the information collection module to perform data mining.

[0075] In other embodiments, if the original data is in image or portable file format, the original data is input into the optical character recognition model, and the recognition result output by the optical character recognition model is obtained. Based on the recognition result, candidate information is determined.

[0076] The optical character recognition (OCR) model is used to identify raw data containing features of interactive resource files. For example, if the raw data in image or portable file format contains specific words corresponding to interactive resource files, the OCR model can identify the text associated with those specific words and generate a recognition result.

[0077] It should be understood that the aforementioned optical character recognition model is generated after training on a training sample set. In some embodiments, the server can obtain training sample data corresponding to the optical character recognition model. The training sample data includes historical image format data and historical portable file format data collected by the information acquisition program. The training sample data carries type labels, which are used to indicate whether the training sample data contains the characteristics of resource interaction files. Subsequently, the server can use the training sample data to train the optical character recognition model and optimize the model parameters of the optical character recognition model based on the training results.

[0078] The aforementioned type labels can be determined based on the characteristics of resource interaction files and can be manually labeled.

[0079] In some embodiments, after the optical character recognition model has been trained, it can be trained again using the recognition results during actual use, thereby continuously optimizing the recognition accuracy of the optical character recognition model.

[0080] S204. Determine the target information from the candidate information based on the information entries set in the information template corresponding to the resource interaction file.

[0081] In this step, after the server identifies candidate information associated with the resource interaction file from the raw data, it can determine the target information from the candidate information based on the information entries set in the information template corresponding to the resource interaction file.

[0082] It should be understood that candidate information is information associated with resource interaction files. However, not all candidate information is useful when generating resource interaction files. In order to speed up the generation of resource interaction files, candidate information can be filtered to determine the target information.

[0083] It should be understood that, in this embodiment of the application, candidate information can be filtered through information entries set in the information template corresponding to the resource interaction file. The aforementioned information template may contain multiple information entries, each used to label a type of information or a characteristic of a type of information. For example, the resource interaction file may contain resource quantity entries and resource price entries; correspondingly, the server can filter target information related to the resource quantity entries and resource price entries from the candidate information.

[0084] It should be understood that the embodiments of this application do not limit how to determine the target information from the candidate information based on the information entries. In some embodiments, the server can obtain the keywords corresponding to the information entries, and determine the successfully matched candidate information as the target information by matching the keywords corresponding to the information entries with the candidate information.

[0085] In some embodiments, after determining the target information, the server can also generate a form file corresponding to the target information based on the information entries corresponding to the target information. Subsequently, after receiving an information retrieval request from the terminal device, the server can send the target information to the terminal device. The information retrieval request is used to request the retrieval of the target information.

[0086] The aforementioned terminal devices can be pre-defined devices with access permissions to the target information. The target information can be categorized and saved in the aforementioned form file based on information entries.

[0087] The information collection method provided in this application first obtains the configuration information of the information collection program. This configuration information includes data source information and collection rule information corresponding to the target information to be collected. The target information is the information to be filled into a resource interaction file. The data source information indicates the address of the target webpage where the target information is located, and the collection rule information indicates the scheduling rules and network proxy rules of the information collection program. Second, based on the data source information and collection rule information, the information collection program is invoked to collect raw data from the target webpage. Third, based on the data format of the raw data, candidate information associated with the resource interaction file is identified from the raw data. Finally, based on the information entries set in the information template corresponding to the resource interaction file, the target information is determined from the candidate information. In this way, the information collection program can be automatically invoked to automatically collect raw data from the target webpage according to the configuration information, and candidate information associated with the resource interaction file can be automatically identified from the raw data, and the required target information can be filtered from the candidate information. This achieves automatic collection and filtering of target webpage information, thereby improving the efficiency of information collection from resource interaction files.

[0088] The following section explains the two types of candidate information identified from the raw data that are associated with resource interaction files.

[0089] Figure 3 This is a flowchart illustrating another information collection method provided in an embodiment of this application. Figure 3 As shown, this information acquisition method parses raw data in text format to obtain candidate information. This information acquisition method includes steps S301-S306:

[0090] S301. Obtain the configuration information of the information collection program.

[0091] The configuration information includes the data source information and collection rule information corresponding to the target information to be collected. The target information is the information to be filled in the resource interaction file. The data source information is used to indicate the address of the target webpage where the target information is located. The collection rule information is used to indicate the scheduling rules and network proxy rules of the information collection program.

[0092] S302. Based on the data source information and collection rule information, call the information collection program to collect the raw data from the target webpage.

[0093] S303. If the original data is in text format, the original data is parsed by calling the information collection program to obtain the parsed data.

[0094] In some embodiments, the above configuration information may also include parsing configuration information. Accordingly, by parsing the configuration information, the information collection program can automatically parse the raw data.

[0095] For example, the aforementioned parsing configuration file can specify the Hyper Text Markup Language (HTML) data to be parsed in the original data, as well as the parser for parsing the HTML data. Subsequently, the server parses the HTML data by calling the HTML data parser in the information collection program to obtain the parsed data.

[0096] S304. Based on the characteristics of the resource interaction file, perform data mining on the parsed data to determine the hidden information corresponding to the parsed data.

[0097] The features of the aforementioned resource interaction files may include specific words, object names, numbers, etc., corresponding to the resource interaction files.

[0098] S305. Based on the hidden information corresponding to the parsed data, determine the candidate information.

[0099] In some embodiments, the configuration information also includes mining configuration information. After the server performs data mining on the parsed data to determine the hidden information corresponding to the parsed data, the semantics of the hidden information can be corrected and duplicate hidden information can be removed based on the mining configuration information to generate candidate information.

[0100] The configuration information mined includes anomaly retry information, semantic correction information, and deduplication mechanism information. This hidden information refers to information related to resource interaction files contained in the original data.

[0101] S306. Determine the target information from the candidate information based on the information entries set in the information template corresponding to the resource interaction file.

[0102] Figure 4 This is a flowchart illustrating another information collection method provided in an embodiment of this application. Figure 4 As shown, this information collection method parses raw data in image or portable file formats to obtain candidate information. This information collection method includes steps S401-S405:

[0103] S401. Obtain the configuration information of the information collection program.

[0104] The configuration information includes the data source information and collection rule information corresponding to the target information to be collected. The target information is the information to be filled in the resource interaction file. The data source information is used to indicate the address of the target webpage where the target information is located. The collection rule information is used to indicate the scheduling rules and network proxy rules of the information collection program.

[0105] S402. Based on the data source information and collection rule information, call the information collection program to collect the raw data from the target webpage.

[0106] S403. If the original data is in image or portable file format, the original data is input into the optical character recognition model, and the recognition result output by the optical character recognition model is obtained. The optical character recognition model is used to recognize the original data containing the features of resource interaction files.

[0107] It should be understood that the aforementioned optical character recognition model is generated after training on a training sample set. In some embodiments, the server can obtain training sample data corresponding to the optical character recognition model. The training sample data includes historical image format data and historical portable file format data collected by the information acquisition program. The training sample data carries type labels, which are used to indicate whether the training sample data contains the characteristics of resource interaction files. Subsequently, the server can use the training sample data to train the optical character recognition model and optimize the model parameters of the optical character recognition model based on the training results.

[0108] S404. Based on the recognition results, determine the candidate information.

[0109] S405. Determine the target information from the candidate information based on the information entries set in the information template corresponding to the resource interaction file.

[0110] The information collection method provided in this application first obtains the configuration information of the information collection program. This configuration information includes data source information and collection rule information corresponding to the target information to be collected. The target information is the information to be filled into a resource interaction file. The data source information indicates the address of the target webpage where the target information is located, and the collection rule information indicates the scheduling rules and network proxy rules of the information collection program. Second, based on the data source information and collection rule information, the information collection program is invoked to collect raw data from the target webpage. Third, based on the data format of the raw data, candidate information associated with the resource interaction file is identified from the raw data. Finally, based on the information entries set in the information template corresponding to the resource interaction file, the target information is determined from the candidate information. In this way, the information collection program can be automatically invoked to automatically collect raw data from the target webpage according to the configuration information, and candidate information associated with the resource interaction file can be automatically identified from the raw data, and the required target information can be filtered from the candidate information. This achieves automatic collection and filtering of target webpage information, thereby improving the efficiency of information collection from resource interaction files.

[0111] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0112] Based on the same inventive concept, this application also provides an information acquisition device for implementing the information acquisition method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more information acquisition device embodiments provided below can be found in the limitations of the information acquisition method described above, and will not be repeated here.

[0113] In one embodiment, such as Figure 5 As shown, an information acquisition device 500 is provided, including: an acquisition module 501, a collection module 502, a determination module 503, and a distribution module 504, wherein:

[0114] The acquisition module 501 acquires the configuration information of the information collection program. The configuration information includes the data source information and collection rule information corresponding to the target information to be collected. The target information is the information to be filled in the resource interaction file. The data source information is used to indicate the address of the target webpage where the target information is located. The collection rule information is used to indicate the scheduling rules and network proxy rules of the information collection program.

[0115] The data collection module 502 is used to call the data collection program to collect raw data from the target webpage based on the data source information and collection rule information.

[0116] The determination module 503 is used to identify candidate information associated with the resource interaction file from the original data according to the data format of the original data; and to determine the target information from the candidate information according to the information entries set in the information template corresponding to the resource interaction file.

[0117] In one embodiment, the determining module 503 is specifically used to, if the original data is in text format, parse the original data by calling the information collection program to obtain the parsed data; perform data mining on the parsed data according to the characteristics of the resource interaction file to determine the hidden information corresponding to the parsed data; and determine candidate information according to the hidden information corresponding to the parsed data.

[0118] In one embodiment, the configuration information further includes a mining configuration information determination module 503, which is specifically used to correct the semantics of the hidden information and remove duplicate hidden information based on the mining configuration information to generate candidate information.

[0119] In one embodiment, the determining module 503 is specifically used to input the original data into the optical character recognition model if the original data is in image format or portable file format, and obtain the recognition result output by the optical character recognition model. The optical character recognition model is used to recognize the original data containing features of resource interaction files; and to determine candidate information based on the recognition result.

[0120] In one embodiment, the determining module 503 is further configured to acquire training sample data corresponding to the optical character recognition model. The training sample data includes historical image format data and historical portable file format data collected by the information acquisition program. The training sample data carries a type label, which is used to indicate whether the training sample data contains the characteristics of resource interaction files. The training sample data is used to train the optical character recognition model, and the model parameters of the optical character recognition model are optimized based on the training results.

[0121] In one embodiment, the information acquisition device further includes:

[0122] The distribution module 504 is used to generate a form file corresponding to the target information based on the information entries corresponding to the target information; receive an information acquisition request sent by the terminal device, the information acquisition request being used to request the acquisition of the target information; and send the target information to the terminal device.

[0123] Each module in the aforementioned information acquisition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0124] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When executed by the processor, the computer program implements an information acquisition method.

[0125] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0126] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described information acquisition method.

[0127] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the above-described information acquisition method.

[0128] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described information acquisition method.

[0129] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0130] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0131] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An information collection method, characterized in that, The method includes: The configuration information of the information collection program is obtained. The configuration information includes the data source information and collection rule information corresponding to the target information to be collected. The target information is the information to be filled in the resource interaction file. The target information includes the quantity information of the resource transaction and the initiator information of the resource transaction. The data source information is used to indicate the address of the target webpage where the target information is located. The collection rule information is used to indicate the scheduling rules and network proxy rules of the information collection program. The scheduling rules are used to indicate the information collection program to be called. The network proxy rules are used to indicate the proxy configuration required to schedule the information collection program. Based on the data source information and the collection rule information, the information collection program is invoked to collect the raw data from the target webpage; Based on the data format of the original data, candidate information associated with the resource interaction file is identified from the original data; The target information is determined from the candidate information based on the information entries set in the information template corresponding to the resource interaction file. The step of identifying candidate information associated with the resource interaction file from the original data according to the data format of the original data includes: If the original data is in text format, the information collection program is invoked to parse the original data to obtain parsed data; based on the characteristics of the resource interaction file, data mining is performed on the parsed data to determine the hidden information corresponding to the parsed data; and based on the hidden information corresponding to the parsed data, the candidate information is determined. If the original data is in image or portable file format, the original data is input into the optical character recognition model, and the recognition result output by the optical character recognition model is obtained. The optical character recognition model is used to recognize the original data containing the features of the resource interaction file; based on the recognition result, the candidate information is determined. The configuration information also includes mining configuration information; determining the candidate information based on the hidden information corresponding to the parsed data includes: Based on the mining configuration information, the semantics of the hidden information are corrected and duplicate hidden information is removed to generate the candidate information; Before inputting the raw data into the optical character recognition model and obtaining the recognition result output by the optical character recognition model, the method further includes: Acquire training sample data corresponding to the optical character recognition model. The training sample data includes historical image format data and historical portable file format data collected by the information collection program. The training sample data carries a type label, which is used to indicate whether the training sample data contains the features of the resource interaction file. The optical character recognition model is trained using the training sample data, and the model parameters of the optical character recognition model are optimized based on the training results.

2. The method according to claim 1, characterized in that, After determining the target information from the candidate information based on the information entries set in the resource interaction file information template, the method further includes: Generate a form file corresponding to the target information based on the information entries corresponding to the target information; Receive an information acquisition request sent by a terminal device, wherein the information acquisition request is used to request the acquisition of the target information; The target information is sent to the terminal device.

3. An information acquisition device, characterized in that, The device includes: The acquisition module acquires the configuration information of the information collection program. The configuration information includes the data source information and collection rule information corresponding to the target information to be collected. The target information is the information to be filled in the resource interaction file. The target information includes the quantity information of the resource transaction and the initiator information of the resource transaction. The data source information is used to indicate the address of the target webpage where the target information is located. The collection rule information is used to indicate the scheduling rules and network proxy rules of the information collection program. The scheduling rules are used to indicate the information collection program to be called, and the network proxy rules are used to indicate the proxy configuration required to schedule the information collection program. The data collection module is used to call the information collection program to collect raw data from the target webpage based on the data source information and the data collection rule information. The determination module is used to identify candidate information associated with the resource interaction file from the original data according to the data format of the original data; and to determine the target information from the candidate information according to the information entries set in the information template corresponding to the resource interaction file. The determining module is specifically used to, if the original data is in text format, parse the original data by calling the information collection program to obtain the parsed data; perform data mining on the parsed data based on the characteristics of the resource interaction file to determine the hidden information corresponding to the parsed data; and determine candidate information based on the hidden information corresponding to the parsed data. The determining module is specifically used to input the original data into the optical character recognition model if the original data is in image format or portable file format, and obtain the recognition result output by the optical character recognition model. The optical character recognition model is used to recognize the original data containing the features of resource interaction files; and to determine candidate information based on the recognition result. The configuration information also includes mining configuration information. The determining module is specifically used to correct the semantics of the hidden information and remove duplicate hidden information based on the mining configuration information in order to generate candidate information. The determining module is further used to acquire training sample data corresponding to the optical character recognition model. The training sample data includes historical image format data and historical portable file format data collected by the information acquisition program. The training sample data carries type labels, which are used to indicate whether the training sample data contains the characteristics of resource interaction files. The training sample data is used to train the optical character recognition model, and the model parameters of the optical character recognition model are optimized according to the training results.

4. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 2.

6. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 2.

Images