Business information automatic collection method, system and device and medium

Abstract

The invention relates to an automatic business information collection method, system and device and a medium, and the method comprises the steps: determining a collection object of business information according to the demands of a user for the business information; controlling the automatic tool to capture business information from each information detailed page of the collection object; performing structured processing and data format conversion on the business information to obtain multi-field structured data of the business information; according to the multi-field structured data, iterative training is conducted on the selected base model, an SQL processing enhancement model is obtained, and the SQL processing enhancement model can push corresponding structured business information to a user side based on query requirements input by a user. According to the invention, the business information in the industry can be captured in a directional manner, and the dimension information of the business field desired by the user can be obtained in an inquiry manner, so that the business information can be automatically collected and sorted without a specially-assigned person, the cost is reduced, and the timeliness, comprehensiveness and accuracy of business information collection are improved.

Description

Technical Field

[0001] The present invention relates to the field of robot application technology, and in particular to a method, system, device and medium for automatically collecting commercial information. Background Art

[0002] Collecting business information is an important foundation for enterprises to carry out strategic deployment, market insight, competition analysis, decision support, risk management, innovation and development. By making full use of business information, enterprises can more accurately grasp market dynamics and customer needs, formulate more scientific and effective business strategies, and thus gain an advantageous position in the fierce market competition.

[0003] Currently, there are two main ways to collect business information in the existing technology:

[0004] The first method is to manually search for relevant business information from various web pages, and then analyze and organize the collected business information to finally obtain useful business information. However, the cost of collecting business information in this way is high and the efficiency is low, and the collected business information is often out of date and lacks content.

[0005] The second type: With the rapid development of information technology, artificial intelligence (AI) technology has shown great potential in many fields and has gradually replaced some of the work traditionally done manually. Therefore, by directly asking the AI ​​system, the AI ​​system directly gives the corresponding answer through a large model. However, since the current AI system is mainly based on general functions, when asking the AI ​​system for business information, users need to interact with the AI ​​system for a long time to correct the query prompts, and in the process of asking for business information, it is difficult for the AI ​​system to accurately give accurate information, so it is necessary to further screen the business information manually. Summary of the invention

[0006] 1. Technical issues to be resolved

[0007] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a method, system, device and medium for automatically collecting business information, which solves the technical problems of high cost, low efficiency, incompleteness and low accuracy of automatic collection of business information in the prior art.

[0008] (II) Technical solution

[0009] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0010] In a first aspect, an embodiment of the present invention provides a method for automatically collecting commercial information, comprising:

[0011] Determine the objects of commercial information collection based on users' demand for commercial information;

[0012] Controlling a preset automated tool to crawl commercial information from each information detail page of the collection object;

[0013] Performing structured processing and data format conversion on the business information to obtain multi-field structured data of the business information;

[0014] According to the multi-field structured data, the selected base model is iteratively trained to obtain an SQL processing enhancement model, which can push corresponding structured business information to the user end based on the query requirements input by the user.

[0015] Optionally, before controlling the automation tool to crawl the commercial information from each information detail page of the collection object, the method further includes:

[0016] Control the automated tool to crawl the website list page of the collection object;

[0017] Determine whether there is a verification code input window on the website list page;

[0018] If there is a verification code input window on the website list page, the control automation tool will recognize the verification code input into the corresponding page position;

[0019] If there is no verification code input window on the website list page, the control automation tool sequentially opens the information detail page of each list on the website list page.

[0020] Optionally, if there is a verification code input window on the website list page, the control automation tool will recognize that the verification code is input into the corresponding page position including:

[0021] Obtaining the verification code image in the verification code input window;

[0022] Performing preprocessing including noise reduction, grayscale conversion and binarization on the verification code image;

[0023] Segment the characters in the preprocessed verification code image to obtain all verification code characters;

[0024] A deep learning model is used to recognize each verification code character, and the recognized verification code is input into the corresponding page position through an automated tool.

[0025] Optionally, controlling the preset automated tool to capture the commercial information from each information detail page of the collection object includes:

[0026] Traversing all information detail pages in the collection object, and controlling the automation tool to open each information detail page of the collection object in sequence;

[0027] Based on preset keywords, commercial information including URL information, title information and core text information in the current page is extracted from the information detail page.

[0028] Optionally, performing structured processing and data format conversion on the business information to obtain multi-field structured data of the business information includes:

[0029] Performing structural processing on the commercial information in a specified data format to obtain structured data of the commercial information;

[0030] Using a large language model and combining the keywords, extracting keyword fields from each structural layer of the structured data to obtain initial multi-field structured data of the business information;

[0031] The initial multi-field structured data is preprocessed including data parsing, data cleaning and data deduplication to obtain the multi-field structured data of the business information.

[0032] Optionally, the large language model includes: a data extraction module and a label extraction module;

[0033] A data extraction module, used to extract a text field containing at least one keyword from the structured data, the keywords including project name, unit name, project number, project location and contact information;

[0034] The tag extraction module is used to extract text fields matching the semantic information of the keyword from the natural language text by parsing the natural language text in the structured data, and convert the text fields into tag fields of the commercial announcement type.

[0035] Optionally, iteratively training the selected base model according to the multi-field structured data to obtain an SQL processing enhanced model, which can push corresponding structured business information to the user end based on the query requirements input by the user, including:

[0036] Obtaining an embedding model, a vector database, and a base model;

[0037] Training the structured data through an embedding model, and storing the word vectors generated by the training in the vector database;

[0038] After converting the SQL question input by the user into vector data, retrieving the word vector with the greatest correlation with the vector data from the vector database, and combining the vector data with the retrieved word vector as a prompt word;

[0039] Input the prompt word into the base model to generate a corresponding SQL statement;

[0040] Modifying the SQL statement according to user feedback on the SQL statement;

[0041] The corrected SQL statements and the corresponding prompt words are used as training data, and the base model is iteratively trained to obtain the SQL processing enhanced model.

[0042] In a second aspect, an embodiment of the present invention provides an automatic collection system for commercial information, including:

[0043] An information collection object determination module is used to determine the collection object of commercial information according to the user's demand for commercial information;

[0044] An automatic information capture module, used to control a preset automated tool to capture commercial information from each information detail page of the collection object;

[0045] An information processing module, used for performing structured processing and data format conversion on the business information to obtain multi-field structured data of the business information;

[0046] A model training module, used for iteratively training the selected base model according to the multi-field structured data to obtain an SQL processing enhanced model;

[0047] The information push module is used to control the SQL processing enhancement model to push the corresponding structured business information to the user end based on the query requirements input by the user.

[0048] In a third aspect, an embodiment of the present invention provides an electronic device, including:

[0049] A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the above-mentioned steps of a method for automatically collecting commercial information when executed by the processor.

[0050] In a fourth aspect, an embodiment of the present invention provides a computer-readable medium having computer-executable instructions stored thereon, and when the executable instructions are executed by a processor, the above-mentioned method steps for automatically collecting commercial information are implemented.

[0051] (III) Beneficial effects

[0052] The beneficial effects of the present invention are as follows: the present invention proposes a method for automatically collecting business information. By automatically capturing business information from selected collection objects and structuring the business information as training data to train a base model, a technical solution of an SQL processing enhancement model that can push corresponding structured business information to the user end based on the query requirements input by the user is obtained. Compared with the prior art, it can capture business information within the industry in a targeted manner and obtain dimensional information of the business field that the user wants by asking, so that business information can be automatically collected and sorted without the need for a dedicated person, thereby reducing costs and improving the timeliness, accuracy and precision of collecting business information.

[0053] At the same time, the present invention converts the data format of the commercial information captured from the collection object so that the structured data of the commercial information is output in a fixed data format, further improving the analyzability and comparability of the collected commercial information. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A schematic diagram of a flow chart of a method for automatically collecting commercial information provided by an embodiment of the present invention;

[0055] Figure 2 A schematic diagram of a process for identifying and cracking a verification code on a website list page provided by an embodiment of the present invention;

[0056] Figure 3-6 A display result diagram of the SQL processing enhancement model provided in one embodiment of the present invention pushing structured business information to a user's email mailbox;

[0057] Figure 7-9 This is a diagram showing the display results after the SQL processing enhancement model provided by one embodiment of the present invention is queried. DETAILED DESCRIPTION

[0058] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0059] refer to Figure 1-2As shown, an embodiment of the present invention proposes an automatic collection method, system, device and medium for business information. The execution subject of the method can be a controller that controls RPA (Robotic Process Automation) tools, scripts and other automation tools. The method includes: first, determining the collection object of business information according to the user's demand for business information; second, controlling the preset automation tool to capture business information from each information detail page of the collection object; third, performing structured processing and data format conversion on the business information to obtain multi-field structured data of the business information; finally, iteratively training the selected base model according to the multi-field structured data to obtain a SQL (Structured Query Language) processing enhancement model, which can push the corresponding structured business information to the user end based on the query requirements input by the user.

[0060] This embodiment adopts the method of automatically capturing business information from the selected collection object, and structures the business information as training data to train the base model, thereby obtaining a technical solution of an SQL processing enhancement model that can push corresponding structured business information to the user end based on the query requirements input by the user. Compared with the existing technology, it can capture business information within the industry in a targeted manner and obtain dimensional information of the business field that the user wants through inquiries, so that business information can be automatically collected and organized without the need for dedicated personnel, thereby reducing costs and improving the timeliness, comprehensiveness and accuracy of collecting business information.

[0061] At the same time, in this embodiment, the data format of the commercial information captured from the collection object is converted so that the structured data of the commercial information is output in a fixed data format, further improving the analyzability and comparability of the collected commercial information.

[0062] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0063] Specifically, refer to Figure 1 As shown, this embodiment provides a method for automatically collecting commercial information, which includes:

[0064] S100: Determine the collection object of commercial information according to the user's demand for commercial information.

[0065] For example, when a user needs to collect commercial information related to the Ningxia Hui Autonomous Region Government, the scope of commercial information collection can be limited to the procurement website of the Ningxia Hui Autonomous Region Government. This will reduce the risk of confusion with other types of commercial information during the process of collecting commercial information and improve the accuracy of collected commercial information.

[0066] S200, controlling a preset automated tool to capture business information from each information detail page of the collection object.

[0067] In this embodiment, reference Figure 2 As shown, when the website of the collection object is determined, and it is not possible to determine whether the website needs to enter a verification code for verification before entering the website to collect commercial information, therefore, before step S200, the following method steps F100 to F300 are also included:

[0068] F100, control the automated tool to grab the website list page of the collection object.

[0069] F200. Determine whether there is a verification code input window on the website list page.

[0070] F300a. If there is a verification code input window on the website list page, the control automation tool will recognize the verification code input into the corresponding page position.

[0071] F300b. If there is no verification code input window on the website list page, the automation tool is controlled to open the information detail page of each list on the website list page in sequence.

[0072] Further, step F300a may include the following sub-steps F310 to F340:

[0073] F310. Obtain the verification code image in the verification code input window.

[0074] F320. Preprocess the verification code image including noise reduction, grayscale and binarization.

[0075] F330. Segment the characters in the preprocessed verification code image to obtain all verification code characters.

[0076] F340. Use the deep learning model to recognize each verification code character, and input the recognized verification code into the corresponding page position through an automated tool.

[0077] For example, when it is necessary to collect business information from the procurement website of the Ningxia Hui Autonomous Region government, the automation tool selects the RPA tool and uses AI technology to identify and crack the verification code. The specific process is as follows: First, the website list page of the procurement website is captured by controlling the RPA tool, and the verification code image in the website list page is directly obtained from the website list page. Secondly, the verification code image is preprocessed, including denoising, grayscale, and binarization steps to simplify the verification code image and highlight the important features of the verification code characters. Thirdly, the characters in the preprocessed image are segmented to extract each independent verification code character. Then, the segmented characters are recognized using a deep learning model (such as a convolutional neural network CNN). Finally, the recognized verification code is input into the corresponding page position through the RPA tool to complete the verification.

[0078] In this embodiment, step S200 may include the following sub-steps S210 to S230:

[0079] S210, traversing all information detail pages in the collection object, and controlling the automation tool to open each information detail page of the collection object in sequence.

[0080] S220: extracting commercial information including URL information, title information and core text information from the information detail page based on preset keywords.

[0081] For example, when it is necessary to collect business information from the procurement website of the Ningxia Hui Autonomous Region Government, the automation tool selects the RPA tool. The specific process is as follows: First, traverse all the information detail pages in the collection object, and control the RPA tool to open each information detail page of the procurement website in turn. Then, based on the keyword information, extract the business information of the current page including URL (Uniform Resource Locator) information, title information and core text information from each information detail page. Finally, traverse all the information detail pages in the procurement website, and store the obtained business information in a pre-set database. Among them, the keyword information includes project name, unit name, project number, project location and contact information, and the core text information is the text content with keyword information.

[0082] S300: Structuring the business information and converting the data format to obtain multi-field structured data of the business information.

[0083] Structuring business information can further improve the comprehensibility, analyzability and usability of business information in subsequent processing. Converting the data format of business information captured from the collection object so that the structured data of business information is output in a fixed data format can further improve the accuracy of collecting business information.

[0084] In this embodiment, step S300 may include the following sub-steps S310 to S330:

[0085] S310: Perform structural processing on the business information in a specified data format to obtain structured data of the business information.

[0086] S320. Utilize the large language model and combine the keywords to extract the keyword fields from each structural layer of the structured data to obtain the initial multi-field structured data of the business information. The large language model includes: a data extraction module and a label extraction module. The data extraction module is used to extract a text field containing at least one keyword from the structured data, and the keywords include project name, unit name, project number, project location and contact information. The label extraction module is used to extract the text field that matches the semantic information of the keyword from the natural language text by parsing the natural language text in the structured data, and convert the text field into a label field of the business announcement type.

[0087] In a specific embodiment, the large language model setting includes:

[0088] {

[0089] SYSTEM_PROMPT = 'You are an assistant who can extract data based on semantics. You can process a variety of text data and output it into common data formats, including json, csv, yaml, etc.'

[0090] USER_PROMPT="'According to the above information, extract the key information of the project's bidding and tendering, and output it in json format. Do not carry comments in json: the fields in json include project name, unit name, project number, project date, project amount, announcement type, project location, contact information and other information. The value range of announcement type: [bidding information, bidding information, winning bid information, procurement information, other]; the project location field is output to the provincial and municipal levels, and does not include districts, counties, towns and villages below the municipal level; if the project date lacks a year, the current year is used, and the format is converted to 'yyyy-MM-dd'; the unit of the project amount is uniformly converted to 10,000 yuan, and the unit is removed. The json field type is a numeric type. If no information is found, the value in the json field is assigned to null."'

[0091] }

[0092] For large language models, we focus on the data extraction and label extraction capabilities of the model. The data extraction module of the model converts the input sequence (structured data) into an embedding vector through the self-attention mechanism, and then the embedding vector is sent to the encoder structure and decoder structure to complete the data extraction. The encoder consists of multiple encoding blocks, each of which contains a self-attention layer and a feedforward neural network layer. The output of the self-attention layer can be expressed as:

[0093]

[0094] Among them, Q represents the query vector, K represents the key vector, and V represents the value vector. Represents the scaling factor.

[0095] The tag extraction module first parses the natural language text in the structured data and performs data preprocessing including text cleaning, word segmentation, word form restoration and stem extraction. Then, the feature text (key information) in the preprocessed natural language text is extracted using any of the following methods: TF-IDF (Term Frequency-Inverse Document Frequency), word embedding and N-gram (N characters or words that appear consecutively in the text). Finally, the feature text is matched with the corresponding tag, and the javelin is optimized as the title field of the announcement type in the commercial information.

[0096] S330, preprocessing including data analysis, data cleaning and data deduplication is performed on the initial multi-field structured data to obtain multi-field structured data of business information.

[0097] S400. Iteratively train the selected base model according to the multi-field structured data to obtain an SQL processing enhancement model, which can push corresponding structured business information to the user end based on the query requirements input by the user.

[0098] In this embodiment, step S400 may include the following sub-steps S410 to S460:

[0099] S410, acquiring an embedded model, a vector database, and a base model.

[0100] In a specific embodiment, the embedding model uses bge-large-zh to convert multi-field structured data into a numerical form that can be understood and processed by a computer. The embedding model can capture the semantic information of words, phrases, sentences, and even entire documents, and map this information to vectors in a high-dimensional space. The vector database uses Milvus to store and retrieve vector data. The base model uses glm-4, which performs iterative training based on multi-field structured data and outputs SQL analysis statements that conform to actual business, so as to obtain an SQL processing enhancement model based on the base model.

[0101] S420: Train the structured data through the embedding model, and store the word vectors generated by the training into the vector database.

[0102] In a specific embodiment, the specific process of step S420 may include: first, use the selected bge-large-zh embedding model to train the structured data. During the training process, the bge-large-zh embedding model learns the semantic information in the structured data and maps the semantic information to a vector in a high-dimensional space. Then, after the bge-large-zh embedding model is trained, the bge-large-zh embedding model is used to convert the structured data into a word vector, which will be used as a numerical representation of database metadata and business information. Finally, the generated word vector is stored in a vector database for rapid retrieval and query when needed.

[0103] S430: After converting the SQL question input by the user into vector data, retrieve the word vector with the greatest correlation with the vector data from the vector database, and combine the vector data and the retrieved word vector into a prompt word.

[0104] In a specific embodiment, the specific process of step S430 may include: first, using NLP (natural language processing) technology to convert the SQL-related questions or descriptions input by the user into vector data. Then, using a similarity measurement method (K nearest neighbor search), retrieve the word vector with the greatest correlation with the vector data from the vector database. Finally, combine the vector data with the retrieved word vector as a prompt word so that the base model can generate a more accurate SQL statement.

[0105] S440: Input the prompt word into the base model to generate a corresponding SQL statement.

[0106] S450: The SQL statement is modified according to the user's feedback information on the SQL statement.

[0107] S460: Use the corrected SQL statement and the corresponding prompt word as training data, iteratively train the base model, and obtain an SQL processing enhanced model.

[0108] It is worth mentioning that the SQL processing enhancement model can push the corresponding structured business information to the user end according to the set time based on the query requirements entered by the user, where the user end includes one or more of the user's mobile end, email, and online chat application. Figure 3-6 As shown in the figure, the SQL processing enhancement model sends the commercial information related to the Ningxia Hui Autonomous Region Government to the user's email box via email. Figure 7-9 As shown, the user obtains the corresponding structured business information by querying the SQL processing enhancement model, thereby improving the timeliness, comprehensiveness and accuracy of collecting business information.

[0109] In addition, this embodiment also provides an automatic collection system for business information, which includes:

[0110] The information collection object determination module is used to determine the collection object of commercial information based on the user's demand for commercial information.

[0111] The automatic information capture module is used to control the preset automation tool to capture business information from each information detail page of the collection object.

[0112] The information processing module is used to perform structured processing and data format conversion on business information to obtain multi-field structured data of business information.

[0113] The model training module is used to iteratively train the selected base model based on multi-field structured data to obtain an SQL processing enhanced model.

[0114] The information push module is used to control the SQL processing enhancement model to push the corresponding structured business information to the user end based on the query requirements input by the user.

[0115] In addition, this embodiment further provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the above-mentioned steps of a method for automatically collecting commercial information when executed by the processor.

[0116] Finally, this embodiment also provides a computer-readable medium on which computer-executable instructions are stored. When the executable instructions are executed by a processor, the above-mentioned method steps for automatically collecting commercial information are implemented.

[0117] In summary, the embodiments of the present invention propose an automatic collection method, system, device and medium for business information, and the method may include: first, using an automated tool to capture business information. Secondly, the business information is structured and converted into a data format to obtain multi-field structured data of the business information. Then, based on the multi-field structured data, the selected base model is iteratively trained to obtain an SQL processing enhancement model that can push corresponding structured business information to the user end based on the query requirements input by the user. The method proposed in the embodiments of the present invention can periodically capture the information of a given vertical website, and after processing, an ideal result can be obtained, so that the user no longer needs to pay special attention to and look for other consultations, but only needs to pay attention to the business information provided by the SQL processing enhancement model. Similarly, the business information provided by the SQL processing enhancement model is structured data, and the user does not need to organize the business information.

[0118] Since the system / device described in the above embodiments of the present invention is a system / device used to implement the method of the above embodiments of the present invention, a person skilled in the art can understand the specific structure and deformation of the system / device based on the method described in the above embodiments of the present invention, and thus will not be described in detail here. All systems / devices used in the method of the above embodiments of the present invention belong to the scope of protection of the present invention.

[0119] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0120] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions.

[0121] It should be noted that in the claims, any reference numerals placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In the claims enumerating several means, several of these means may be embodied by the same hardware. The use of the words first, second, third, etc., is for convenience of expression only and does not indicate any order. These words may be understood as part of the component name.

[0122] In addition, it should be noted that, in the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0123] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments after knowing the basic creative concept. Therefore, the claims should be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0124] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention should also include these modifications and variations.

Claims

1. A method for automatically collecting commercial information, characterized in that: include: Determine the objects of commercial information collection based on users' demand for commercial information; Controlling a preset automated tool to crawl commercial information from each information detail page of the collection object; Performing structured processing and data format conversion on the business information to obtain multi-field structured data of the business information; According to the multi-field structured data, the selected base model is iteratively trained to obtain an SQL processing enhancement model, which can push corresponding structured business information to the user end based on the query requirements input by the user.

2. The method according to claim 1, characterized in that Before controlling the automated tool to crawl the commercial information from each information detail page of the collection object, the method further includes: Control the automated tool to crawl the website list page of the collection object; Determine whether there is a verification code input window on the website list page; If there is a verification code input window on the website list page, the control automation tool will recognize the verification code input into the corresponding page position; If there is no verification code input window on the website list page, the control automation tool sequentially opens the information detail page of each list on the website list page.

3. The method according to claim 2, characterized in that If there is a verification code input window on the website list page, the control automation tool will recognize that the verification code is input into the corresponding page position including: Obtaining the verification code image in the verification code input window; Performing preprocessing including noise reduction, grayscale conversion and binarization on the verification code image; Segment the characters in the preprocessed verification code image to obtain all verification code characters; A deep learning model is used to recognize each verification code character, and the recognized verification code is input into the corresponding page position through an automated tool.

4. The method according to claim 1, characterized in that Controlling the preset automated tool to capture business information from each information detail page of the collection object includes: Traversing all information detail pages in the collection object, and controlling the automation tool to open each information detail page of the collection object in sequence; Based on preset keywords, commercial information including URL information, title information and core text information in the current page is extracted from the information detail page.

5. The method according to claim 4, characterized in that Structuring the business information and converting the data format to obtain multi-field structured data of the business information includes: Performing structural processing on the commercial information in a specified data format to obtain structured data of the commercial information; Using a large language model and combining the keywords, extracting keyword fields from each structural layer of the structured data to obtain initial multi-field structured data of the business information; The initial multi-field structured data is preprocessed including data parsing, data cleaning and data deduplication to obtain the multi-field structured data of the business information.

6. The method according to claim 5, characterized in that The large language model includes: a data extraction module and a label extraction module; A data extraction module, used to extract a text field containing at least one keyword from the structured data, the keywords including project name, unit name, project number, project location and contact information; The tag extraction module is used to extract text fields matching the semantic information of the keyword from the natural language text by parsing the natural language text in the structured data, and convert the text fields into tag fields of the commercial announcement type.

7. The method according to claim 1, characterized in that According to the multi-field structured data, the selected base model is iteratively trained to obtain an SQL processing enhancement model, which can push corresponding structured business information to the user end based on the query requirements input by the user, including: Obtaining an embedding model, a vector database, and a base model; Training the structured data through an embedding model, and storing the word vectors generated by the training in the vector database; After converting the SQL question input by the user into vector data, retrieving the word vector with the greatest correlation with the vector data from the vector database, and combining the vector data with the retrieved word vector as a prompt word; Input the prompt word into the base model to generate a corresponding SQL statement; Modifying the SQL statement according to user feedback on the SQL statement; The corrected SQL statements and the corresponding prompt words are used as training data, and the base model is iteratively trained to obtain the SQL processing enhanced model.

8. An automatic collection system for commercial information, characterized in that: include: An information collection object determination module is used to determine the collection object of commercial information according to the user's demand for commercial information; An automatic information capture module, used to control a preset automated tool to capture commercial information from each information detail page of the collection object; An information processing module, used for performing structured processing and data format conversion on the business information to obtain multi-field structured data of the business information; A model training module, used for iteratively training the selected base model according to the multi-field structured data to obtain an SQL processing enhanced model; The information push module is used to control the SQL processing enhancement model to push the corresponding structured business information to the user end based on the query requirements input by the user.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of a method for automatically collecting commercial information as described in any one of claims 1 to 7 are implemented.

10. A computer-readable medium having computer-executable instructions stored thereon, characterized in that: When the executable instructions are executed by the processor, the steps of the method for automatically collecting commercial information as described in any one of claims 1 to 7 are implemented.

Images