Method for extracting information of bidding keywords in recruitment websites based on regular expression and scrapy

Abstract

The application discloses a method for extracting keywords information of bidding and opening bidding based on regular expression and scrapy, comprising the following steps: S1, determining the analysis field of the bidding and opening bidding project; S2, collecting and sorting the keyword list of the field, wherein the keyword list is created as an ordered keyword list or an unordered keyword list according to the field requirement; and S3, selecting the subsequent processing method according to the specific field condition. By combining the regular expression and the scrapy, the application can extract multiple keywords information of the bidding and opening bidding project, so as to avoid the problem of missing search. The application respectively extracts the keywords information of the bidding, the winning bidding and the opening bidding project, can carry out targeted query extraction according to the different keyword extraction modes of the opening bidding project, realizes the quick extraction of important information in the text, removes the useless data, solves the problems of information redundancy, reading difficulty and the like.

Description

A Method for Extracting Keyword Information from Bidding Processes Based on Regular Expressions and Scrapy Technical Field

[0001] This invention relates to the field of bidding and tendering project retrieval, and more particularly to a method for extracting bidding and tendering keyword information based on regular expressions and Scrapy. Background Technology

[0002] Tendering and bidding are internationally recognized, organized market transactions involving the buying and selling of engineering projects, goods, or services. Typically, the procuring entity sets forth conditions and requirements beforehand, inviting numerous potential buyers to compete and selecting a successful bidder according to prescribed procedures. In today's information age, one approach for bidders is to access various local tendering websites to gather information, then manually search and filter the necessary data. Another approach is to use large tendering information websites and perform full-text searches. However, both methods are time-consuming and labor-intensive, and the built-in search functions of tendering websites cannot guarantee quality, potentially causing bidders to miss important information.

[0003] Furthermore, in the existing technology, the keyword extraction for bidding projects is generally the same as that for bidding and winning projects. However, the bidding documents are different from those for bidding and winning projects. The bidding records contain multiple sets of information that meet the requirements, and each set of information needs to contain all the required fields, and the fields must correspond one-to-one. Therefore, the keyword extraction method for bidding projects should be different from that for bidding and winning projects. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides a method for extracting keyword information during bidding and tendering based on regular expressions and Scrapy.

[0005] This invention is achieved through the following technical solution:

[0006] A method for extracting keyword information from bidding processes based on regular expressions and Scrapy includes the following steps:

[0007] S1. Determine the parsing fields for the bidding and tendering projects;

[0008] S2. Collect and organize the keyword list of the field. The keyword list is created into an ordered keyword list or an unordered keyword list according to the field requirements. The ordered keyword list is a keyword list in which the keywords have a priority order. The unordered keyword list is a keyword list in which the keywords have a sibling relationship. The keyword list includes one or more of the following: a starting keyword list and an ending keyword list.

[0009] S3. Select a subsequent processing method based on the specific field conditions. The subsequent processing methods include the Scrapy bidding and winning method, the regular expression bidding and winning method, the Scrapy bid opening method, and the regular expression bid opening method. The Scrapy bidding and winning method and the regular expression bidding and winning method are used to extract keyword information from the bidding and winning projects, and the Scrapy bid opening method and the regular expression bid opening method are used to extract keyword information from the bid opening projects.

[0010] Furthermore, the Scrapy bidding process specifically includes the following sub-steps:

[0011] S3011. Using Scrapy's Selector class, the content of the details page is constructed into a selector object. By processing and extracting the object information, the required value is obtained.

[0012] S3012. Use the methods of the selector object to retrieve all table labels and construct a list of table objects;

[0013] S3013. Loop through the table object and pass it into a pre-set table processing model. The table processing model is used to determine the order of the keyword list, locate the table header, and locate the keyword position.

[0014] S1014. Search for keyword information in the table object.

[0015] Furthermore, step S3014 specifically includes the following sub-steps:

[0016] S30141. If no keyword information is found, repeat step S3013 until all tables have been searched;

[0017] S30142. When keyword information is found, the return value of the table processing model is passed to the demand value processing model, which is used to extract demand values, determine demand values, and standardize demand values.

[0018] Furthermore, the regular expression bidding processing method specifically includes the following sub-steps:

[0019] S3021. Expand the keywords in the keyword list into regular expression statements and combine them into a new keyword list. Use regular expression replacement and BeautifulSoup to process the details page content into plain text information without HTML tags. Obtain the required values ​​by processing and extracting the text information.

[0020] S3022. Pass the list of all keywords into a pre-set text processing model, which is used to determine the order of the keyword list and locate the keyword positions;

[0021] S3023. Perform keyword search on text information.

[0022] Furthermore, step S3023 also includes, when keyword information is found, passing the return value of the text processing model into the demand value processing model, which is used to extract demand values, determine demand values, and standardize demand values.

[0023] Furthermore, the subsequent processing method also includes:

[0024] S3031. First, use the scrapy processing method to search for keyword information in the table object. If the search yields results, output the results. If the search yields no results, proceed to step S3032.

[0025] S3032. If no results are found, use the selector object's methods to extract the list of plain text information from the selector.

[0026] S3033. Retrieve plain text string elements from the plain text information list, and check whether the starting keyword is in the string element. If the result is yes, proceed to step S3034; if the result is no, repeat step S3032.

[0027] S3034. Pass the string and the list of ending keywords into the field processing function to determine whether a value that meets the requirements can be extracted. If the result is yes, output the value as the matching value of the field. If the result is no, repeat step S3032.

[0028] Furthermore, when step S3032 is executed repeatedly and the plain text information list is looped through without matching the required value, keyword search is performed using regular expression processing methods.

[0029] Furthermore, the scrapy index opening processing method includes the following steps:

[0030] S3041. Using Scrapy's Selector class, the content of the details page is constructed into a selector object. By processing and extracting the object information, the required value is obtained.

[0031] S3042. Use the methods of the selector object to locate all table tags and construct a list of table objects;

[0032] S3043. Take the table object from the table object list and put it into the table processing model. The table processing model is used to locate the enterprise name position, process each group of data horizontally by row, match the required value with the field name one by one, and process the format of the required value.

[0033] S3044. Output a list containing multiple sets of data, each set containing all required fields and their corresponding values.

[0034] Furthermore, when no results are found, the keyword information search is performed using regular expression indexing methods, specifically including the following steps:

[0035] S3051. Using regular expression replacement and BeautifulSoup, the main text of the details page is processed into plain text information without HTML tags, and the required values ​​are extracted by processing the text.

[0036] S3052. Match company name, divide the text into several parts according to the company name, and input the divided string and all field keywords into the text processing model. The processing model is used to orderly divide all requirement values, match the requirement values ​​with field names one by one, and process the requirement value format.

[0037] The beneficial effects of this invention are:

[0038] (1) By combining regular expressions and scrapy, this invention can extract multiple keyword information from bidding projects to avoid missing the problem. Based on the extracted important information, the data can be retrieved, clustered and classified more efficiently, solving problems such as information redundancy and reading difficulties.

[0039] (2) This invention extracts keyword information for bidding, winning bid and opening bid projects respectively, and can perform targeted query extraction according to different keyword extraction methods for opening bid projects. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 is a flowchart of the method for extracting bidding keyword information based on regular expressions and Scrapy proposed in an embodiment of the present invention;

[0042] Figure 2 is a schematic diagram of the terminal device for extracting bidding keyword information based on regular expressions and Scrapy according to an embodiment of the present invention.

[0043] Figure 3 is a schematic diagram of the computer-readable storage medium structure of the method for extracting bidding keyword information based on regular expressions and Scrapy proposed in an embodiment of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0045] Example 1

[0046] As shown in Figure 1, this embodiment proposes a method for extracting keyword information during bidding and tendering based on regular expressions and Scrapy. This embodiment processes nearly 30 fields, including bidding company, bidding contact person, bidding contact number, bidding agent company, bidding agent contact person, bidding agent contact number, project scale, qualification requirements, project amount, etc.; and more than 20 fields, including the name of the winning bidder company, the winning bid amount, the project manager, and the discount rate.

[0047] The specific implementation principle of this embodiment is as follows:

[0048] 1. Determine the fields to be parsed;

[0049] 2. Collect and organize the keyword list for the field (starting keyword list, ending keyword list (if any)). The keyword list is generally created as an ordered keyword list or an unordered keyword list according to the field requirements. Ordered means that there is a priority order among the keywords, while unordered means that the keywords are of the same level. For special fields, both types of keyword lists will be created. Then, the subsequent processing method will be selected according to the specific field situation.

[0050] 3. Methods one, two, and three are only applicable to bidding and winning bids.

[0051] The first method specifically includes the following steps:

[0052] Use Scrapy's Selector class to construct a selector object from the details page body (HTML text with tags), and then process and extract the object information to obtain the required value;

[0053] Use the methods of the selector object to retrieve all table labels and build a list of table objects;

[0054] The table object is retrieved in a loop and passed to a pre-written table processing model. The main functions of this model are: determining the order of the keyword list, locating the table header, and locating the keyword position.

[0055] If the result is none, continue the loop until all tables have been searched; if the result is yes, pass the return value of the table processing model to the requirement value processing model. The main functions of this model are: extracting requirement values, determining requirement values, and standardizing requirement values.

[0056] The second method specifically includes the following steps:

[0057] Expand the keywords in the keyword list into regular expression statements to form a new keyword list. Then, use regular expression replacement and BeautifulSoup to process the details page text (HTML text with tags) into plain text information without HTML tags. Extract the required values ​​by processing the text.

[0058] Pass in the list of all keywords to a pre-written text processing model. The main functions of this model are: determining the order of the keyword list and locating the keyword positions.

[0059] If the result is yes, the return value of the table processing model is passed to the requirement value processing model. The main functions of this model are: extracting requirement values, determining requirement values, and standardizing requirement values.

[0060] The third method specifically includes the following steps:

[0061] First, use method one. If the result is yes, output the result.

[0062] If the result is none, use the selector object's methods to extract the list of plain text information from the selector;

[0063] Retrieve plain text string elements (without repetition) from the plain text information list, and check if the starting keyword is in the string element. If the result is no, repeat this step until the plain text information list is completely looped. If the required value is not matched after the plain text information list is completely looped, proceed to method two.

[0064] If the result is yes, the string and the list of ending keywords are passed together to the field processing function to determine whether a value that meets the requirements can be extracted. If the result is yes, the value is output as the matching value of the field. If the result is no, the previous step is repeated until the list of plain text information is completely looped. If the required value is not matched after the list of plain text information is completely looped, then proceed to method two.

[0065] Bid opening extraction method: Unlike bidding and winning bid, the bid opening record will contain multiple sets of information that meet our needs, and each set of information needs to contain all the required fields, and the fields must correspond one-to-one. Methods four and five are applicable to bid opening projects.

[0066] The fourth method specifically includes the following steps:

[0067] Use the methods of the selector object to locate all table tags and build a list of table objects;

[0068] Retrieve table objects from the list of table objects (without repeating values ​​during looping) and place them into the table processing model. The main functions of this model are: locating the company name, processing each group of data horizontally by row, mapping the required values ​​to field names one by one, and processing the format of the required values, etc.

[0069] Finally, the output will be a list containing multiple sets of data, each set containing all required fields and their corresponding values.

[0070] Method five specifically includes the following steps:

[0071] If method four fails to match any results, use regular expression replacement and BeautifulSoup to process the details page body (HTML text with tags) into plain text information without HTML tags, and extract the required values ​​by processing the text.

[0072] First, match the company name, then use the company name to divide the text into several parts, and pass the divided strings along with all field keywords into the processing model. The main functions of this model are: orderly dividing all requirement values, mapping requirement values ​​to field names one by one, and processing the format of requirement values, etc.

[0073] Example 2

[0074] Based on Example 1, this example further proposes a bidding keyword information extraction system based on regular expressions and Scrapy, including:

[0075] Field parsing module: used to determine the fields to be parsed for bidding items;

[0076] Keyword list creation module: collects and organizes the keyword list of the field. The keyword list can be created into an ordered keyword list or an unordered keyword list according to the field requirements. The ordered keyword list is a list of keywords with priority order, and the unordered keyword list is a list of keywords with a sibling relationship. The keyword list includes one or more of the following: a starting keyword list and an ending keyword list.

[0077] Keyword extraction module: Selects subsequent processing methods based on specific field conditions. These subsequent processing methods include Scrapy bidding and winning bid processing method, regular expression bidding and winning bid processing method, Scrapy bid opening processing method, and regular expression bid opening processing method. Specifically, the Scrapy bidding and winning bid processing method and the regular expression bidding and winning bid processing method are used to extract keyword information from bidding and winning bid projects, while the Scrapy bid opening processing method and the regular expression bid opening processing method are used to extract keyword information from bid opening projects.

[0078] Furthermore, the keyword extraction module also includes:

[0079] Scrapy bidding and tendering processing module;

[0080] Regular expression-based bidding processing module;

[0081] Scrapy bidding processing module;

[0082] Regular expression index opening processing module.

[0083] Furthermore, the scrapy bidding and tendering processing module includes:

[0084] Demand value acquisition unit: Using Scrapy's Selector class, the content of the details page is constructed into a selector object, and the demand value is obtained by processing and extracting the object information;

[0085] Table object list building unit: Use the methods of the selector object to retrieve all table tags and build a list of table objects;

[0086] Keyword information search unit: It retrieves the table object in a loop and passes it into a pre-set table processing model. The table processing model is used to determine the order of the keyword list, locate the table header, and locate the keyword position.

[0087] Furthermore, when the keyword information search unit finds keyword information, it passes the return value of the table processing model to the demand value processing model, which is used to extract demand values, determine demand values, and standardize demand values.

[0088] Furthermore, the regular expression bidding processing module includes:

[0089] Demand value acquisition unit: Expands the keywords in the keyword list into regular expression statements and combines them into a new keyword list. Through regular expression replacement and BeautifulSoup, the main text of the details page is processed into plain text information without HTML tags. By processing and extracting the text information, the demand value is obtained.

[0090] Keyword information search unit: Inputs all keyword lists into a pre-set text processing model, which is used to determine the order of the keyword list and locate the keyword positions;

[0091] Perform keyword search on text information.

[0092] Furthermore, when the keyword information search unit finds keyword information, it passes the return value of the table processing model to the demand value processing model, which is used to extract demand values, determine demand values, and standardize demand values.

[0093] Furthermore, the subsequent processing method also includes:

[0094] The Scrapy bidding and tendering processing unit first uses Scrapy processing methods to search for keyword information in the table object. If the search yields results, the results are output. If no results are found, the selector object's methods are used to extract the list of plain text information within the selector.

[0095] Starting Keyword Retrieval Unit: Retrieves plain text string elements from the plain text information list, searches for the starting keyword in the string element, and if the result is yes, passes the string and the ending keyword list together into the field processing function to determine if a value matching the requirements can be extracted. If the result is yes, outputs the value as the matching value for the field. If the result is no, the process is repeated.

[0096] Furthermore, if the list of plain text information is looped through without finding the required value, then keyword searching is performed using regular expression processing methods.

[0097] Furthermore, the scrapy index opening processing method includes the following steps:

[0098] Demand value acquisition unit: Using Scrapy's Selector class, the content of the details page is constructed into a selector object, and the demand value is obtained by processing and extracting the object information;

[0099] The unit for building a list of table objects. Use the methods of the selector object to locate all table tags and build a list of table objects;

[0100] Keyword Information Extraction Unit. This unit retrieves table objects from the table object list and places them into the table processing model. The table processing model is used to locate the company name, process each group of data horizontally row by row, map the required values ​​to the field names, and process the required value format. It outputs a list containing multiple groups of data, each group containing all required fields and their corresponding values.

[0101] Furthermore, the regular expression indexing module includes:

[0102] Demand value acquisition unit: Through regular expression replacement and BeautifulSoup, the main text of the details page is processed into plain text information without HTML tags, and the demand value is extracted by processing the text;

[0103] Keyword information extraction unit: Matches company name, divides text into several parts according to company name, and inputs the divided string and all field keywords into the processing model. The processing model is used to orderly divide all demand values, match demand values ​​with field names one by one, and process the demand value format.

[0104] Example 3

[0105] As shown in Figure 2, based on Examples 1-2, this example proposes a terminal device for extracting bidding keyword information based on regular expressions and Scrapy. The terminal device 200 includes at least one memory 210, at least one processor 220, and a bus 230 connecting different platform systems.

[0106] The memory 210 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 211 and / or cache memory 212, and may further include read-only memory (ROM) 213.

[0107] The memory 210 also stores a computer program that can be executed by the processor 220. This program enables the processor 220 to execute any of the above-described methods for extracting bidding keyword information based on regular expressions and Scrapy in this application embodiment. The specific implementation and technical effects are consistent with those described in the embodiments of the above methods, and some details will not be repeated here. The memory 210 may also include a program / utility 214 having a set (at least one) of program modules 215. Such program modules include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment.

[0108] Accordingly, processor 220 can execute the aforementioned computer program, as well as executable program / utility 214.

[0109] Bus 230 can represent one or more of several types of bus structures, including a memory bus or memory controller, peripheral bus, graphics acceleration port, processor, or a local bus using any of the various bus structures.

[0110] Terminal device 200 can also communicate with one or more external devices 240, such as keyboards, pointing devices, Bluetooth devices, etc., and with one or more devices capable of interacting with it, and / or with any device that enables it to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be performed via input / output (I / O) interface 250. Furthermore, terminal device 200 can also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapter 260. Network adapter 260 can communicate with other modules of terminal device 200 via bus 230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with terminal device 200, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0111] Example 4

[0112] Building upon Examples 1-2, this embodiment proposes a computer-readable storage medium for extracting bidding keyword information based on regular expressions and Scrapy. The computer-readable storage medium stores instructions that, when executed by a processor, implement any of the aforementioned methods for extracting bidding keyword information based on regular expressions and Scrapy. The specific implementation method and the achieved technical effects are consistent with those described in the embodiments above, and some details will not be repeated.

[0113] Figure 3 illustrates a program product 300 provided in this embodiment for implementing the above-described method. It may employ a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a terminal device, such as a personal computer. However, the program product 300 of this invention is not limited thereto. In this embodiment, the readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. The program product 300 may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0114] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on a user computing device, partially on a user device, as a standalone software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to user computing devices via any type of network, including local area networks (LANs) or wide area networks (WANs), or they can be connected to external computing devices (e.g., via the Internet using an Internet service provider).

[0115] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for extracting keyword information during bidding and tendering based on regular expressions and Scrapy, characterized in that, Includes the following steps: S1. Determine the parsing fields for the bidding and tendering projects; S2. Collect and organize the keyword list for the field. Based on the field requirements, create either an ordered keyword list or an unordered keyword list. The ordered keyword list is a list of keywords with a priority order, while the unordered keyword list is a list of keywords with a sibling relationship. The keyword list includes one or more of a starting keyword list and an ending keyword list. S3. Select a subsequent processing method based on the specific field conditions. The subsequent processing methods include the Scrapy bidding and winning method, the regular expression bidding and winning method, the Scrapy bidding and opening method, and the regular expression bidding and opening method. The Scrapy bidding and winning method and the regular expression bidding and winning method are used to extract keyword information from the bidding and winning projects, and the Scrapy bidding and opening method and the regular expression bidding and opening method are used to extract keyword information from the bidding and opening projects. The Scrapy bidding and winning method specifically includes the following sub-steps: S3011. Using the Scrapy Selector class, construct a selector object from the main text of the details page, and obtain the required value by processing and extracting the object information; S3012. Using the selector object's method, retrieve all table tags and construct a list of table objects; S3013. Loop through the table objects and pass them to a pre-set table processing model, which is used to determine the order of the keyword list, locate the table header, and locate the keyword position; S3014. Search for keyword information in the table objects; Step S3014 specifically includes the following sub-steps: S30141. When no keyword information is found, loop through step S3013 until all tables have been searched; S30142. When keyword information is found, pass the return value of the table processing model to the demand value processing model, which is used to extract demand values, determine demand values, and standardize demand values; The regular expression bidding processing method specifically includes the following sub-steps: S3021. Expand the keywords in the keyword list into regular expression statements and combine them into a new keyword list. Through regular expression replacement and BeautifulSoup, process the details page text into plain text information without HTML tags. By processing and extracting text information, obtain the demand value; S3022. All keyword lists are input into a pre-set text processing model, which is used to determine the order of the keyword list and locate the keyword positions; S3023. Keyword search is performed on the text information; Step S3023 further includes, when keyword information is found, inputting the return value of the text processing model into a demand value processing model, which is used to extract demand values, determine demand values, and standardize demand values; The subsequent processing method further includes: S3031.First, the Scrapy processing method is used to search for keyword information in the table object. If a result is found, it is output; otherwise, step S3032 is executed. S3032. If no result is found, the selector object's method is used to extract the plain text information list within the selector. S3033. Plain text string elements are retrieved from the plain text information list, and the starting keyword is checked against these elements. If the result is yes, step S3034 is executed; otherwise, step S3032 is executed repeatedly. S3034. The string and the ending keyword list are passed together to the field processing function to determine if a matching value can be extracted. If the result is yes, the value is output as the matching value for the field; otherwise, step S3032 is executed repeatedly. When step S3032 is executed repeatedly and the plain text information list has been looped through without finding a matching value, a keyword search is performed using regular expression processing. The Scrapy bidding processing method includes the following steps: S3041. Using Scrapy's Selector class, the main text of the details page is constructed into a selector object. The required values ​​are obtained by processing and extracting the object information. S3042. Using the selector object's methods, all table tags are located, and a list of table objects is constructed. S3043. Table objects are retrieved from the list and placed into a table processing model. This model is used to locate the company name, process each group of data horizontally row by row, and map the required values ​​to the field names, processing the required value format. S3044. A list containing multiple groups of data is output, each group containing all required fields and their corresponding values. When no matching results are found, keyword information is searched using regular expression keyword processing methods, specifically including the following steps: S3051. The main text of the details page is processed into plain text information without HTML tags using regular expression replacement and BeautifulSoup. The required values ​​are extracted by processing the text. S3052. The system matches company names, divides the text into several parts based on the company name, and inputs the divided strings along with all field keywords into a text processing model. This model is used to systematically divide all required values, map each required value to a field name, and process the required value format.

Images