Method and system for extracting and standardizing file text information

Abstract

The invention discloses a file text information extraction and standardization method and system, and the method comprises the steps: scanning a received directory, and obtaining a file path and a file name list contained in the directory; reading the files in the directory in sequence, and extracting original text information in the files; processing the original text information and a preset question statement through a preset large model, obtaining original field information corresponding to the question statement from the original text information, and storing the original field information in a database; original field information needing to be standardized in the database is input into the preset large model for format standardization processing; and updating the fields in the database based on a format standardization processing result of the preset large model. According to the scheme, the file content can be automatically extracted from the file, and the field format in the file content is standardized, so that massive information and data information can be effectively managed and utilized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data processing, and in particular to a method and system for extracting and standardizing file text information. BACKGROUND

[0002] In the process of enterprise operation and development, with the continuous expansion of business scale and the continuous deepening of operation management, a large amount of unstructured or semi-structured data materials will be accumulated. These data usually exist in the form of electronic documents, such as business contracts, financial invoices, test reports, accident reports, etc. Traditionally, enterprises mainly rely on manual methods to read, understand and input these documents. This method not only has extremely low processing efficiency and high labor cost, but also inevitably has an error rate due to human negligence, and the data quality and consistency cannot be guaranteed.

[0003] With the development of information technology, rule-based document data extraction technology has emerged. This kind of technology usually relies on fixed format parsing, keyword matching, text analysis and regular expressions to automatically extract specific information (such as date, personnel, amount, event, etc.) in the text. This method improves the processing efficiency to a certain extent and has achieved preliminary application in relatively fixed format scenarios. For example, in the medical field, this technology is used to quickly process structured medical record forms to shorten patient waiting time.

[0004] However, the above-mentioned rule-based technology path has its inherent and difficult-to-overcome limitations. First, the formats of documents in the real world are different, and there are many expression forms for dates, such as “2025-01-01 12:13”, “2025 / 01 / 01 12:13”, “2025-01-01 12:13”, etc. Writing and maintaining corresponding parsing rules and regular expressions for each possible format variant will result in extremely complex program logic, large code size, difficult system maintenance and poor scalability. Second, when new formats or complex semantic contexts that are not covered by the rule library appear, the extraction accuracy of the system will decrease sharply, lacking the necessary robustness and flexibility.

[0005] In addition, the data extracted from documents of different sources and different formats is extremely prone to form “data islands” due to the non-uniformity of their expression methods. These non-standardized data cannot be directly and effectively associated and analyzed by business systems, seriously restricting the ability of enterprises to conduct deep data mining, intelligent analysis and scientific decision-making, and hindering the process of digital transformation.

[0006] Therefore, there is an urgent need in the art for a new technical solution to overcome the deficiencies of the existing technology, such as low efficiency, poor flexibility, limited accuracy and insufficient data standardization. SUMMARY

[0007] The embodiment of the present application provides a file text information extraction and standardization method and system, solves the technical problems that massive documents are processed by manual work, low efficiency and high error rate exist, and data islands are caused by the differences in data formats between files, and realizes automatic extraction of file contents and standardization processing on field formats in the file contents.

[0008] In a first aspect, the embodiment of the present application provides a file text information extraction and standardization method, characterized in that the method comprises: scanning a received directory to obtain a file path and a file name list contained in the directory; reading files in the directory in sequence and extracting original text information in the files; processing the original text information and a preset question sentence by a preset large model to obtain original field information corresponding to the question sentence from the original text information and store the original field information in a database; inputting original field information needing standardization in the database into the preset large model for format standardization processing; updating fields in the database based on the format standardization processing result of the preset large model.

[0009] Preferably, the step of reading files in the directory in sequence and extracting text information in the files comprises: identifying type information of the file; determining a corresponding text extraction program based on the type information; extracting text information from the file based on the text extraction program.

[0010] Preferably, the type information of the file at least includes one of a text file, a picture file, a composite file and an audio / video file, wherein: the format of the text file is at least one of a txt format, a word format and a wps format; the format of the picture file is at least one of a jpg format, a png format, a bmp format and a GIF format; the format of the composite file is at least one of a PDF format, an EPUB format, a DjVu format and a MOBI format the format of the audio / video file is at least one of an mp3 format, an mp4 format, a wmv format and an avi format.

[0011] Preferably, the processing of the original text information and the preset question sentence by the preset large model to obtain the original field information corresponding to the question sentence from the original text information and store the original field information into the database comprises: combining the original text information and the question sentence to form a first input prompt of the preset large model; processing the input prompt by the preset large model; analyzing the processing result of the preset large model to obtain structured original field information; storing the original field information into the corresponding field in the database.

[0012] Preferably, the original field information in the database that needs to be standardized is input into the preset large model for format standardization processing, comprising: reading all original field information that needs to be standardized from the database, and aggregating the original field information into a to-be-processed list; combining the to-be-processed list with a preset standardization instruction to construct a second input prompt of the preset large model; inputting the second input prompt into the preset large model to obtain a list of standardized field information; generating a mapping relationship between the list of standardized field information and the to-be-processed list.

[0013] Preferably, the field in the database is updated based on the format standardization processing result of the preset large model, comprising: associating the list of standardized field information with the original field information in the database based on the mapping relationship to generate a to-be-updated data set; iterating the to-be-updated data set to generate an update instruction of the database; updating the original field information that needs to be standardized in the database based on the update instruction.

[0014] In a second aspect, an embodiment of the present application provides a file text information extraction and standardization system, comprising: a file information extraction module, configured to scan a received directory, read files in the directory and extract original text information; a preset large model processing module, configured to process the original text information to obtain standardized field information corresponding to the file; a database module, configured to store the standardized field information output by the preset large model processing module.

[0015] Preferably, the preset large model processing module comprises: a field extraction unit configured to extract the original field information from the original text information based on the preset question sentence; a standardization unit configured to process the original field information to be standardized based on a preset standardization instruction to obtain standardized field information.

[0016] In a third aspect, an embodiment of the present application provides a computing device, the device comprising: a processor; a memory for storing processor-executable instructions; and the processor, when executing the executable instructions, implements the file text information extraction and standardization method as described above.

[0017] In a fourth aspect, an embodiment of the present application provides a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium comprising a computer program or instructions for storing, when the computer program or instructions are executed, the file text information extraction and standardization method as described above is implemented.

[0018] The one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: The technical solution of the embodiments of the present application extracts the original text information by reading and recognizing the text in the directory, extracts the original field information from the original text information, updates and replaces the original field information saved in the database after standardizing the original field information, solves the technical problems in the prior art that massive documents are processed by manual work, which has low efficiency and high error rate, and the differences in data formats between files result in "data islands", and realizes automatic extraction of file content for files, standardization of field formats in file content, further saves labor costs, and enables effective management and utilization of massive information and data. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0020] Figure 1 A flowchart of the file text information extraction and standardization method provided by the embodiments of the present application; Figure 2 A flowchart of sequentially reading files in a directory and extracting text information provided by the embodiments of the present application; Figure 3A flowchart of extracting original field information from original text information provided by the embodiment of the present application is shown in the figure. Figure 4 A flowchart of standardizing the original field information that needs to be standardized provided by the embodiment of the present application is shown in the figure. Figure 5 A flowchart of updating the database based on the standardized field information provided by the embodiment of the present application is shown in the figure. Figure 6 A structural diagram of the extraction and standardization system of file text information provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0022] The following describes some technologies related to the embodiments of the present application to help understanding, which should be considered as only exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Also, for clarity and conciseness, the description of some well-known functions and structures is omitted in the following description.

[0023] Reference Figure 1 As shown in the figure, the method for extracting and standardizing file text information provided by the embodiment of the present application comprises the following steps. S1, scanning the received directory to obtain the file path and file name list contained in the directory.

[0024] In the embodiments of the present application, the directory to be scanned needs to be determined before starting the scanning. In the embodiments, one exemplary way is to determine the directory to be scanned through the selection operation of the user, and another exemplary way is to determine the corresponding directory to be scanned according to the relevant information input by the user, such as news, catering, technology, etc. In the embodiments, the file path, file name list, file size, etc. can be obtained by scanning the directory.

[0025] S2, reading the files in the directory in sequence and extracting the original text information in the files.

[0026] In the embodiments of the present application, according to the file name list and the file storage path obtained in the above steps, all the files in the directory are read in sequence, and the original text information in the files is extracted. For example, the text information in the text file is directly extracted from the text file; the text information in the picture file is obtained through OCR technology; the voice information in the audio and video file is recognized through voice recognition technology, and is converted into text information for extraction.

[0027] Reference Figure 2 In the embodiments of the present application, the files in the directory are read in sequence, and the text information in the files is extracted, which is implemented according to the following steps: S21, recognizing the type information of the file; S22, determining the corresponding text extraction program based on the type information; S23, extracting the text information from the file based on the text extraction program.

[0028] In the embodiments of the present application, with the expansion of business scale and the deepening of operation, a large amount of information and data materials will be generated. These data often exist in various formats. For example, contracts, invoices and reports exist in the form of text, such as word, TXT and WPS text formats; for example, advertisements, promotional materials, lecture materials exist in the form of pictures, such as jpg, png, bmp and GIF picture formats; for example, PDF format can combine pictures and text in the same file; and promotional videos, lecture audios and other audio and video formats.

[0029] In the present embodiment, since the types of files in the directory are various, after reading the files, the types of the files need to be further recognized, the corresponding text extraction program is determined according to the recognized file type, and the text information is extracted from the file according to the text extraction program. For example: when the file is a text file, the corresponding text information is directly read from the text file when the original text information is extracted through the corresponding text extraction program. For example, for word files, wps files and txt files, the text content can be directly read from the file through the text extraction program. For example, when the file is a picture file, the text extraction program needs to recognize the text content of the picture file through OCR technology, and the recognized text content is extracted as text information. For example, when the file is an audio and video file, the text extraction program needs to recognize the voice information in the audio and video file through voice recognition technology, and then convert the recognized voice information into text content as text information for extraction.

[0030] S3, process the original text information and the preset question sentence through the preset large model, obtain original field information corresponding to the question sentence from the original text information, and store the original field information into a database.

[0031] In the embodiment of the application, after the extraction of the original text information of all the files in the directory is completed, further, the preset large model processes the original text through the preset question sentence, extracts the original field information corresponding to the question sentence from the original text, and saves the changed original field information into the database.

[0032] Reference Figure 3 As shown in the embodiment of the application, the original text information is processed by the preset large model and the preset question sentence to obtain the original field information corresponding to the question sentence from the original text information and store the original field information into a database, which is realized according to the following steps: S31, combine the original text information and the question sentence to form a first input prompt of the preset large model.

[0033] In the embodiment of the application, the extracted original text information and the question sentence need to be spliced together first to obtain a complete prompt ready to be sent to the preset large model. For example: Original text information: "On October 28, 2025, at 9:15 am, at the intersection of Science Road and Innovation Avenue in the city's high-tech zone, an incident occurred..."

[0034] Preset question sentence: "Please extract the following information: event occurrence time: [], location: [], facility vehicle license plate number: []".

[0035] The spliced and combined prompt information is: "Please extract the following information: event occurrence time: [], location: [], facility vehicle license plate number: []\n\nText: On October 28, 2025, at 9:15 am, at the intersection of Science Road and Innovation Avenue in the city's high-tech zone, an incident occurred..."

[0036] S32, process the input prompt through the preset large model.

[0037] In the embodiment of the application, after obtaining the prompt information according to the above steps, the API or local interface of the preset large model is called, the prepared prompt information in the last step is input, and the returned text of the preset large model is received, for example: "Event occurrence time: October 28, 2025, at 9:15 am, location: intersection of Science Road and Innovation Avenue in the city's high-tech zone, facility vehicle license plate number: ShanAZ6668F".

[0038] S33. Analyze the processing result of the preset large model to obtain structured original field information.

[0039] In an embodiment of the present application, structured data objects (such as dictionaries, JSON objects, or arrays, etc.) are obtained by analyzing the text feedback by the preset large model. Specifically, the unstructured answer is split into independent data values that can be used by the program. An exemplary structured data after parsing is as follows: “json { "Incident occurrence time": "Around 9:15 am, October 28, 2025", "Location": "Intersection of Keji Road and Innovation Avenue, High-tech Zone, City", "License plate number of the vehicle or facility": "Shaanxi A Z6668F" }”.

[0040] S34. Store the original field information into the corresponding fields in the database.

[0041] In an embodiment of the present application, by executing the INSERT or UPDATE operation of SQL, each field value is stored into the corresponding column of the database. For example, the executed SQL logic: “INSERT INTO Accident Record Table (Time, Location, License Plate Number) VALUES ('Around 9:15 am, October 28, 2025', 'Intersection of Keji Road and Innovation Avenue, High-tech Zone, City', 'Shaanxi A Z6668F');”, a new record or an updated record is added to the database.

[0042] In an exemplary implementation: The extracted original text information is: Around 9:15 am, October 28, 2025, at the intersection of Keji Road and Innovation Avenue, High-tech Zone, City, a traffic accident occurred where a motor vehicle collided with a non-motor vehicle.

[0043] It was the morning rush hour at that time. Zhang was driving a red Toyota Camry sedan (license plate number: Shaanxi A Z6668F) and was traveling from east to west on Keji Road to the intersection of Innovation Avenue. When encountering a red light signal, he stopped to wait. After the green light came on, when the driver of the car started, he didn't pay attention to observing the right side and collided with an electric bicycle that passed through the intersection against the red light from south to north. The driver of the electric bicycle, Mr. Li (45 years old), was knocked to the ground, suffering minor abrasions, and the front part of the electric bicycle was severely damaged.

[0044] After the accident, the Toyota driver immediately got out of the car to check the situation and called 122 (the traffic accident reporting hotline) and 120 (the emergency medical services number). At the same time, after ensuring safety, he placed a warning triangle 50 meters behind the vehicle and took photos of the scene as evidence, including key evidence such as the collision location, skid marks, and the status of traffic lights.

[0045] Paramedics arrived quickly and conducted a preliminary examination and treatment on Mr. Zhang, confirming that his injuries were minor and did not require hospitalization. Subsequently, traffic police arrived at the scene and conducted an on-site investigation, including measuring the accident scene, drawing a diagram, and interviewing the parties involved and witnesses.

[0046] According to the relevant provisions of the Road Traffic Safety Law, after investigation and evidence collection, the traffic police department determined that Mr. Zhang, the electric bicycle rider, violated traffic signals, which was the main cause of the accident, and he should bear the primary responsibility (70%). Mr. Li, the car driver, failed to carefully observe the road conditions on the right side when starting, and bears the secondary responsibility (30%) for the accident.

[0047] The traffic police prepared a "Road Traffic Accident Determination Certificate (Simplified Procedure)" on the spot. Both parties had no objection to the determination of responsibility and signed the certificate to confirm it.

[0048] After the accident liability was determined, both parties jointly requested the traffic police to mediate the compensation claim. Through mediation, they reached an agreement: Mr. Li, the car driver, would bear 30% of the 800 yuan repair cost for the electric bicycle, amounting to 240 yuan; Mr. Zhang, the electric bicycle driver, would bear the remaining 70%. Both parties fulfilled the compensation agreement on the spot.

[0049] Subsequently, Mr. Li contacted the insurance company, reported the incident, and submitted the relevant claim documents. Based on the accident liability determination and compensation agreement, the insurance company quickly completed the claim processing.

[0050] Although the accident caused some property damage, it was handled quickly and properly by both parties who promptly called the police, protected the scene, and cooperated with the investigation, preventing traffic congestion and secondary accidents.

[0051] The preset query is: "Output the time, location, facility / vehicle license plate number, vehicle brand, event type, and processing result of the event. The output format should be as follows:" Time of the incident: Place: Vehicle license plate number for facility: Vehicle brand: Event type: Processing result:

[0052] The pre-defined large model identifies the original text information through the information in the question statement. For example, based on the "event occurrence time" in the question statement, it searches for date-related information in the original text information and finally obtains the original field information of the corresponding question statement "event occurrence time" "approximately 9:15 AM on October 28, 2025". Furthermore, the original field information is stored in the database.

[0053] S4. Input the original field information that needs to be standardized from the database into the preset large model for format standardization processing.

[0054] In the embodiments of this application, after extracting the original fields from all the original text information, all the original field information is identified, and the original field information that needs to be standardized is screened. For example, by using a predefined list of standardized fields, all the original field information is identified to determine the original field information that needs to be standardized; by analyzing the diversity of data formats, the original field information that needs to be standardized is determined; by using a large model to analyze the standardization requirements of the original field information, the original field information that needs to be standardized is determined; by using a multi-strategy comprehensive identification system to comprehensively detect the standardization requirements of all original fields in the database, the original field information that needs to be standardized is determined; and by using a standardization configuration management system to detect all the original field information in the database, the original field information that needs to be standardized is determined. Further, the original field information that needs to be standardized is input into a preset large model for format standardization processing.

[0055] refer to Figure 4 As shown in the embodiment of this application, the original field information that needs to be standardized in the database is input into the preset large model for format standardization processing, which is completed according to the following steps: S41. Read all the original field information that needs to be standardized from the database, and aggregate the original field information into a list to be processed.

[0056] In this embodiment, an SQL query is executed to select the values ​​of all records from the target column of the target table in the database. Specifically, by querying the database table name and the field name that needs to be standardized, such as the "time field", a set containing all the original field information of that field is obtained.

[0057] For example, executing the SQL query "SELECT Incident Time FROM Accident Record Table" will return the following set: "["January 1, 2025, 12:13 PM", "2025 / 01 / 01 12:13", "2025-01-01 12:13", "2025_01_0112", ...]".

[0058] S42. Combine the list to be processed with preset standardized instructions to construct the second input prompt of the preset large model.

[0059] In the embodiments of this application, the original field information list obtained in the previous step is concatenated with an instruction template describing the target format to obtain a complete prompt for format standardization.

[0060] For example, the original field information list is: "January 1, 2025, 12:13 PM, 2025 / 01 / 01 12:13, 2025-01-01 12:13, 2025_01_01 12".

[0061] Standardized instruction: "Please convert all the following times to 'YYYY / MM / DD HH:MM' format. If only the date is provided, please supplement it with '00:00' or '12:00' based on the context such as 'AM' or 'PM'. Please output the results directly, separated by commas. Input:".

[0062] The combined message indicates that the instruction and the original field information have been concatenated. S43. Input the second input prompt into the preset large model to obtain a list of standardized field information.

[0063] In the embodiments of this application, the above-mentioned prompts are passed to the preset large model for processing by calling the API. The preset large model returns standardized field information that is formatted as required and connected by delimiters.

[0064] For example, the standardized field information output by this preset large model is "2025 / 01 / 01 12:13, 2025 / 01 / 01 12:13, 2025 / 01 / 01 12:13, 2025 / 01 / 01 12:00".

[0065] S44. Generate a mapping relationship between the list of standardized field information and the list to be processed.

[0066] In the embodiments of this application, the string of standardized field information returned by the preset large model is split by a delimiter (such as a comma) to generate a list of standardized field information.

[0067] For example: ["2025 / 01 / 01 12:13", "2025 / 01 / 01 12:13", "2025 / 01 / 01 12:13", "2025 / 01 / 01 12:00"].

[0068] S5. Update the fields in the database based on the format standardization processing results of the preset large model.

[0069] In the embodiments of this application, after processing the original field information through the above steps, standardized field information corresponding to the original field information is obtained. By executing a data update instruction, the original field information stored in the database is updated using this standardized field information. In the embodiments of this application, in order to facilitate the traceability of the original information, a new field can be added to the database to store the standardized field information and establish an index relationship with the original field information. This ensures that the standardized field information can be used directly and that the original field information can be traced through the standardized field information, thereby ensuring the accuracy of the information.

[0070] refer to Figure 5 As shown, this embodiment of the application updates the fields in the database based on the format standardization processing result of the preset large model, and implements this according to the following steps: S51. Based on the mapping relationship, associate the list of standardized field information with the original field information in the database to generate a dataset to be updated.

[0071] In this embodiment, the unstructured or semi-structured text returned by the preset large model is processed to extract the specific standardized field information that needs to be updated, and it is ensured that this standardized field information can correspond to the correct database records and columns. For example: Large model output: "Event time: 2025 / 01 / 01 12:13, Vehicle brand: Toyota" After parsing and mapping: The system recognizes that the "Event Time" field needs to be updated to "2025 / 01 / 01 12:13" and the "Vehicle Brand" field needs to be updated to "Toyota", and knows that these updates are for the record with ID 105 that is currently being processed.

[0072] S52. Traverse the dataset to be updated and generate update instructions for the database.

[0073] In the embodiments of this application, one or more database operation commands, such as one or more database update operation instructions, are constructed using the data obtained in the previous step.

[0074] S53. Update the original field information in the database that needs to be standardized based on the update instruction.

[0075] In the embodiments of this application, by executing one or more database update operation instructions constructed above, the corresponding fields in the database are updated based on the dataset to be updated, and one or more database update records are formed.

[0076] refer to Figure 6 As shown in the figure, an embodiment of this application provides a system for extracting and standardizing text information from documents, including: a text information extraction module, a preset large model processing module, and a database module.

[0077] In the embodiments of this application, the system uses a text information extraction module to scan the directory selected by the user or to determine the corresponding directory based on relevant information input by the user, such as news, catering, technology, etc., to obtain information such as file path, file name list, and file size. Then, it reads all files in the directory in sequence and extracts the original text information in the files.

[0078] Furthermore, the original text information and the preset question statements are uploaded to the preset large model processing module. The field extraction unit in the preset large model processing module extracts the corresponding original field information from the original text information according to the preset question statements and stores the original field information in the database module. After the original field information of all original text information is extracted, the standardization unit of the preset large model processing module further identifies the original field information stored in the database module, performs standardization processing on the original field information that needs to be standardized, and obtains standardized field information. The original field information stored in the database module is then updated and replaced according to the standardized field information.

[0079] This application embodiment also provides a computing device, the device including: a processor; a memory for storing processor-executable instructions; when the processor executes the executable instructions, it implements the file text information extraction and standardization method described above.

[0080] This application also provides a non-volatile computer-readable storage medium, which includes a computer program or instructions for storing a computer program or instructions that, when executed, enable the extraction and standardization method of the file text information described above to be implemented.

[0081] While this application provides the method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive labor. The order of steps listed in this embodiment is merely one possible execution order among many and does not represent the only execution order. In actual device or client product execution, the methods shown in this embodiment or the accompanying drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0082] Some modules in the apparatus described in this application can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0083] The apparatus or module described in the above embodiments can be implemented by a computer chip or physical entity, or by a product with a certain function. For ease of description, the above apparatus is described by dividing it into various modules according to their functions. When implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware. Of course, a module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0084] The methods, apparatus, or modules described in this application can be implemented in a computer-readable program code manner. The controller can be implemented in any suitable manner, such as a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of a memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code manner, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included within it for implementing various functions can also be considered as structures within the hardware component. Alternatively, the device used to implement various functions can be viewed as either a software module that implements the method or a structure within a hardware component.

[0085] Furthermore, in the various embodiments of the present invention, each functional module can be integrated into a processing module, or each module can exist independently, or two or more modules can be integrated into a single module.

[0086] The aforementioned storage media include, but are not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), Cache, Hard Disk Drive (HDD), or Memory Card. The memory can be used to store computer program instructions.

[0087] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, or it can be embodied in the process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0088] The various embodiments described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. All or part of this application can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, mobile communication terminals, multiprocessor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0089] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of this application.

Claims

1. A method for extracting and standardizing text information from a document, characterized in that, The method includes: The received directory is scanned to obtain a list of file paths and file names contained in the directory; Read the files in the directory sequentially and extract the original text information from the files; The original text information and the preset question statement are processed by a preset large model, and the original field information corresponding to the question statement is obtained from the original text information and stored in the database. The original field information that needs to be standardized in the database is input into the preset large model for format standardization processing; The fields in the database are updated based on the format standardization processing results of the preset large model.

2. The method for extracting and standardizing document text information according to claim 1, characterized in that, The step of sequentially reading files in the directory and extracting text information from the files includes: Identify the type information of the file; The corresponding text extraction program is determined based on the type information; The text extraction program extracts text information from the file.

3. The method for extracting and standardizing document text information according to claim 2, characterized in that, The file type information includes at least one of the following: text file, image file, composite file, and audio / video file: wherein, The text file must be in at least one of the following formats: txt, word, or wps. The image file format is at least one of the following: jpg, png, bmp, and GIF formats; The format of the composite file is at least one of the following: PDF, EPUB, DjVu, and MOBI formats; The audio and video files must be in at least one of the following formats: mp3, mp4, wmv, and avi.

4. The method for extracting and standardizing document text information according to claim 1, characterized in that, The step of processing the original text information and the preset question statement using a preset large model, and obtaining the original field information corresponding to the question statement from the original text information and storing it in the database, includes: The original text information is combined with the question statement to form the first input prompt of the preset large model; The input prompts are processed using the preset large model; The processing results of the preset large model are parsed to obtain structured raw field information; The original field information is stored in the corresponding field in the database.

5. The method for extracting and standardizing document text information according to claim 1, characterized in that, The original field information that needs to be standardized in the database is input into the preset large model for format standardization processing, including: Read all the original field information that needs to be standardized from the database, and aggregate the original field information into a list to be processed; The list of tasks to be processed is combined with preset standardized instructions to construct the second input prompt of the preset large model; Input the second input prompt into the preset large model to obtain a list of standardized field information; Generate a mapping relationship between the list of standardized field information and the list to be processed.

6. The method for extracting and standardizing document text information according to claim 5, characterized in that, The fields in the database are updated based on the format standardization processing results of the preset large model, including: Based on the mapping relationship, the list of standardized field information is associated with the original field information in the database to generate a dataset to be updated; The dataset to be updated is traversed to generate update instructions for the database; The update instruction is used to update the original field information in the database that needs to be standardized.

7. A system for extracting and standardizing text information from documents, characterized in that, The system includes: The file information extraction module is used to scan the received directory, read the files in the directory, and extract the original text information. A pre-defined large model processing module is used to process the original text information to obtain the standardized field information corresponding to the file; The database module is used to store the standardized field information output by the preset large model processing module.

8. The document text information extraction and standardization system according to claim 7, characterized in that, The preset large model processing module includes: The field extraction unit processes the original text information and the preset question statement to obtain the corresponding original field information; The standardization unit processes the original field information that needs to be standardized based on preset standardization instructions to obtain standardized field information.

9. A computing device, characterized in that, include: processor; Memory used to store processor-executable instructions; When the processor executes the executable instructions, it implements the method for extracting and standardizing file text information as described in any one of claims 1 to 6.

10. A storage medium, characterized in that, Includes a method for storing computer programs or instructions that, when executed, enable the extraction and standardization of document text information as described in any one of claims 1 to 6.

Images