Text extraction method and apparatus, electronic device and storage medium

Abstract

A text extraction method and apparatus, an electronic device, and a storage medium. The method comprises: reading configuration information for text extraction (S101), wherein the configuration information comprises an attribute mapping relationship; extracting text content from a code file, and comparing the text content with previously recorded text to determine whether the text content exists in the previously recorded text (S102); if the text content does not exist in the previously recorded text, using the attribute mapping relationship to generate text content attributes of the text content (S103); and recording the text content by means of the text content attributes (S104).

Description

Text extraction methods, devices, electronic equipment and storage media

[0001] Related applications

[0002] This application claims priority to Chinese patent application No. 2024109958170, filed on July 23, 2024, entitled “Text Extraction Method, Apparatus, Electronic Device and Storage Medium”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to the field of data processing, and in particular to a text extraction method, apparatus, electronic device, and storage medium. Background Technology

[0004] The content of code files differs significantly from written language. If text is contained within a code file, the text must first be extracted from the code file before any text processing, such as translation, can be performed.

[0005] Therefore, how to extract valid text from code files is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] In view of this, the purpose of this disclosure is to provide a text extraction method, apparatus, electronic device and storage medium that can specifically solve existing problems.

[0007] Based on the above objectives, in a first aspect, this disclosure proposes a text extraction method, comprising: reading configuration information for text extraction, wherein the configuration information includes attribute mapping relationships, the attribute mapping relationships including text content and corresponding text content attributes; extracting text content from a code file, comparing the text content with previously recorded text to determine whether text content exists in the previously recorded text; if text content does not exist in the previously recorded text, generating text content attributes of the text content using the attribute mapping relationships; and recording the text content using the text content attributes.

[0008] Secondly, a text extraction device is also provided, comprising: a reading unit configured to read configuration information for text extraction, wherein the configuration information includes an attribute mapping relationship, the attribute mapping relationship including text content and corresponding text content attributes; an extraction unit configured to extract text content from a code file and compare the text content with previously recorded text to determine whether text content exists in the previously recorded text; a generation unit configured to generate text content attributes of the text content using the attribute mapping relationship if no text content exists in the previously recorded text; and a recording unit configured to record the text content using the text content attributes.

[0009] Thirdly, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the method of the first aspect.

[0010] Fourthly, a computer-readable storage medium is also provided, on which a computer program is stored, the computer program being executed by a processor to implement the method of any one of the first aspects.

[0011] Fifthly, a computer program product is also provided, including a computer program, wherein the computer program is executed by a processor to implement any one of the first aspects.

[0012] Details of one or more embodiments of this application are set forth in the following drawings and description. Other features, objects, and advantages of this application will become apparent from the specification, drawings, and claims. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the disclosed drawings without creative effort.

[0014] Figure 1 is a flowchart of a text extraction method according to an embodiment of the present disclosure.

[0015] Figure 2 illustrates a method for generating configuration information according to an embodiment of this disclosure.

[0016] Figure 3a is another flowchart of a text extraction method according to an embodiment of the present disclosure.

[0017] Figure 3b is another flowchart of a text extraction method according to an embodiment of the present disclosure.

[0018] Figure 4 is a schematic diagram of a text extraction device according to an embodiment of the present disclosure.

[0019] Figure 5 is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure.

[0020] Figure 6 is a schematic diagram of a storage medium provided in an embodiment of this disclosure. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] In related technologies, there is a significant difference between the content of code files and the language they contain. If processing of the text in a code file is required, such as translation, the valid text content must be extracted beforehand. Therefore, how to efficiently and accurately extract text has become a pressing technical challenge in this field.

[0023] Figure 1 illustrates the text extraction method of this disclosure. In an embodiment of this disclosure, the method includes:

[0024] Step S101: Read the configuration information used for text extraction, wherein the configuration information includes attribute mapping relationships, and the attribute mapping relationships include text content and corresponding text content attributes.

[0025] In this embodiment, the entity executing the text extraction method can read configuration information. Specifically, this configuration information is used for text extraction. The configuration information is the information used when performing text extraction.

[0026] Text content attributes are the attributes to which text content belongs; they are the categories obtained by classifying text content. For example, if the text content is "Zhang San," the text content attributes could be "name," etc.

[0027] The aforementioned executing entity can be various electronic devices, such as terminals or servers.

[0028] Step S102: Extract text content from the code file and compare the text content with previously recorded text to determine whether the text content exists in the previously recorded text.

[0029] In this embodiment, the aforementioned execution entity can extract text content from the code file and compare the extracted text content with the previously recorded text content, i.e., the previously recorded text. By comparing the text content, it can be determined whether the extracted text content exists in the previously recorded text.

[0030] The previously recorded text refers to text content that has already been recorded. The aforementioned executing entity performs deduplication and duplicate checking on the extracted text content from the already recorded text content to avoid recording the same text content repeatedly.

[0031] Step S103: If there is no text content in the previously recorded text, then the attribute mapping relationship is used to generate the text content attribute of the text content.

[0032] In this embodiment, if the deduplication result shows that no identical text has been recorded, an attribute mapping relationship can be used to generate text content attributes for the text content. Specifically, the aforementioned executing entity can determine the text content attribute corresponding to the text content in the attribute mapping relationship, and use it as the text content attribute of the text content.

[0033] Optionally, if text content already exists in the previously recorded text, the text extraction step is terminated and the text extraction method is no longer executed.

[0034] Step S104: Record the text content through text content attributes.

[0035] In this embodiment, the execution entity can record text using various methods through text content attributes. For example, the execution entity can use the text content attribute as the key and the text content as the value to record the text content. Specifically, the text content can be recorded in JSON format.

[0036] This embodiment can generate text content attributes for text content and record the text content using these attributes. This clearly and explicitly indicates the recorded content, helping to improve the utilization efficiency of the recorded text. Furthermore, by checking for duplicates, it can avoid recording repetitive text, thereby avoiding redundancy and extracting valid text.

[0037] In some optional implementations of any embodiment of this disclosure, extracting text content from a code file includes: determining the folders under each file directory corresponding to the configuration information in the code file; and extracting text content from the files by scanning the files in each folder.

[0038] In these optional implementations, the execution entity can determine the file directories corresponding to the configuration information in the code file and locate the folders under these file targets. Within each folder of the file directory corresponding to the configuration information, the files are all extracted using that configuration information. The extracted text is the text content.

[0039] The aforementioned execution entity can scan files within a folder, rather than using individual files as the scanning unit as in related technologies. This expands the scanning scope and allows for the identification of text content within each file within the folder.

[0040] Figure 2 illustrates a method for generating configuration information according to an embodiment of the present disclosure. As shown in Figure 2, the steps for generating the configuration information include: step S201, configuring the attribute mapping relationship between text content attributes and text content; step S202, configuring deduplication information, which is used to indicate whether to perform the step of determining whether text content exists in previously recorded text; and step S203, generating configuration information including deduplication information and attribute mapping relationship.

[0041] In this embodiment, the execution entity can configure the attribute mapping relationship. Specifically, the attribute mapping relationship is a rule for generating text content attributes from text content. The deduplication information, also known as deduplication switch information, indicates whether to perform the deduplication step. Thus, the generated configuration information can include the aforementioned deduplication information and attribute mapping relationship.

[0042] Specifically, the configuration information can include not only deduplication information and attribute mapping relationships, but also the generated file type and the scanned targets. The generated file type refers to the format in which the text content will ultimately be recorded.

[0043] This embodiment allows for pre-configuration of the information used during text extraction, thereby facilitating smooth text extraction. Furthermore, by configuring attribute mapping relationships and deduplication information, the text content attributes can be determined and deduplication steps performed during the text extraction process, helping to ensure the accuracy of text extraction.

[0044] Optionally, the steps in the text extraction method after reading the configuration information used for text extraction are executed by calling the target plugin; the target plugin supports feature extensions.

[0045] The aforementioned execution entity can invoke the target plugin to execute the steps following the reading of configuration information used for text extraction in the text extraction method. Specifically, it invokes the target plugin to perform the following extraction steps: extract text content from the code file; compare the text content with previously recorded text to determine if the text content exists in the previously recorded text; if the text content does not exist in the previously recorded text, then use attribute mapping to generate text content attributes; and finally, record the text content using the text content attributes.

[0046] The target plugin here refers to a plugin capable of performing the extraction steps described above. The plugin's functionality is not limited to extracting text content; it may also include translating text content.

[0047] These optional implementations allow for text extraction via plugins, facilitating flexible configuration and decoupling of the extraction tool. Furthermore, these implementations support feature extensions, enriching plugin functionality and enhancing the accuracy of the text extraction process.

[0048] In some optional implementations of any embodiment of this disclosure, recording text content through text content attributes may include: obtaining the current timestamp, determining the filename used to record the text content based on the current timestamp, determining the index of the text content based on the text content attributes and the filename, and associating the record index with the text content.

[0049] In these optional implementations, the executing entity can obtain the current timestamp and determine the filename corresponding to the text content based on the current timestamp using various methods. For example, the filename can be the timestamp itself, or it can be the timestamp plus a random number.

[0050] The aforementioned execution entity can use various methods to determine the index of the text content, associate the record index, and the text content based on the text content attributes and filename. For example, the execution entity can use both the text content attributes and the filename as the index of the text content.

[0051] These implementations can generate filenames that include timestamps, and then generate an index of the text content to ensure that different indexes can be distinguished by timestamps.

[0052] Optionally, determining the index of the text content based on the text content attributes and filename may include: determining the index number in the recorded folder according to the order of the recorded text content; and generating an index that includes the text attribute content, filename, and index number.

[0053] Specifically, the numbering of each recorded text entry can be determined by incrementing the numerical value. Within the folder containing the recorded text, the index number can be determined by the sequential numbering of the recorded text entries. For example, the first record in the folder might be numbered 1, the second record 2, and so on. In essence, the index can be represented as key-filename+number.

[0054] These optional implementations can determine the index number by recording the text content in the correct order, which makes it easier to find the text content and avoids confusion caused by duplicate numbering in the same folder.

[0055] In some optional implementations of any embodiment of this disclosure, comparing text content with previously recorded text includes: filtering out annotation content in the text content, and comparing the filtered text content with previously recorded text.

[0056] In these optional implementations, the code file may contain comments. The execution entity mentioned above can filter out the comments, thus filtering out the main text and avoiding the extraction of useless text content.

[0057] In some optional implementations of any embodiment of this disclosure, the method further includes: obtaining at least one translated text of the recorded text content; generating a multilingual text set including the text content and at least one translated text, wherein the multilingual text set corresponds to the text content attribute; using a preset search function to replace the text content at the position of the text content in the code file; in response to receiving a language display instruction for the position, calling the preset search function to determine the multilingual text set corresponding to the text content attribute; searching for the language text indicated by the language display instruction in the multilingual text set; and outputting the language text to the target display device.

[0058] In these implementations, the aforementioned executing entity can translate the recorded text content into at least one language on its own device to obtain at least one translated text, or output the text content and receive at least one translated text returned by other electronic devices. In this way, multiple languages ​​can be obtained, namely the aforementioned text content and at least one translated text.

[0059] The aforementioned execution entity can use a lookup function to replace text content in the code file. Upon receiving a language display instruction that specifies the display of a particular language, it determines the set of multilingual characters corresponding to the aforementioned text content attributes, searches for that language within that set, and outputs it. Specifically, the final target device for the output is the display device for that language.

[0060] Figure 3a shows another flowchart of the text extraction method. Specifically, the configuration items to be generated in the configuration information may include the generated file type, key generation rules, scan target information, whether to deduplicate, and code replacement rules. The generated file type refers to the file type of the file obtained by recording the text content. The key generation rules refer to the generation of attribute mapping relationships. The scan directory information refers to the scan-related configuration information for files in folders under the scan directory. The code replacement rules refer to using a preset search function to replace the text content in the code file.

[0061] Afterwards, the aforementioned execution entity can scan the code file according to the configuration items, record the text content, and replace the text content in the code file based on the recorded text content using a preset search function.

[0062] Figure 3b shows another flowchart of the text extraction method. The execution entity can begin scanning for terms, i.e., scanning text content, in the code file. Then, the execution entity can read configuration information and call the target plugin to perform the following steps: file comparison (duplicate checking), key generation, and then generating the file location (index). The text content is recorded to generate new files. The generated new files can include JSON files or Excel spreadsheets. These new files can then be exported.

[0063] This disclosure provides a text extraction device for executing the text extraction method described in the above embodiments. As shown in FIG4, the device includes: a reading unit 401 configured to read configuration information for text extraction, wherein the configuration information includes attribute mapping relationships, and the attribute mapping relationships include text content and corresponding text content attributes; an extraction unit 402 configured to extract text content from a code file and compare the text content with previously recorded text to determine whether text content exists in the previously recorded text; a generation unit 403 configured to generate text content attributes of the text content using the attribute mapping relationships if no text content exists in the previously recorded text; and a recording unit 404 configured to record the text content using the text content attributes.

[0064] Optionally, the extraction unit 402 is further configured to extract text content from the code file in the following manner: determine the folders under each file directory corresponding to the configuration information in the code file; and extract the text content from the files by scanning the files in each folder.

[0065] Optionally, the steps for generating configuration information include: configuring the attribute mapping relationship between text content attributes and text content; configuring deduplication information, which is used to indicate whether to perform the step of determining whether text content exists in the previously recorded text; and generating configuration information including deduplication information and attribute mapping relationship.

[0066] Optionally, the steps in the text extraction method after reading the configuration information used for text extraction are executed by calling the target plugin; the target plugin supports feature extensions.

[0067] Optionally, the recording unit 404 is further configured to record text content by means of text content attributes in the following manner: obtain the current timestamp, determine the file name to be used for recording the text content based on the current timestamp; determine the index of the text content based on the text content attributes and the file name, and associate the record index with the text content.

[0068] Optionally, the recording unit 404 is further configured to perform the following steps: determining the index of the text content based on the text content attributes and file name; determining the index number in the recorded folder according to the recording order of the text content; and generating an index including text attribute content, file name, and index number.

[0069] Optionally, the method further includes: obtaining at least one translated text of the recorded text content; generating a multilingual text set including the text content and at least one translated text, wherein the multilingual text set corresponds to the text content attribute; using a preset lookup function to replace the text content at the position of the text content in the code file; in response to receiving a language display instruction for the position, calling the preset lookup function to determine the multilingual text set corresponding to the text content attribute; searching for the language text indicated by the language display instruction in the multilingual text set; and outputting the language text to the target display device.

[0070] The text extraction device and the text extraction method provided in the above embodiments of this disclosure are based on the same inventive concept and have the same beneficial effects as the methods used, run or implemented by the applications stored therein.

[0071] This disclosure also provides an electronic device corresponding to the text extraction method provided in the foregoing embodiments, for executing the aforementioned text extraction method. This disclosure is not limiting.

[0072] Please refer to Figure 5, which shows a schematic diagram of an electronic device provided by some embodiments of the present disclosure. As shown in Figure 5, the electronic device 50 includes: a processor 500, a memory 501, a bus 502, and a communication interface 503. The processor 500, the communication interface 503, and the memory 501 are connected via the bus 502. The memory 501 stores a computer program that can run on the processor 500. When the processor 500 runs the computer program, it executes the method provided by any of the foregoing embodiments of the present disclosure.

[0073] The memory 501 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 503 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0074] Bus 502 can be an ISA bus, PCI bus, or EISA bus, etc. Buses can be divided into address buses, data buses, control buses, etc. Memory 501 is used to store programs. After receiving execution instructions, processor 500 executes the programs. The text extraction method disclosed in any of the foregoing embodiments of this disclosure can be applied to processor 500, or implemented by processor 500.

[0075] The processor 500 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed through integrated logic circuits in the hardware of the processor 500 or instructions in software form. The processor 500 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this disclosure can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 501. The processor 500 reads the information in memory 501 and, in conjunction with its hardware, completes the steps of the above method.

[0076] The electronic device provided in this disclosure and the text extraction method provided in this disclosure are based on the same inventive concept and have the same beneficial effects as the methods they employ, operate, or implement.

[0077] This disclosure also provides a computer-readable storage medium corresponding to the text extraction method provided in the foregoing embodiments. Please refer to FIG6, which shows a computer-readable storage medium 60, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it will execute the text extraction method provided in any of the foregoing embodiments.

[0078] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0079] The computer-readable storage medium provided in the above embodiments of this disclosure and the text extraction method provided in the embodiments of this disclosure are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0080] It should be noted that:

[0081] In the foregoing text, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase “comprising one…” does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0082] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the disclosed technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this disclosure.

[0083] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0084] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for extracting text, comprising: reading configuration information for text extraction, wherein the configuration information comprises attribute mapping relationships, the attribute mapping relationships comprising text content and corresponding text content attributes; extracting text content from a code file, and comparing the text content with previously recorded text to determine whether the text content exists in the previously recorded text; if the text content does not exist in the previously recorded text, generating a text content attribute of the text content using the attribute mapping relationships; recording the text content by using the text content attribute. 2.The method of claim 1, wherein the extracting text content from a code file comprises: determining folders under each file directory corresponding to the configuration information in the code file; and extracting text content from files in each folder by scanning the files in the folder. 3.The method of claim 1, wherein the generating configuration information comprises: configuring attribute mapping relationships between text content attributes and text content; configuring deduplication information, the deduplication information being used to indicate whether to perform the step of determining whether the text content exists in the previously recorded text; and generating configuration information comprising the deduplication information and the attribute mapping relationships. 4.The method of claim 3, wherein steps after the reading configuration information for text extraction in the method for extracting text are performed by invoking a target plug-in, and the target plug-in supports function extension. 5.The method of claim 1, wherein the recording the text content by using the text content attribute comprises: obtaining a current timestamp, and determining a file name used to record the text content according to the current timestamp; determining an index of the text content according to the text content attribute and the file name, and recording the index and the text content. 6.The method of claim 5, wherein the determining an index of the text content according to the text content attribute and the file name comprises: determining a number of the index according to a recording order of the recorded text content in a recorded folder; and generating an index comprising the text attribute content, the file name and the number. 7.The method of claim 1, further comprising: obtaining at least one translated text of the recorded text content, generating a multilingual text set comprising the text content and the at least one translated text, the multilingual text set corresponding to the text content attribute; replacing the text content at a position of the text content in the code file using a preset search function; in response to receiving a language display instruction for the position, invoking the preset search function to determine a multilingual text set corresponding to the text content attribute, searching for a language text indicated by the language display instruction in the multilingual text set, and outputting the language text to a target display device. 8.A device for extracting text, comprising: reading a configuration information for text extraction, wherein the configuration information comprises an attribute mapping relationship, and the attribute mapping relationship comprises a text content and a corresponding text content attribute; extracting a text content from a code file, and comparing the text content with a previously recorded text to determine whether the text content exists in the previously recorded text; generating a text content attribute of the text content by using the attribute mapping relationship if the text content does not exist in the previously recorded text; recording the text content by using the text content attribute.

9. The apparatus of claim 8, wherein the extraction unit is further configured to extract the text content from the code file in the following manner: determining a folder under each file directory corresponding to the configuration information in the code file; extracting the text content from the file by scanning the file in each folder.

10. The apparatus of claim 8, wherein the generating of the configuration information comprises: configuring an attribute mapping relationship between the text content attribute and the text content; configuring a deduplication information for indicating whether the step of determining whether the text content exists in the previously recorded text is performed; generating the configuration information comprising the deduplication information and the attribute mapping relationship.

11. The apparatus of claim 10, wherein the steps after the reading of the configuration information for text extraction in the text extraction method are performed by invoking a target plug-in, and the target plug-in supports function extension.

12. The apparatus of claim 8, wherein the recording unit 404 is further configured to record the text content by using the text content attribute in the following manner: obtaining a current timestamp, and determining a file name used for recording the text content according to the current timestamp; determining an index of the text content according to the text content attribute and the file name, and recording the index and the text content in association.

13. The apparatus of claim 12, wherein the recording unit 404 is further configured to determine the index of the text content according to the text content attribute and the file name in the following manner: determining a number of the index according to a recording sequence of the recorded text content in a recorded folder; and generating an index comprising the text attribute content, the file name and the number.

14. The apparatus of claim 8, wherein the method further comprises: obtaining at least one translated text of the recorded text content, and generating a multilingual text set comprising the text content and the at least one translated text, the multilingual text set corresponding to the text content attribute; replacing the text content at a position of the text content in the code file by using a preset search function. In response to receiving the language display instruction for the position, the preset search function is invoked, a multilingual character set corresponding to the character content attribute is determined, a language character indicated by the language display instruction is searched in the multilingual character set, and the language character is output to a target display device.

15. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the computer program to implement the method of any one of claims 1-7.

16. A computer readable storage medium having stored thereon a computer program, the program being executed by a processor to implement the method of any one of claims 1-7.

Images