A text processing method, apparatus, device and medium

Abstract

The application discloses a text processing method, device and equipment and medium. The method comprises the following steps: determining a target group to which a to-be-processed text line belongs in a plurality of groups. The plurality of groups are generated, and each group comprises one or more to-be-matched text lines. For example, the plurality of groups can be generated in advance according to an original resume provided by a user, and each group comprises different types of content, such as basic information, education experience, work experience and the like. Then, in the one or more to-be-matched text lines, a target text line matched with the to-be-processed text line is determined, and the to-be-processed text line is spliced with the target text line. Through the text processing method provided by the application, the text lines having a connection relationship can be spliced, the accuracy of entity recognition is improved, and therefore the accuracy of generating a structured resume is improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a text processing method, apparatus, device, and medium. Background Technology

[0002] Resume parsing can extract information from text resumes to create structured resumes with different sections, such as basic information and education. Structured resumes are easier to manage and use than text resumes. Considering that Word documents may contain garbled characters due to version incompatibility, most people convert their Word resumes to PDF or image format for submission.

[0003] Upon receiving a resume in PDF or image format, the initial step is to extract the text information using an OCR model. Subsequent named entity recognition (NAME) is then used to extract entities and segment the text. However, long text in Word automatically wraps, and after OCR extraction, the long text is displayed in its original format. This means a single line of text might become multiple unconnected lines, and a complete entity might appear on different lines, reducing the accuracy of subsequent entity recognition and thus affecting the accuracy of the generated structured resume. Summary of the Invention

[0004] In view of this, this application provides a text processing method, apparatus, device, and medium to improve the accuracy of entity recognition, thereby improving the accuracy of generating structured resumes.

[0005] To achieve the above objectives, the technical solution provided in this application is as follows:

[0006] In a first aspect of this application, a text processing method is provided, the method comprising:

[0007] Among multiple groups, the target group to which the text line to be processed belongs is determined. The multiple groups are predetermined, and each of the multiple groups includes one or more text lines to be matched.

[0008] Among the one or more text lines to be matched, determine the target text line that matches the text line to be processed;

[0009] The text line to be processed is concatenated with the target text line.

[0010] In a second aspect of this application, a text processing apparatus is provided, the apparatus comprising:

[0011] The first determining unit is configured to determine the target group to which the text line to be processed belongs among multiple groups, wherein the multiple groups are predetermined and each of the multiple groups includes one or more text lines to be matched;

[0012] The second determining unit is used to determine, among the one or more text lines to be matched, a target text line that matches the text line to be processed;

[0013] The splicing unit is used to splice the text line to be processed with the target text line.

[0014] In a third aspect of this application, an electronic device is provided, the device comprising: a processor and a memory;

[0015] The memory is used to store instructions or computer programs;

[0016] The processor is configured to execute the instructions or computer program in the memory to cause the electronic device to perform the method described in the first aspect.

[0017] In a fourth aspect of this application, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a device, cause the device to perform the method described in the first aspect.

[0018] In a fifth aspect of this application, a computer program product is provided, the computer program product comprising a computer program / instructions that, when executed by a processor, implement the method described in the first aspect.

[0019] Therefore, this application has the following beneficial effects:

[0020] In the above implementation of this application, to generate an accurate structured resume, the target group to which the text line to be processed belongs is first determined from multiple groups. These multiple groups are generated, and each group includes one or more text lines to be matched. For example, multiple groups can be generated based on the original resume provided by the user, each group including different types of content, such as basic information, education experience, work experience, etc. Then, among the one or more text lines to be matched, the target text line that matches the text line to be processed is determined, and the text line to be processed is concatenated with the target text line. Through the text processing method provided in this application, text entities with connecting relationships can be concatenated, improving the accuracy of entity recognition, thereby improving the accuracy of generating a structured resume. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a text processing method provided in an embodiment of this application;

[0023] Figure 2 A schematic diagram of a text processing device provided in an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0026] Upon receiving a resume in PDF or image format, the initial step is to extract the text information using an OCR model. Subsequent named entity recognition (NAME) is then used to extract entities and segment the text. However, long text in Word automatically wraps, and after OCR extraction, the long text is displayed in its original format. This means a single line of text might become multiple unconnected lines, and a complete entity might appear on different lines, reducing the accuracy of subsequent entity recognition and thus affecting the accuracy of the generated structured resume.

[0027] Based on this, embodiments of this application provide a text processing method to improve the accuracy of entity recognition, thereby improving the accuracy of generating structured resumes. Specifically, to generate an accurate structured resume, the target group to which the text line to be processed belongs is first determined from multiple groups. These multiple groups are generated, and each group includes one or more text lines to be matched. For example, multiple groups can be pre-generated based on the user-provided original resume, each group including different types of content, such as basic information, education experience, work experience, etc. Then, among the one or more text lines to be matched, the target text line that matches the text line to be processed is determined, and the text line to be processed is concatenated with the target text line. Through the text processing method provided by this application, text lines with a connection relationship can be concatenated, improving the accuracy of entity recognition, thereby improving the accuracy of generating structured resumes.

[0028] To facilitate understanding of the technical methods provided in the embodiments of this application, a detailed description will be given below in conjunction with the accompanying drawings.

[0029] See Figure 1 , Figure 1 This is a flowchart illustrating a text processing method provided in an embodiment of this application. The method can be executed by a text processing device, which can be an electronic device or other types of device. The method may include the following steps:

[0030] S101: Among multiple groups, determine the target group to which the text line to be processed belongs.

[0031] The multiple groups can be pre-generated, and each group includes one or more text lines to be matched, i.e., text lines that can be concatenated with the text lines to be processed. Optionally, upon receiving the original resume, the text processing device can recognize the text in the resume based on its content and extract the title-type text content for grouping. That is, all group types can be determined based on the titles in the resume, such as basic information, education experience, work experience, etc. Basic information can include name, gender, age, mobile phone number, etc. Education experience can include the time of university attended, major, and research projects participated in at school, etc. Work experience can include previous companies, positions held, and projects participated in, etc. In other words, each group corresponds to a semantic attribute, and then semantic recognition is performed on the text lines in the resume to determine the group to which each text line belongs. The text processing device can store a text recognition model, such as OCR, for recognizing the text in the resume.

[0032] When concatenating text lines, only lines within the same group can be concatenated. Therefore, it is necessary to predetermine each text line, including the group to which the text line to be processed belongs. In one possible implementation, determining the target group to which the text line belongs can be achieved through a semantic recognition model. This semantic recognition model is pre-trained so that it can accurately identify the semantics corresponding to the text lines. The training process of this semantic recognition model may include: after obtaining a set of sample text lines, determining the semantic attribute label for each sample text line in the set based on the semantic attributes corresponding to each group in multiple groups. For example, the semantics of each text line can be manually identified to determine the semantic attributes of the group to which the text line belongs, thereby determining a semantic attribute label for each sample text line. Then, the set of sample text lines with semantic attribute labels is input into the initial recognition model to obtain the semantic attribute output value. Based on the multiple semantic attribute labels corresponding to the sample text line set and the semantic attribute output value of the initial recognition model, the loss is determined. When the loss is greater than or equal to the threshold, it indicates that the accuracy of the initial recognition model does not meet the requirements. The parameters of the initial recognition model can be adjusted based on the loss, and the initial recognition model after the adjustment of the sample text line set and subsequent training processes can be re-executed until the loss is less than the threshold, indicating that the accuracy of the initial recognition model meets the requirements, thereby obtaining a well-trained semantic recognition model.

[0033] The text line to be processed is then input into the semantic recognition model to obtain the first semantic attribute corresponding to the text line. Among the multiple semantic attributes corresponding to multiple groups, the target semantic attribute that matches the first semantic attribute is determined, thus identifying the group corresponding to the target semantic attribute as the target group. Similarly, each text line in the original resume can be grouped using the above method, ensuring that each group contains one or more text lines. These are the text lines that can be concatenated with the text line to be processed in subsequent checks.

[0034] S102: Among one or more lines of text to be matched, identify the target text line that matches the text line to be processed.

[0035] Once the target group to which the text line to be processed belongs and the text lines to be matched included in that target group are determined, the target text line, that is, the text line that can be concatenated with the text line to be processed, can be determined from one or more text lines to be matched.

[0036] S103: Concatenate the text line to be processed with the target text line.

[0037] After obtaining the target text line, you can concatenate the text line to be processed with the target text line. The following sections will describe in detail the methods for determining the target text line and concatenating the text line to be processed with the target text line, depending on the specific circumstances.

[0038] In one possible implementation, text lines can be concatenated based on time entities within them. In practical applications, a single line of text in a resume typically contains a maximum of two time entities, such as the start and end dates of educational or work experience. Therefore, when two text lines each contain one time entity, it can be determined that the two text lines can be concatenated. Specifically, a time matching rule can be used to obtain the first time entity of the text line to be processed, as well as the text lines to be matched that include the second time entity. This time matching rule is determined based on time entities in different formats. For example, different formats of time entities can include "10 / 04 / 2022", "Dec 25,2021-May 17,2019", "07-08 / 2018", and "October 21, 2022 - November 21, 2022", etc. By training the time matching rule using multiple time entities in different formats, the time entity can be accurately identified. Then, the target text line is determined from the text lines to be matched that include the second time entity. Specifically, when the text line containing the second time entity to be matched is a single text line, the target text line can be determined. When there are multiple text lines containing the second time entity to be matched, the target text line can be determined based on the coordinate information of the multiple text lines and the coordinate information of the text line to be processed. For example, when the start and end times of educational experience are in two separate text lines, and the start and end times of work experience are also in two separate text lines, if the text line to be processed includes the end time of the work experience, there are three text lines containing time entities to be matched. Therefore, the target text line that is connected to the first time entity can be determined by combining the coordinate information of the multiple second time entities. For example, the line coordinates of the text line to be processed and the line coordinates of the text line containing the second time entity to be matched are adjacent lines.

[0039] Since two lines of text with a connection in the original resume are usually adjacent lines, optionally, the adjacent lines of the text line to be processed can be obtained, and the existence of a second time entity in the adjacent lines can be determined based on the time matching rules. If it exists, the text line to be processed can be concatenated with the adjacent lines of text based on the relationship between the first time entity and the second time entity.

[0040] Once the target text line containing the second time entity is identified, the text line to be processed can be concatenated with the target text line. Optionally, when the first time entity of the text line to be processed is earlier than the second time entity of the target text line, the end of the text line to be processed is concatenated with the beginning of the target text line. When the first time entity of the text line to be processed is later than the second time entity of the target text line, the end of the target text line is concatenated with the beginning of the text line to be processed.

[0041] In one possible implementation, text lines can be concatenated using a complete word lexicon. This lexicon contains multiple semantically complete words. Specifically, any one of one or more text lines to be matched can be retrieved, and its end is concatenated with the beginning of the text line to be processed. Then, multiple complete words are extracted from the concatenated text line based on the complete word lexicon. When a complete word is present at the concatenation point, it indicates that the original complete word is separated into two text lines, and the text line to be matched is identified as the target text line. The text line to be processed can then be concatenated to the end of the text line to be matched, forming semantically complete text. Alternatively, the end of the text line to be processed can be concatenated with the beginning of the text line to be matched, and multiple complete words can be extracted from the concatenated text line based on the complete word lexicon. When a complete word is present at the concatenation point, that text line is identified as the target text line. The text line to be processed can then be concatenated to the beginning of the text line to be matched, forming semantically complete text.

[0042] Taking one application scenario as an example, when the last part of the previous matching text line is "the job title is algorithm", and the beginning part of the text line to be processed is "engineer", after concatenating the previous matching text line with the text line to be processed, the complete word "algorithm engineer" can be extracted using the complete word lexicon. It is located at the concatenation of the two text lines, so the text line to be processed can be concatenated at the end of the previous matching text line.

[0043] Optionally, to improve the speed of extracting complete words from text lines by traversing a complete word lexicon, an AC automaton can be constructed using the complete word lexicon. Then, the AC automaton can be used to identify multiple complete words in the concatenated text line. For example, given n words and an article containing m characters, the AC automaton can be used to find out how many words appear in the article.

[0044] In one possible implementation, text lines can be concatenated based on conjunctions within them. Since two connected text lines in an original resume are usually adjacent, the presence of conjunctions in the lines above and below the text line to be processed can be used to determine the target text line. Specifically, the coordinates of the text line to be processed can be used to obtain the preceding and following lines. Then, it can be determined whether a conjunction exists in either the preceding or following line. If the preceding line ends with a predefined conjunction, it is considered the target text line; if the beginning of the text line to be processed contains a predefined conjunction, it is also considered the target text line. The text line to be processed can then be concatenated after the target text line. Alternatively, if the ending of the text line to be processed contains a predefined conjunction, the following line is considered the target text line; or, if the beginning of the following line contains a predefined conjunction, it is also considered the target text line. The text line to be processed can then be appended to the target text line. Preset conjunctions can include "and", "or", "and", etc.

[0045] In one possible implementation, after a complete text line is split into two, the original complete text lines have the same formatting information, and the content of the split second text line is usually shorter. Based on this, adjacent text lines to be matched can be obtained based on the coordinate information of the text line to be processed. When the adjacent text lines to be matched have the same formatting information as the text line to be processed, and the difference between the first coordinate of the adjacent text lines to be matched and the first coordinate of the text line to be processed is less than a first threshold, and the difference between the last coordinate of the adjacent text lines to be matched and the last coordinate of the text line to be processed is greater than a second threshold, the adjacent text lines to be matched can be determined as the target text lines. That is, the adjacent text lines to be matched can be concatenated with the text line to be processed. The specific concatenation method can be found in the above embodiments and will not be repeated here. Here, the first coordinate represents the left boundary coordinate of the text line, and the last coordinate represents the right boundary coordinate of the text line.

[0046] The text processing method provided in the above embodiments can concatenate text lines with connection relationships, improve the accuracy of entity recognition, and thus improve the accuracy of generating structured resumes.

[0047] Based on the above method embodiments, this application also provides a text processing apparatus. See also Figure 2 , Figure 2 This is a schematic diagram of a text processing device provided in an embodiment of this application.

[0048] The device 200 may include:

[0049] The first determining unit 201 is used to determine the target group to which the text line to be processed belongs among multiple groups, wherein the multiple groups are predetermined, and each of the multiple groups includes one or more text lines to be matched;

[0050] The second determining unit 202 is used to determine, among the one or more text lines to be matched, a target text line that matches the text line to be processed;

[0051] The splicing unit 203 is used to splice the text line to be processed with the target text line.

[0052] In one possible implementation, the second determining unit 202 is specifically used to obtain a first time entity of the text line to be processed based on a time matching rule, wherein the time matching rule is determined based on time entities of different formats; obtain a text line to be matched including a second time entity based on the time matching rule; and determine the target text line based on the text line to be matched including the second time entity.

[0053] In one possible implementation, the second determining unit 202 is specifically used to obtain any one of the one or more text lines to be matched; concatenate the end of the text line to be matched with the beginning of the text line to be processed; determine multiple complete words in the concatenated text line based on a complete word lexicon; and determine the text line to be matched as the target text line when a complete word is located at the concatenation point; or, concatenate the end of the text line to be processed with the beginning of the text line to be matched; determine multiple complete words in the concatenated text line based on the complete word lexicon; and determine the text line to be matched as the target text line when a complete word is located at the concatenation point.

[0054] In one possible implementation, the second determining unit 202 is specifically configured to obtain the previous line of text to be matched and the next line of text to be matched based on the coordinate information of the text line to be processed; when the end of the previous line of text to be matched is a preset conjunction, the previous line of text to be matched is determined to be the target text line; or, when the beginning of the text line to be processed is the preset conjunction, the previous line of text to be matched is determined to be the target text line; or, when the end of the text line to be processed is the preset conjunction, the next line of text to be matched is determined to be the target text line; or, when the beginning of the next line of text to be matched is the preset conjunction, the next line of text to be matched is determined to be the target text line.

[0055] In one possible implementation, the second determining unit 202 is specifically used to obtain adjacent text lines to be matched based on the coordinate information of the text line to be processed; when the adjacent text lines to be matched have the same format information as the text line to be processed, the difference between the first coordinate of the adjacent text lines to be matched and the first coordinate of the text line to be processed is less than a first threshold, and the difference between the last coordinate of the adjacent text lines to be matched and the last coordinate of the text line to be processed is greater than a second threshold, the adjacent text lines to be matched are determined to be the target text line.

[0056] In one possible implementation, the second determining unit 202 is specifically used to determine the target text line when the text line to be matched including the second time entity is a single text line; and to determine the target text line based on the coordinate information of the multiple text lines and the coordinate information of the text line to be processed when the text line to be matched including the second time entity is a multiple text line.

[0057] In one possible implementation, the second determining unit 202 is specifically used to concatenate the end of the text line to be processed with the beginning of the target text line when the first time entity of the text line to be processed is earlier than the second time entity of the target text line; and to concatenate the end of the target text line with the beginning of the text line to be processed when the first time entity of the text line to be processed is later than the second time entity of the target text line.

[0058] In one possible implementation, each of the plurality of groups corresponds to a semantic attribute. The first determining unit 201 is specifically used to input the text line to be processed into a semantic recognition model to obtain the first semantic attribute corresponding to the text line to be processed. The semantic recognition model is trained based on a set of sample text lines. Each sample text line in the set of sample text lines has a semantic attribute label, which is determined based on the plurality of semantic attributes corresponding to the plurality of groups. Among the plurality of semantic attributes corresponding to the plurality of groups, a target semantic attribute that matches the first semantic attribute is determined. The group corresponding to the target semantic attribute is determined as the target group.

[0059] In one possible implementation, the training process of the semantic recognition model includes:

[0060] The sample text line set is input into the initial recognition model to obtain semantic attribute output values;

[0061] The loss is determined based on the multiple semantic attribute labels corresponding to the sample text line set and the semantic attribute output values;

[0062] Based on the loss, the parameters of the initial recognition model are adjusted, and the process of inputting the sample text line set into the initial recognition model and subsequent training is repeated until the loss is less than the threshold, thereby obtaining the semantic recognition model.

[0063] In one possible implementation, the second determining unit 202 is specifically used to construct an AC automaton based on the complete word lexicon; and to use the AC automaton to identify multiple complete words in the concatenated text line.

[0064] In one possible implementation, the preset conjunctions include at least one of "and", "or", and "and".

[0065] The beneficial effects of the text processing apparatus provided in this application embodiment can be found in the above method embodiments, and will not be repeated here.

[0066] It should be noted that the specific implementation of each unit in this embodiment can be found in the relevant descriptions in the above method embodiments. The division of units in this application embodiment is illustrative and only represents a logical functional division; in actual implementation, there may be other division methods. The functional units in this application embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. For example, in the above embodiments, the processing unit and the sending unit can be the same unit or different units. The integrated unit can be implemented in hardware or as a software functional unit.

[0067] See Figure 3 This document illustrates a structural schematic diagram of an electronic device 300 suitable for implementing embodiments of this application. The terminal devices in these embodiments may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0068] like Figure 3As shown, the electronic device 300 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0069] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0070] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 309, or installed from a storage device 308, or installed from a ROM 302. When the computer program is executed by the processing device 301, it performs the functions defined in the methods of the embodiments of this application.

[0071] The electronic device provided in this application embodiment belongs to the same inventive concept as the method provided in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0072] This application provides a computer storage medium storing a computer program that, when executed by a processor, implements the method provided in the above embodiments.

[0073] It should be noted that the computer-readable medium described above in this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0074] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0075] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0076] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, enable the electronic device to perform the aforementioned methods.

[0077] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0078] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0079] The units described in the embodiments of this application can be implemented in software or hardware. The names of the units / modules do not necessarily limit the specific unit itself.

[0080] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0081] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0082] According to one or more embodiments of this application, a text processing method is provided, the method comprising:

[0083] Among multiple groups, the target group to which the text line to be processed belongs is determined. The multiple groups are predetermined, and each of the multiple groups includes one or more text lines to be matched.

[0084] Among the one or more text lines to be matched, determine the target text line that matches the text line to be processed;

[0085] The text line to be processed is concatenated with the target text line.

[0086] According to one or more embodiments of this application, determining a target text line that matches the text line to be processed among the one or more text lines to be matched includes:

[0087] The first time entity of the text line to be processed is obtained based on time matching rules, wherein the time matching rules are determined based on time entities of different formats;

[0088] Based on the time matching rules, obtain the text lines to be matched that include the second time entity;

[0089] The target text line is determined based on the text line to be matched, which includes the second time entity.

[0090] According to one or more embodiments of this application, determining a target text line that matches the text line to be processed among the one or more text lines to be matched includes:

[0091] Get any one of the one or more text lines to be matched;

[0092] The end of the text line to be matched is concatenated with the beginning of the text line to be processed. Multiple complete words in the concatenated text line are determined based on a complete word lexicon. When a complete word is present at the concatenation point, the text line to be matched is determined to be the target text line; or...

[0093] The end of the text line to be processed is concatenated with the beginning of the text line to be matched. Based on the complete word lexicon, multiple complete words in the concatenated text line are determined. When a complete word is located at the concatenation point, the text line to be matched is determined to be the target text line.

[0094] According to one or more embodiments of this application, determining a target text line that matches the text line to be processed among the one or more text lines to be matched includes:

[0095] Based on the coordinate information of the text line to be processed, obtain the previous line of text to be matched and the next line of text to be matched of the text line to be processed;

[0096] When the end of the preceding line of text contains a preset conjunction, the preceding line of text to be matched is determined to be the target line of text; or,

[0097] When the beginning of the text line to be processed is the preset conjunction, the text line to be matched in the previous line is determined to be the target text line; or,

[0098] When the end of the text line to be processed is the preset conjunction, the next line of text to be matched is determined to be the target text line; or,

[0099] When the beginning of the next matching text line is the preset conjunction, the next matching text line is determined to be the target text line.

[0100] According to one or more embodiments of this application, determining a target text line that matches the text line to be processed among the one or more text lines to be matched includes:

[0101] Based on the coordinate information of the text line to be processed, obtain the text lines to be matched that are adjacent to the text line to be processed;

[0102] When the adjacent text line to be matched has the same format information as the text line to be processed, the difference between the first coordinate of the adjacent text line to be matched and the first coordinate of the text line to be processed is less than a first threshold, and the difference between the last coordinate of the adjacent text line to be matched and the last coordinate of the text line to be processed is greater than a second threshold, the adjacent text line to be matched is determined to be the target text line.

[0103] According to one or more embodiments of this application, determining the text line to be matched, which includes a second time entity, based on the time matching rule, includes:

[0104] When the text to be matched, which includes the second time entity, is a text line, the text to be matched, which includes the second time entity, is determined to be the target text line;

[0105] When the text to be matched, including the second time entity, consists of multiple text lines, the target text line is determined based on the coordinate information of the multiple text lines and the coordinate information of the text line to be processed.

[0106] According to one or more embodiments of this application, concatenating the text line to be processed with the target text line includes:

[0107] When the first time entity of the text line to be processed is earlier than the second time entity of the target text line, the end of the text line to be processed is concatenated with the beginning of the target text line.

[0108] When the first time entity of the text line to be processed is later than the second time entity of the target text line, the end of the target text line is concatenated with the beginning of the text line to be processed.

[0109] According to one or more embodiments of this application, each of the plurality of groups corresponds to a semantic attribute, and determining the target group to which the text line to be processed belongs among the plurality of groups includes:

[0110] The text line to be processed is input into the semantic recognition model to obtain the first semantic attribute corresponding to the text line to be processed. The semantic recognition model is trained based on the sample text line set. Each sample text line in the sample text line set has a semantic attribute label. The semantic attribute label is determined based on the multiple semantic attributes corresponding to the multiple groups.

[0111] Among the multiple semantic attributes corresponding to the multiple groups, a target semantic attribute that matches the first semantic attribute is determined;

[0112] The group corresponding to the target semantic attribute is determined as the target group.

[0113] According to one or more embodiments of this application, the training process of the semantic recognition model includes:

[0114] The sample text line set is input into the initial recognition model to obtain semantic attribute output values;

[0115] The loss is determined based on the multiple semantic attribute labels corresponding to the sample text line set and the semantic attribute output values;

[0116] Based on the loss, the parameters of the initial recognition model are adjusted, and the process of inputting the sample text line set into the initial recognition model and subsequent training is repeated until the loss is less than the threshold, thereby obtaining the semantic recognition model.

[0117] According to one or more embodiments of this application, determining multiple complete words in the concatenated text line based on a complete word lexicon includes:

[0118] Construct an AC automaton based on the complete vocabulary corpus;

[0119] The AC automaton is used to identify multiple complete words in the concatenated text line.

[0120] According to one or more embodiments of this application, the preset conjunctions include at least one of "and", "or", and "and".

[0121] According to one or more embodiments of this application, a text processing apparatus is provided, the apparatus comprising:

[0122] The first determining unit is configured to determine the target group to which the text line to be processed belongs among multiple groups, wherein the multiple groups are predetermined and each of the multiple groups includes one or more text lines to be matched;

[0123] The second determining unit is used to determine, among the one or more text lines to be matched, a target text line that matches the text line to be processed;

[0124] The splicing unit is used to splice the text line to be processed with the target text line.

[0125] In one or more embodiments of this application, the second determining unit is specifically configured to obtain a first time entity of the text line to be processed based on a time matching rule, wherein the time matching rule is determined based on time entities of different formats; obtain a text line to be matched including a second time entity based on the time matching rule; and determine the target text line based on the text line to be matched including the second time entity.

[0126] In one or more embodiments of this application, the second determining unit is specifically used to obtain any one of the one or more text lines to be matched; concatenate the end of the text line to be matched with the beginning of the text line to be processed; determine multiple complete words in the concatenated text line based on a complete word lexicon; and determine the text line to be matched as the target text line when a complete word is located at the concatenation point; or, concatenate the end of the text line to be processed with the beginning of the text line to be matched; determine multiple complete words in the concatenated text line based on the complete word lexicon; and determine the text line to be matched as the target text line when a complete word is located at the concatenation point.

[0127] In one or more embodiments of this application, the second determining unit is specifically configured to obtain the previous line of text to be matched and the next line of text to be matched based on the coordinate information of the text line to be processed; when the end of the previous line of text to be matched is a preset conjunction, the previous line of text to be matched is determined to be the target text line; or, when the beginning of the text line to be processed is the preset conjunction, the previous line of text to be matched is determined to be the target text line; or, when the end of the text line to be processed is the preset conjunction, the next line of text to be matched is determined to be the target text line; or, when the beginning of the next line of text to be matched is the preset conjunction, the next line of text to be matched is determined to be the target text line.

[0128] In one or more embodiments of this application, the second determining unit is specifically used to obtain adjacent text lines to be matched based on the coordinate information of the text line to be processed; when the adjacent text lines to be matched have the same format information as the text line to be processed, the difference between the head coordinates of the adjacent text lines to be matched and the head coordinates of the text line to be processed is less than a first threshold, and the difference between the tail coordinates of the adjacent text lines to be matched and the tail coordinates of the text line to be processed is greater than a second threshold, the adjacent text lines to be matched are determined to be the target text line.

[0129] In one or more embodiments of this application, the second determining unit is specifically configured to determine the target text line when the text line to be matched including the second time entity is a single text line; and to determine the target text line based on the coordinate information of the multiple text lines and the coordinate information of the text line to be processed when the text line to be matched including the second time entity is a multiple text line.

[0130] In one or more embodiments of this application, the second determining unit is specifically configured to concatenate the end of the text line to be processed with the beginning of the target text line when the first time entity of the text line to be processed is earlier than the second time entity of the target text line; and to concatenate the end of the target text line with the beginning of the text line to be processed when the first time entity of the text line to be processed is later than the second time entity of the target text line.

[0131] In one or more embodiments of this application, each of the plurality of groups corresponds to a semantic attribute. The first determining unit is specifically configured to input the text line to be processed into a semantic recognition model to obtain the first semantic attribute corresponding to the text line to be processed. The semantic recognition model is trained based on a set of sample text lines. Each sample text line in the set of sample text lines has a semantic attribute label, which is determined based on the plurality of semantic attributes corresponding to the plurality of groups. Among the plurality of semantic attributes corresponding to the plurality of groups, a target semantic attribute that matches the first semantic attribute is determined. The group corresponding to the target semantic attribute is determined as the target group.

[0132] In one possible implementation, the training process of the semantic recognition model includes:

[0133] The sample text line set is input into the initial recognition model to obtain semantic attribute output values;

[0134] The loss is determined based on the multiple semantic attribute labels corresponding to the sample text line set and the semantic attribute output values;

[0135] Based on the loss, the parameters of the initial recognition model are adjusted, and the process of inputting the sample text line set into the initial recognition model and subsequent training is repeated until the loss is less than the threshold, thereby obtaining the semantic recognition model.

[0136] In one or more embodiments of this application, the second determining unit is specifically used to construct an AC automaton based on the complete word lexicon; and to use the AC automaton to identify multiple complete words in the concatenated text line.

[0137] In one or more embodiments of this application, the preset conjunctions include at least one of "and", "or", and "and".

[0138] According to one or more embodiments of this application, an electronic device is provided, the device comprising: a processor and a memory;

[0139] The memory is used to store instructions or computer programs;

[0140] The processor is configured to execute the instructions or computer program in the memory to cause the electronic device to perform the text processing method.

[0141] According to one or more embodiments of this application, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, which, when executed on a device, cause the device to perform the text processing method.

[0142] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems or apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0143] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0144] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0145] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0146] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A text processing method, characterized in that, The method includes: Among multiple groups, the target group to which the text line to be processed belongs is determined. The multiple groups are predetermined, and each of the multiple groups includes one or more text lines to be matched. Among the one or more text lines to be matched, determine the target text line that matches the text line to be processed; The text line to be processed is concatenated with the target text line; The step of determining the target text line that matches the text line to be processed among the one or more text lines to be matched includes: The first time entity of the text line to be processed is obtained based on time matching rules, wherein the time matching rules are determined based on time entities of different formats; Based on the time matching rules, obtain the text lines to be matched that include the second time entity; The target text line is determined based on the text line to be matched that includes the second time entity; or, Get any one of the one or more text lines to be matched; The end of the text line to be matched is concatenated with the beginning of the text line to be processed. Multiple complete words in the concatenated text line are determined based on a complete word lexicon. When a complete word is present at the concatenation point, the text line to be matched is determined to be the target text line; or... The end of the text line to be processed is concatenated with the beginning of the text line to be matched. Based on the complete word lexicon, multiple complete words are determined in the concatenated text line. When a complete word is located at the concatenation point, the text line to be matched is determined to be the target text line; or, Based on the coordinate information of the text line to be processed, obtain the previous line of text to be matched and the next line of text to be matched of the text line to be processed; When the end of the previous line of text to be matched is a preset conjunction, the previous line of text to be matched is determined to be the target text line; or, When the beginning of the text line to be processed is the preset conjunction, the text line to be matched in the previous line is determined to be the target text line; or, When the end of the text line to be processed is the preset conjunction, the next line of text to be matched is determined to be the target text line; or, When the first part of the next line of text to be matched is the preset conjunction, the next line of text to be matched is determined to be the target text line; or, Based on the coordinate information of the text line to be processed, obtain the text lines to be matched that are adjacent to the text line to be processed; When the adjacent text line to be matched has the same format information as the text line to be processed, the difference between the first coordinate of the adjacent text line to be matched and the first coordinate of the text line to be processed is less than a first threshold, and the difference between the last coordinate of the adjacent text line to be matched and the last coordinate of the text line to be processed is greater than a second threshold, the adjacent text line to be matched is determined to be the target text line.

2. The method according to claim 1, characterized in that, The step of determining the text line to be matched, which includes the second time entity, based on the time matching rule includes: When the text to be matched, which includes the second time entity, is a text line, the text to be matched, which includes the second time entity, is determined to be the target text line; When the text to be matched, including the second time entity, consists of multiple text lines, the target text line is determined based on the coordinate information of the multiple text lines and the coordinate information of the text line to be processed.

3. The method according to claim 1, characterized in that, The step of concatenating the text line to be processed with the target text line includes: When the first time entity of the text line to be processed is earlier than the second time entity of the target text line, the end of the text line to be processed is concatenated with the beginning of the target text line. When the first time entity of the text line to be processed is later than the second time entity of the target text line, the end of the target text line is concatenated with the beginning of the text line to be processed.

4. The method according to any one of claims 1 to 3, characterized in that, Each of the plurality of groups corresponds to a semantic attribute, and determining the target group to which the text line to be processed belongs among the plurality of groups includes: The text line to be processed is input into the semantic recognition model to obtain the first semantic attribute corresponding to the text line to be processed. The semantic recognition model is trained based on the sample text line set. Each sample text line in the sample text line set has a semantic attribute label. The semantic attribute label is determined based on the multiple semantic attributes corresponding to the multiple groups. Among the multiple semantic attributes corresponding to the multiple groups, a target semantic attribute that matches the first semantic attribute is determined; The group corresponding to the target semantic attribute is determined as the target group.

5. The method according to claim 4, characterized in that, The training process of the semantic recognition model includes: The sample text line set is input into the initial recognition model to obtain semantic attribute output values; The loss is determined based on the multiple semantic attribute labels corresponding to the sample text line set and the semantic attribute output values; Based on the loss, the parameters of the initial recognition model are adjusted, and the process of inputting the sample text line set into the initial recognition model and subsequent training is repeated until the loss is less than the threshold, thereby obtaining the semantic recognition model.

6. The method according to claim 1, characterized in that, The determination of multiple complete words in the concatenated text line based on a complete word lexicon includes: Construct an AC automaton based on the complete vocabulary corpus; The AC automaton is used to identify multiple complete words in the concatenated text line.

7. The method according to claim 1, characterized in that, The preset conjunctions include at least one of "and", "or", and "and".

8. A text processing device, characterized in that, The device includes: The first determining unit is used to determine the target group to which the text line to be processed belongs among multiple groups, wherein the multiple groups are predetermined and each of the multiple groups includes one or more text lines to be matched; The second determining unit is used to determine, among the one or more text lines to be matched, a target text line that matches the text line to be processed; A splicing unit is used to splice the text line to be processed with the target text line; The second determining unit is specifically used for: The first time entity of the text line to be processed is obtained based on time matching rules, wherein the time matching rules are determined based on time entities of different formats; Based on the time matching rules, obtain the text lines to be matched that include the second time entity; The target text line is determined based on the text line to be matched that includes the second time entity; or, Get any one of the one or more text lines to be matched; The end of the text line to be matched is concatenated with the beginning of the text line to be processed. Multiple complete words in the concatenated text line are determined based on a complete word lexicon. When a complete word is present at the concatenation point, the text line to be matched is determined to be the target text line; or... The end of the text line to be processed is concatenated with the beginning of the text line to be matched. Based on the complete word lexicon, multiple complete words are determined in the concatenated text line. When a complete word is located at the concatenation point, the text line to be matched is determined to be the target text line; or, Based on the coordinate information of the text line to be processed, obtain the previous line of text to be matched and the next line of text to be matched of the text line to be processed; When the end of the previous line of text to be matched is a preset conjunction, the previous line of text to be matched is determined to be the target text line; or, When the beginning of the text line to be processed is the preset conjunction, the text line to be matched in the previous line is determined to be the target text line; or, When the end of the text line to be processed is the preset conjunction, the next line of text to be matched is determined to be the target text line; or, When the first part of the next line of text to be matched is the preset conjunction, the next line of text to be matched is determined to be the target text line; or, Based on the coordinate information of the text line to be processed, obtain the text lines to be matched that are adjacent to the text line to be processed; When the adjacent text line to be matched has the same format information as the text line to be processed, the difference between the first coordinate of the adjacent text line to be matched and the first coordinate of the text line to be processed is less than a first threshold, and the difference between the last coordinate of the adjacent text line to be matched and the last coordinate of the text line to be processed is greater than a second threshold, the adjacent text line to be matched is determined to be the target text line.

9. An electronic device, characterized in that, The device includes: a processor and a memory; The memory is used to store instructions or computer programs; The processor is configured to execute the instructions or computer program in the memory to cause the electronic device to perform the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on the device, cause the device to perform the method according to any one of claims 1-7.

Images