Data annotation system with fast system and slow system cooperatively working

Abstract

The embodiment of the invention relates to a data annotation system with fast and slow systems cooperatively working. The system comprises a data preprocessing module, a slow system, a fast system and an annotation output module, the data preprocessing module is used for performing image / chart content and text content splitting on the scientific literature file, sending an image sequence and a literature text to the slow system, and sending the image sequence to the fast system; the slow system is used for performing annotation suggestion reasoning according to the image sequence and the literature text and outputting high-dimensional image-text fusion features to the fast system in the reasoning process; the fast system is used for carrying out labeling task processing according to the image sequence and the high-dimensional image-text fusion features output by the slow system; and the annotation output module is used for outputting a literature annotation report. According to the method and the device, the labeling efficiency can be ensured, and the labeling quality can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a data labeling system in which fast and slow systems work in coordination. Background Art

[0002] In the application field of artificial intelligence technology, document data annotation is a common application scenario. At present, most of them use some lightweight small models (such as some customized machine learning models, deep learning models, etc.) to handle data annotation tasks according to the recognition / extraction method of entity triples (entity text, entity type, entity confidence). The advantage of this small model processing mechanism is fast annotation speed, but the disadvantage is poor model generalization and insufficient annotation accuracy.

[0003] Large models (such as large language models, cross-modal / multimodal large language models, etc.) have strong context understanding and chain-of-thought (COT) reasoning capabilities, and the model generalization is good. The advantage of processing annotation tasks based on large models is that the annotation accuracy can be improved, but the disadvantage is that if only relying on large models for full-text annotation, the computational complexity is too large and the annotation speed is slow.

[0004] In order to improve the quality of annotation while ensuring the efficiency of annotation, we provide a collaborative work framework based on the combination of large and small models. Under this collaborative work framework, on the one hand, the understanding ability of the large model can be used to enhance the recognition features of the small model, and the small model can complete the rapid annotation. On the other hand, while the small model is processing the annotation task, the thinking chain reasoning ability of the large model can be used to give a reasoning explanation of the annotation process. Based on this collaborative work framework, both the annotation efficiency and the quality of annotation can be improved. The present invention is also a specific implementation scheme of this collaborative work framework. Summary of the invention

[0005] The purpose of the present invention is to provide a data annotation system in which a fast and slow system work in coordination with each other in view of the defects of the prior art. The system comprises: a data preprocessing module, a slow system, a fast system and an annotation output module; wherein the data preprocessing module is connected to the slow system and the fast system respectively; the slow system is connected to the fast system; and the annotation output module is connected to the slow system and the fast system respectively. The data preprocessing module is used to split the image / chart content and the text content of the input scientific literature file to obtain the corresponding image sequence and the text of the literature, and send the image sequence to the slow system, and send the image sequence to the fast system; the slow system is used to perform annotation suggestion reasoning based on the image sequence and the text of the literature to obtain the annotation suggestion description and send it to the annotation output module, and output the high-dimensional image-text fusion features to the fast system during the processing; the fast system is used to perform annotation task processing based on the image sequence and the high-dimensional image-text fusion features output by the slow system to obtain the annotation entity sequence and send it to the annotation output module; the annotation output module is used to form the annotation suggestion description and the annotation object sequence into a corresponding literature annotation report and output it. The fast and slow system collaborative working framework of the present invention can ensure the annotation efficiency and improve the annotation quality.

[0006] To achieve the above-mentioned purpose, an embodiment of the present invention provides a data annotation system in which a fast and slow system work in coordination, the system comprising: a data preprocessing module, a slow system, a fast system and an annotation output module;

[0007] The data preprocessing module is connected to the slow system and the fast system respectively; the slow system is connected to the fast system; the annotation output module is connected to the slow system and the fast system respectively;

[0008] The data preprocessing module is used to receive an input scientific literature file; and to separate the image / chart content and the text content of the scientific literature file to obtain a corresponding first image sequence and a first literature text; and to send the first image sequence and the first literature text to the slow system; and to send the first image sequence to the fast system;

[0009] The slow system is used to perform annotation suggestion reasoning based on the first image sequence and the first document body to obtain corresponding annotation suggestion instructions and send them to the annotation output module; and output high-dimensional image-text fusion features to the fast system during the processing;

[0010] The fast system is used to perform labeling task processing according to the first image sequence and the high-dimensional image-text fusion features output by the slow system to obtain a corresponding labeling entity sequence and send it to the labeling output module;

[0011] The annotation output module is used to combine the annotation suggestion description and the annotation object sequence into a corresponding document annotation report and output it.

[0012] Preferably, the slow system includes a visual encoder, a language encoder, a multimodal feature fusion module and a thought chain reasoning module; the multimodal feature fusion module is connected to the visual encoder, the language encoder and the thought chain reasoning module respectively; the multimodal feature fusion module is also connected to the adapter module of the fast system; the visual encoder is implemented based on the Vision Transformer model; the language encoder is implemented based on the Decoder-only pre-trained language model; the thought chain reasoning module is implemented based on the Transformer model;

[0013] The fast system is formed by sequentially connecting a basic coding module, an adapter module and a labeling decision module; the basic coding module is implemented based on a Vision Transformer model; and the labeling decision module is implemented based on a Diffusion Transformer model.

[0014] Furthermore, the language encoder has a Decoder module with at least 40 layers of Transformer;

[0015] The vocabulary size of the language encoder is not less than 151,936 and supports professional vocabulary expansion.

[0016] Preferably, the slow system is specifically configured to, when the corresponding annotation suggestion description is obtained by performing annotation suggestion reasoning according to the first image sequence and the first document body and is sent to the annotation output module:

[0017] The visual encoder is used to receive the first image sequence sent by the data preprocessing module; and perform image feature encoding processing on the first image sequence to obtain a corresponding first visual feature vector and send it to the multimodal feature fusion module;

[0018] The language encoder is used to receive the first document text sent by the data preprocessing module; and perform text feature encoding processing on the first document text to obtain a corresponding first text feature vector and send it to the multimodal feature fusion module;

[0019] The multimodal feature fusion module is used to perform feature fusion processing on the first visual feature vector and the first text feature vector according to the cross attention mechanism to obtain a corresponding first fused feature vector and send it to the thinking chain reasoning module;

[0020] The thought chain reasoning module is used to perform multi-step reasoning according to the first fused feature vector according to a preset annotation suggestion chain reasoning paradigm to obtain the corresponding annotation suggestion description and send it to the annotation output module; the annotation suggestion description at least includes a literature background description, a keyword description, and an annotation strategy description.

[0021] Preferably, the slow system is specifically used for outputting high-dimensional image-text fusion features to the fast system during the processing, and the multimodal feature fusion module sends the obtained first fusion feature vector to the adapter module.

[0022] Preferably, the fast system is specifically used for, when the labeling task is processed according to the first image sequence and the high-dimensional image-text fusion features output by the slow system to obtain a corresponding labeling entity sequence and sending it to the labeling output module:

[0023] The basic encoding module is used to receive the first image sequence sent by the data preprocessing module; and perform image feature encoding processing on the first image sequence to obtain a corresponding second visual feature vector and send it to the adapter module;

[0024] The adapter module is used to regard the feature vector space of the second visual feature vector as a low-dimensional space, regard the feature vector space of the first fused feature vector as a high-dimensional space, and perform feature vector projection processing on the second visual feature vector from the low-dimensional space to the high-dimensional space to obtain a corresponding first projected feature vector; and perform standardization processing on the first projected feature vector to obtain a corresponding first standardized vector; and perform feature fusion processing on the first standardized vector and the first fused feature vector based on a skip connection method to obtain a corresponding second fused feature vector and send it to the labeling decision module;

[0025] The labeling decision module is used to perform labeling entity recognition according to the second fused feature vector to obtain a corresponding labeling entity sequence; and send the labeling entity sequence to the labeling output module; the labeling entity sequence includes multiple labeling entities; each of the labeling entities is composed of an entity triple, and the entity triple includes entity text, entity type, and entity confidence.

[0026] The embodiment of the present invention provides a data annotation system in which a fast and slow system work in coordination, and the system includes: a data preprocessing module, a slow system, a fast system and an annotation output module; wherein the data preprocessing module is connected to the slow system and the fast system respectively; the slow system is connected to the fast system; and the annotation output module is connected to the slow system and the fast system respectively. The data preprocessing module is used to split the image / chart content and the text content of the input scientific literature file to obtain the corresponding image sequence and the text of the literature, and send it to the slow system, and send the image sequence to the fast system; the slow system is used to perform annotation suggestion reasoning based on the image sequence and the text of the literature to obtain the annotation suggestion description and send it to the annotation output module, and output high-dimensional image-text fusion features to the fast system during the processing; the fast system is used to perform annotation task processing based on the image sequence and the high-dimensional image-text fusion features output by the slow system to obtain the annotation entity sequence and send it to the annotation output module; the annotation output module is used to combine the annotation suggestion description and the annotation object sequence into a corresponding literature annotation report and output it. The fast and slow system collaborative working framework of the embodiment of the present invention not only ensures the annotation efficiency, but also improves the annotation quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A module structure diagram of a data annotation system in which fast and slow systems work together, provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] like Figure 1 As shown in the module structure diagram of a data annotation system for the cooperation of a fast and slow system provided by an embodiment of the present invention, the data annotation system provided by an embodiment of the present invention mainly includes: a data preprocessing module 1, a slow system 2, a fast system 3 and an annotation output module 4. The data preprocessing module 1 is connected to the slow system 2 and the fast system 3 respectively; the slow system 2 is connected to the fast system 3; the annotation output module 4 is connected to the slow system 2 and the fast system 3 respectively.

[0030] (I) Data preprocessing module 1:

[0031] The data preprocessing module 1 of the embodiment of the present invention is used to receive input scientific literature files; and to split the image / chart content and the text content of the scientific literature files to obtain the corresponding first image sequence and the first document text; and to send the first image sequence and the first document text to the slow system 2; and to send the first image sequence to the fast system 3.

[0032] (II) Slow system 2:

[0033] The slow system 2 of the embodiment of the present invention includes a visual encoder 21, a language encoder 22, a multimodal feature fusion module 23 and a thought chain reasoning module 24. Among them, the multimodal feature fusion module 23 is connected to the visual encoder 21, the language encoder 22 and the thought chain reasoning module 24 respectively; the multimodal feature fusion module 23 is also connected to the adapter module 32 of the fast system 3. The visual encoder 21 is implemented based on the Vision Transformer model; the language encoder 22 is implemented based on the Decoder-only pre-trained language model; the thought chain reasoning module 24 is implemented based on the Transformer model. In addition, the language encoder 22 of the embodiment of the present invention has a Decoder module with at least 40 layers of Transformer; the vocabulary size of the language encoder 22 is not less than 151,936, and supports professional vocabulary expansion.

[0034] The slow system 2 is used to perform annotation suggestion reasoning based on the first image sequence and the first document text to obtain corresponding annotation suggestion instructions and send them to the annotation output module 4; and output high-dimensional image-text fusion features to the fast system 3 during the processing.

[0035] It should be noted that, before the slow system 2 of the embodiment of the present invention is applied, the encoding and decoding performance of each module and the overall annotation suggestion reasoning function have completed model training.

[0036] In a specific implementation of the embodiment of the present invention, the slow system 2 is specifically configured to, when inferring the annotation suggestion based on the first image sequence and the first document body to obtain the corresponding annotation suggestion description and sending it to the annotation output module 4:

[0037] The visual encoder 21 is used to receive the first image sequence sent by the data preprocessing module 1; and perform image feature encoding processing on the first image sequence to obtain a corresponding first visual feature vector and send it to the multimodal feature fusion module 23;

[0038] The language encoder 22 is used to receive the first document body sent by the data preprocessing module 1; and perform text feature encoding processing on the first document body to obtain a corresponding first text feature vector and send it to the multimodal feature fusion module 23;

[0039] The multimodal feature fusion module 23 is used to perform feature fusion processing on the first visual feature vector and the first text feature vector according to the cross attention mechanism to obtain a corresponding first fused feature vector and send it to the thinking chain reasoning module 24;

[0040] The thought chain reasoning module 24 is used to perform multi-step reasoning according to the preset annotation suggestion chain reasoning paradigm, based on the first fusion feature vector, to obtain the corresponding annotation suggestion description and send it to the annotation output module 4; here, the annotation suggestion description of the embodiment of the present invention at least includes a literature background description, a keyword description, and a annotation strategy description.

[0041] Here, the annotation suggestion chain reasoning paradigm of the embodiment of the present invention is a pre-set chain reasoning paradigm based on multiple-segment formatting prompt instructions, and this chain reasoning paradigm is used to reason about the annotation suggestions according to the annotation specifications / templates of scientific and technological literature data.

[0042] In another specific implementation of the embodiment of the present invention, when the slow system 2 is specifically used to output high-dimensional image-text fusion features to the fast system 3 during processing, the multimodal feature fusion module 23 sends the obtained first fusion feature vector to the adapter module 32. Here, the multimodal feature fusion module 23 plays a role in feature transfer by sending the first fusion feature vector to the adapter module 32. Through this feature transfer, the modal understanding ability of the fast system 3 can be quickly improved without increasing the complexity of the fast system 3, thereby achieving the purpose of improving the annotation accuracy / quality.

[0043] (III) Fast System 3:

[0044] The fast system 3 of the embodiment of the present invention is formed by sequentially connecting a basic encoding module 31, an adapter module 32 and a labeling decision module 33; wherein the basic encoding module 31 is implemented based on the Vision Transformer model; and the labeling decision module 33 is implemented based on the Diffusion Transformer model.

[0045] The fast system 3 is used to perform labeling task processing according to the first image sequence and the high-dimensional image-text fusion features output by the slow system 2 to obtain a corresponding labeling entity sequence and send it to the labeling output module 4 .

[0046] It should be noted that, before the fast system 3 of the embodiment of the present invention is applied, the encoding and decoding performance of each module and the overall labeling task processing function have completed model training.

[0047] In another specific implementation of the embodiment of the present invention, the fast system 3 is specifically used to perform annotation task processing according to the first image sequence and the high-dimensional image-text fusion features output by the slow system 2 to obtain a corresponding annotation entity sequence and send it to the annotation output module 4:

[0048] The basic encoding module 31 is used to receive the first image sequence sent by the data preprocessing module 1; and perform image feature encoding processing on the first image sequence to obtain a corresponding second visual feature vector and send it to the adapter module 32;

[0049] The adapter module 32 is used to regard the feature vector space of the second visual feature vector as a low-dimensional space, regard the feature vector space of the first fused feature vector as a high-dimensional space, and perform feature vector projection processing on the second visual feature vector from the low-dimensional space to the high-dimensional space to obtain a corresponding first projected feature vector; and perform standardization processing on the first projected feature vector to obtain a corresponding first standardized vector; and perform feature fusion processing on the first standardized vector and the first fused feature vector based on a skip connection (Skip Connection) / residual connection (Residual Connection) method to obtain a corresponding second fused feature vector and send it to the labeling decision module 33;

[0050] The labeling decision module 33 is used to perform labeling entity recognition according to the second fused feature vector to obtain a corresponding labeling entity sequence; and send the labeling entity sequence to the labeling output module 4; wherein the labeling entity sequence includes multiple labeling entities; each labeling entity is composed of an entity triplet, and the entity triplet includes entity text, entity type, and entity confidence.

[0051] (IV) Labeling output module 4:

[0052] The annotation output module 4 of the embodiment of the present invention is used to combine the annotation suggestion description and the annotation object sequence into a corresponding document annotation report and output it.

[0053] It should be noted that it should be understood that the division of the various modules of the above system is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the adapter module can be a separately established processing element, or it can be integrated in a chip of the above-mentioned device. In addition, it can also be stored in the memory of the above-mentioned device in the form of program code, and called by a processing element of the above-mentioned system and execute the functions of the above-mentioned determined modules. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each method step of the aforementioned method or each module processing step of the aforementioned system can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0054] For example, the modules of the above system may be one or more integrated circuits configured to implement the aforementioned method, such as one or more application specific integrated circuits (ASIC), or one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module of the above system is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0055] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the above method embodiments are generated. The above computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above-mentioned computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the above-mentioned computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, Bluetooth, microwave, etc.) methods. The above-mentioned computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The above-mentioned available medium can be a magnetic medium (such as a floppy disk, a hard disk, a tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0056] The embodiment of the present invention provides a data annotation system in which a fast and slow system work in coordination, and the system includes: a data preprocessing module, a slow system, a fast system and an annotation output module; wherein the data preprocessing module is connected to the slow system and the fast system respectively; the slow system is connected to the fast system; and the annotation output module is connected to the slow system and the fast system respectively. The data preprocessing module is used to split the image / chart content and the text content of the input scientific literature file to obtain the corresponding image sequence and the text of the literature, and send it to the slow system, and send the image sequence to the fast system; the slow system is used to perform annotation suggestion reasoning based on the image sequence and the text of the literature to obtain the annotation suggestion description and send it to the annotation output module, and output high-dimensional image-text fusion features to the fast system during the processing; the fast system is used to perform annotation task processing based on the image sequence and the high-dimensional image-text fusion features output by the slow system to obtain the annotation entity sequence and send it to the annotation output module; the annotation output module is used to combine the annotation suggestion description and the annotation object sequence into a corresponding literature annotation report and output it. The fast and slow system collaborative working framework of the embodiment of the present invention not only ensures the annotation efficiency, but also improves the annotation quality.

[0057] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0058] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0059] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A data annotation system in which fast and slow systems work together, characterized in that: The system comprises: a data preprocessing module, a slow system, a fast system and a labeling output module; The data preprocessing module is connected to the slow system and the fast system respectively; the slow system is connected to the fast system; the annotation output module is connected to the slow system and the fast system respectively; The data preprocessing module is used to receive an input scientific literature file; and to separate the image / chart content and the text content of the scientific literature file to obtain a corresponding first image sequence and a first literature text; and to send the first image sequence and the first literature text to the slow system; and to send the first image sequence to the fast system; The slow system is used to perform annotation suggestion reasoning based on the first image sequence and the first document body to obtain corresponding annotation suggestion instructions and send them to the annotation output module; and output high-dimensional image-text fusion features to the fast system during the processing; The fast system is used to perform labeling task processing according to the first image sequence and the high-dimensional image-text fusion features output by the slow system to obtain a corresponding labeling entity sequence and send it to the labeling output module; The annotation output module is used to combine the annotation suggestion description and the annotation object sequence into a corresponding document annotation report and output it.

2. The data annotation system for fast and slow systems working together according to claim 1, characterized in that: The slow system includes a visual encoder, a language encoder, a multimodal feature fusion module and a thought chain reasoning module; the multimodal feature fusion module is connected to the visual encoder, the language encoder and the thought chain reasoning module respectively; the multimodal feature fusion module is also connected to the adapter module of the fast system; the visual encoder is implemented based on the Vision Transformer model; the language encoder is implemented based on the Decoder-only pre-trained language model; the thought chain reasoning module is implemented based on the Transformer model; The fast system is formed by sequentially connecting a basic coding module, an adapter module and a labeling decision module; the basic coding module is implemented based on a Vision Transformer model; and the labeling decision module is implemented based on a Diffusion Transformer model.

3. The data annotation system for fast and slow systems working together according to claim 2, characterized in that: The language encoder has a Decoder module with at least 40 layers of Transformer; The vocabulary size of the language encoder is not less than 151,936 and supports professional vocabulary expansion.

4. The data annotation system for fast and slow systems working together according to claim 2, characterized in that: The slow system is specifically configured to, when the annotation suggestion inference is performed based on the first image sequence and the first document body to obtain a corresponding annotation suggestion description and send it to the annotation output module: The visual encoder is used to receive the first image sequence sent by the data preprocessing module; and perform image feature encoding processing on the first image sequence to obtain a corresponding first visual feature vector and send it to the multimodal feature fusion module; The language encoder is used to receive the first document text sent by the data preprocessing module; and perform text feature encoding processing on the first document text to obtain a corresponding first text feature vector and send it to the multimodal feature fusion module; The multimodal feature fusion module is used to perform feature fusion processing on the first visual feature vector and the first text feature vector according to the cross attention mechanism to obtain a corresponding first fused feature vector and send it to the thinking chain reasoning module; The thought chain reasoning module is used to perform multi-step reasoning according to the first fused feature vector according to a preset annotation suggestion chain reasoning paradigm to obtain the corresponding annotation suggestion description and send it to the annotation output module; The annotation suggestion description at least includes a document background description, a keyword description, and an annotation strategy description.

5. The data annotation system for fast and slow systems working together according to claim 2, characterized in that: The slow system is specifically used for outputting high-dimensional image-text fusion features to the fast system during the processing, and the multimodal feature fusion module sends the first fusion feature vector obtained to the adapter module.

6. The data annotation system for fast and slow systems working together according to claim 2, characterized in that: The fast system is specifically used for, when the labeling task is processed according to the first image sequence and the high-dimensional image-text fusion feature output by the slow system to obtain a corresponding labeling entity sequence and sending it to the labeling output module: The basic encoding module is used to receive the first image sequence sent by the data preprocessing module; and perform image feature encoding processing on the first image sequence to obtain a corresponding second visual feature vector and send it to the adapter module; The adapter module is used to regard the feature vector space of the second visual feature vector as a low-dimensional space, regard the feature vector space of the first fused feature vector as a high-dimensional space, and perform feature vector projection processing on the second visual feature vector from the low-dimensional space to the high-dimensional space to obtain a corresponding first projected feature vector; and perform standardization processing on the first projected feature vector to obtain a corresponding first standardized vector; and perform feature fusion processing on the first standardized vector and the first fused feature vector based on a skip connection method to obtain a corresponding second fused feature vector and send it to the labeling decision module; The labeling decision module is used to perform labeling entity recognition according to the second fused feature vector to obtain a corresponding labeling entity sequence; and send the labeling entity sequence to the labeling output module; the labeling entity sequence includes multiple labeling entities; each of the labeling entities is composed of an entity triple, and the entity triple includes entity text, entity type, and entity confidence.

Images