Content information identification method, device, computer equipment and storage medium

Abstract

The present application relates to a content information recognition method, apparatus, computer equipment and storage medium. The method includes: obtaining a content feature vector sequence obtained by extracting features from the target content, obtaining an information feature vector sequence obtained by extracting features from each task description information, arranging the content feature vector sequence and each information feature vector sequence according to an arrangement strategy to obtain a current recognition feature vector sequence corresponding to the current moment; obtaining a current attention weight sequence corresponding to each task description information at the current moment based on the current recognition feature vector sequence; shielding the current attention weight outside the target position in the current attention weight sequence, performing attention processing based on the current attention weight corresponding to the target position to obtain an attention feature vector; and obtaining the current recognized character based on the attention feature vector. The present application can use a machine learning model for character recognition, and the use of this method can improve the efficiency of information recognition.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a content information identification method, apparatus, computer equipment, and storage medium. Background Art

[0002] With the development of Internet technology, the amount of text information on the Internet has gradually increased, becoming an important source of information for people. For example, search engines can be used to retrieve required information from text information on the Internet.

[0003] Currently, machine learning models can be used to identify the required information from text information. However, the current method of identifying information is time-consuming, resulting in low efficiency in identifying information. Summary of the Invention

[0004] Based on this, it is necessary to provide a content information identification method, apparatus, computer equipment and storage medium that can improve the efficiency of identifying information in order to address the above technical problems.

[0005] A method for identifying content information, the method comprising: obtaining target content to be identified, obtaining task description information corresponding to multiple information identification tasks corresponding to the target content; obtaining a content feature vector sequence obtained by performing feature extraction on the target content, obtaining an information feature vector sequence obtained by performing feature extraction on each of the task description information, arranging the content feature vector sequence and each of the information feature vector sequences according to an arrangement strategy to obtain a current identification feature vector sequence corresponding to the current moment; obtaining a current attention weight sequence corresponding to each of the task description information at the current moment based on the current identification feature vector sequence; for each of the task description information, masking the current attention weight outside the target position in the current attention weight sequence, performing attention processing based on the current attention weight corresponding to the target position and the current identification feature vector sequence to obtain an attention feature vector; the target position comprises: a feature vector arrangement position corresponding to the task description information in the current identification feature vector sequence, and a feature vector arrangement position corresponding to the target content in the current identification feature vector sequence; obtaining a current identification character of the task description information at the current moment based on the attention feature vector, and obtaining an information identification result corresponding to the task description information based on the identification characters of the task description information at each moment.

[0006] A content information identification device, the device comprising: a task description information acquisition module for acquiring target content to be identified, and acquiring task description information corresponding to a plurality of information identification tasks corresponding to the target content; a current identification feature vector sequence acquisition module for acquiring a content feature vector sequence obtained by extracting features from the target content, and acquiring an information feature vector sequence obtained by extracting features from each of the task description information, arranging the content feature vector sequence and each of the information feature vector sequences according to an arrangement strategy to obtain a current identification feature vector sequence corresponding to the current moment; a current attention weight sequence acquisition module for obtaining, based on the current identification feature vector sequence, the current attention weight sequence corresponding to each of the task description information at the current moment. sequence; an attention feature vector obtaining module, used for shielding the current attention weight outside the target position in the current attention weight sequence for each task description information, performing attention processing based on the current attention weight corresponding to the target position and the current recognition feature vector sequence, to obtain an attention feature vector; the target position includes: the feature vector arrangement position of the task description information corresponding to the current recognition feature vector sequence, and the feature vector arrangement position corresponding to the target content in the current recognition feature vector sequence; an information recognition result obtaining module, used for obtaining the current recognition character of the task description information at the current moment based on the attention feature vector, and obtaining the information recognition result corresponding to the task description information based on the recognition characters of the task description information at each moment.

[0007] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned content information identification method when executing the computer program.

[0008] A computer-readable storage medium stores a computer program, which implements the steps of the above-mentioned content information identification method when executed by a processor.

[0009] A computer program product includes a computer program, characterized in that when the computer program is executed by a processor, the steps of the above-mentioned content information identification method are implemented.

[0010] The above-mentioned content information recognition method, device, computer equipment and storage medium, for each task description information, masks the current attention weight outside the target position in the current attention weight sequence, performs attention processing based on the current attention weight corresponding to the target position and the current recognition feature vector sequence, and obtains the attention feature vector. Therefore, the attention feature vector corresponding to each task description information is obtained at the current moment, and the current recognition character of the task description information at the current moment is obtained based on the attention feature vector. Then, the recognition characters corresponding to multiple task description information are obtained at the current moment, that is, multiple recognition characters of multiple task description information at the current moment can be obtained through a current recognition feature vector sequence, thereby improving the efficiency of information recognition. In addition, for each task description information, the current attention weight outside the target position in the current attention weight sequence is masked, and attention processing is performed based on the current attention weight corresponding to the target position and the current recognition feature vector sequence to obtain an attention feature vector. Since the target position includes the feature vector arrangement position of the corresponding task description information in the current recognition feature vector sequence and the feature vector arrangement position corresponding to the target content in the current recognition feature vector sequence, the information related to the task description information is taken into account during attention processing and the information unrelated to the task description information is ignored, thereby improving the accuracy of attention processing and thus improving the accuracy of information recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A diagram illustrating an application environment of a content information identification method in some embodiments;

[0012] Figure 2 is a flowchart of a content information identification method in some embodiments;

[0013] Figure 3 is a schematic diagram of a content identification interface in some embodiments;

[0014] Figure 4 is an architectural diagram of a content information identification model in some embodiments;

[0015] Figure 5 is a structural diagram of a Transformer block in some embodiments;

[0016] Figure 6 is a schematic diagram of a target weight shielding matrix in some embodiments;

[0017] Figure 7 is a schematic diagram of a target weight shielding matrix in some embodiments;

[0018] Figure 8 is a schematic diagram of identifying a feature vector sequence in some embodiments;

[0019] Figure 9 is a schematic diagram of identifying a feature vector sequence in some embodiments;

[0020] Figure 10 is a schematic diagram of a content information identification method in some embodiments;

[0021] Figure 11 is a schematic diagram of identifying a feature vector sequence in some embodiments;

[0022] Figure 12 is a flowchart of a content information identification method in some embodiments;

[0023] Figure 13 is a structural block diagram of a content information identification device in some embodiments;

[0024] Figure 14 is a diagram of the internal structure of a computer device in some embodiments;

[0025] Figure 15 1 is a diagram of the internal structure of a computer device in some embodiments. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0027] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0028] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and smart transportation.

[0029] Computer vision (CV) is the science of making machines "see." Specifically, it refers to the use of cameras and computers to replace the human eye in identifying, tracking, and measuring objects. Further image processing is performed to transform the computer-generated images into images more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, autonomous driving, and smart transportation. It also includes common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0030] Natural language processing (NLP) is a key area of research in computer science and artificial intelligence. It studies the theories and methods that enable effective communication between humans and computers using natural language. Natural language processing (NLP) integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language we use in everyday life—and is closely linked to the study of linguistics. Natural language processing technologies typically include text processing, semantic understanding, machine translation, robotic question answering, and knowledge graphs.

[0031] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0032] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, Internet of Vehicles, automatic driving, smart transportation, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0033] The solutions provided in the embodiments of this application involve technologies such as machine learning based on artificial intelligence, and are specifically described through the following embodiments:

[0034] The content information identification method provided by this application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 via a network.

[0035] Specifically, the server 104 can obtain the target content to be identified. The target content can be stored in the server 104 or sent to the server 104 by the terminal 102. The server 104 can obtain the task description information corresponding to multiple information identification tasks corresponding to the target content. The task description information can be stored in the server 104 or sent to the server 104 by the terminal 102. For example, the terminal 102 can send a content identification request to the server 104, and the content identification request can carry the target content and task description information. The server 104 can obtain a content feature vector sequence obtained by extracting features from the target content, obtain an information feature vector sequence obtained by extracting features from each task description information, arrange the content feature vector sequence and each information feature vector sequence according to an arrangement strategy, obtain a current recognition feature vector sequence corresponding to the current moment, obtain a current attention weight sequence corresponding to each task description information at the current moment based on the current recognition feature vector sequence, for each task description information, mask the current attention weight outside the target position in the current attention weight sequence, perform attention processing based on the current attention weight corresponding to the target position and the current recognition feature vector sequence, obtain an attention feature vector, the target position includes: the feature vector arrangement position of the corresponding task description information in the current recognition feature vector sequence, and the feature vector arrangement position corresponding to the target content in the current recognition feature vector sequence, obtain the current recognition character of the task description information at the current moment based on the attention feature vector, and obtain the information recognition result corresponding to the task description information based on the recognition character of the task description information at each moment. The server 104 can send the information recognition result corresponding to each task description information to the terminal 102, and the terminal 102 can display the task description information and the information recognition result accordingly.

[0036] The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0037] It can be understood that the above application scenario is only an example and does not constitute a limitation on the content information identification method provided in the embodiment of the present application. The method provided in the embodiment of the present application can also be applied in other application scenarios. For example, the content information identification method provided in the present application can be executed by the terminal 102, and the terminal 102 can upload the obtained information identification results to the server 104. The server 104 can store the information identification results and forward the information identification results to other terminal devices.

[0038] In some embodiments, as Figure 2 As shown, a content information identification method is provided, which is applied to Figure 1 Taking the server 104 in the example as an example, the following steps are included:

[0039] S202: Acquire target content to be identified, and acquire task description information corresponding to a plurality of information identification tasks corresponding to the target content.

[0040] The content can be any form of data, including at least one of text, video, audio, or images. The target content can be any type of content, such as text. The target content can include multiple characters, each of which represents a basic language unit. For Chinese, one character can correspond to one Chinese character, and for English, one character can correspond to one English word. The target content can be any of a Chinese article, an English article, or a mixed article. A mixed article refers to an article that includes at least two languages. For example, if the target content is a Chinese article that includes 100 Chinese characters, then the target content includes 100 characters.

[0041] An information recognition task refers to a task of identifying information. Different information recognition tasks indicate the recognition of different information. An information recognition task may include at least one of a task of identifying location information, a task of identifying time information, a task of identifying a name, a task of identifying an ID card, or a task of identifying a bill number.

[0042] Task description information refers to the descriptive information corresponding to the information recognition task. Task description information is semantically meaningful and reflects the purpose of the information recognition task. Target information refers to the information identified by the information recognition task. The purpose of the information recognition task is identified from the target content. The task description information may include information related to the purpose information, such as a name, location, or information type. For example, if the purpose of the information recognition task is "Xiao Ming's age," the task description information may be "How old is Xiao Ming this year?". "How old is Xiao Ming this year?" includes information related to the purpose information "Xiao Ming's age," namely, "Xiao Ming," "how old," and "this year." If the target content is invoice information and the information recognition task is to identify an ID card number, the task description information may be "ID card" or "document." The target content can correspond to multiple information recognition tasks, with each information recognition task corresponding to one task description information. The target content and task description information may be pre-stored on the server or sent to the server by the terminal.

[0043] Specifically, the terminal can send a content identification request to the server, and the content identification request can carry the target content and task description information. The server can respond to the content identification request, extract the target content and task description information from the content identification request, and identify the corresponding purpose information from the target content based on the task description information. The server can return the identified purpose information to the terminal, and the terminal can display the task description information and the corresponding purpose information accordingly.

[0044] In some embodiments, the terminal can display a content recognition interface, which may include a content input area and a description information filling area. The terminal can use the content input through the content input area as the target content and the information input through the description information filling area as the task description information. When the terminal receives a content recognition trigger operation, it generates a content recognition request based on the target content and the task description information, and sends the content recognition request to the server. The server can respond to the content recognition request, identify the purpose information corresponding to each task description information from the target content, and return the purpose information to the terminal. The terminal can display the task description information and the corresponding purpose information in the recognition result interface. Figure 3As shown, a content recognition interface 300 is displayed, in which "Xiao Ming's Autobiography" is uploaded in the content input area 302, and a plurality of task description information are filled in the description information filling area 304, namely "Xiao Ming's birthplace", "Xiao Ming's birth year" and "Xiao Ming's hobbies". When the terminal receives a trigger operation on the "recognition" control 306, it sends a content recognition request to the server, and receives the purpose information corresponding to each task description information returned by the server in response to the content recognition request, namely, the purpose information "Shanghai" corresponding to the task description information "Xiao Ming's birthplace", the purpose information "2002" corresponding to the task description information "Xiao Ming's birth year" and the purpose information "painting" corresponding to the task description information "Xiao Ming's hobbies", and displays the purpose information and the task description information in correspondence in the recognition result interface 308, that is, the purpose information is displayed below the corresponding task description information. Figure 3 The figure is just an example. The relative position between the target information and the task description information can be set as needed. The target information does not necessarily have to be displayed below the task description information.

[0045] S204, obtain a content feature vector sequence obtained by extracting features from the target content, obtain an information feature vector sequence obtained by extracting features from each task description information, arrange the content feature vector sequence and each information feature vector sequence according to the arrangement strategy, and obtain a current recognition feature vector sequence corresponding to the current moment.

[0046] Among them, the target content includes multiple characters, and the content feature vector sequence includes content feature vectors corresponding to each character in the target content. The content feature vector corresponding to the character is a feature vector obtained by feature extraction of the character. The content feature vectors in the content feature vector sequence are arranged according to the order of the characters in the target content. The higher the order of the character in the target content, the higher the order of the content feature vector corresponding to the character in the content feature vector sequence.

[0047] The task description information includes at least one character. One task description information corresponds to an information feature vector sequence. The information feature vector sequence includes information feature vectors corresponding to each character in the task description information. The information feature vector corresponding to the character is a feature vector obtained by feature extraction of the character. The information feature vectors in the information feature vector sequence are arranged according to the order of the characters in the task description information. The higher the order of the character in the task description information, the higher the order of the information feature vector corresponding to the character in the information feature vector sequence.

[0048] The arrangement strategy refers to the strategy relied upon for arranging the current identification feature vector sequence. The arrangement strategy may be preset or set as needed. The arrangement strategy may include the arrangement position of the content feature vector sequence and the arrangement positions corresponding to each information feature vector sequence.

[0049] The current moment is the current moment. The purpose information corresponding to the task description information may include at least one character. The characters in the purpose information may be recognized at different moments. For example, one or more characters in the purpose information may be recognized at each moment, and then the characters recognized at each moment are arranged to obtain the purpose information. For example, the recognized characters are arranged according to the moment to obtain the purpose information. The earlier the moment, the higher the character is ranked in the purpose information. Taking a character recognized in the purpose information at a moment as an example, if the task description information is "Xiao Ming's birthplace", the character recognized at the first moment is "Shang", the character recognized at the second moment is "Hai", and the character recognized at the subsequent moment is "Space", then the purpose information is "Shanghai". Of course, the characters in the purpose information may also be recognized at the same moment. For example, if the purpose information only includes 2 characters, and 2 characters in the purpose information are recognized at each moment, then the characters in the purpose information are recognized at the same moment.

[0050] The current recognition feature vector sequence refers to the recognition feature vector sequence corresponding to the current moment. The current recognition feature vector sequence includes the content feature vector sequence and each information feature vector sequence. The recognition feature vector sequence at each moment includes the content feature vector sequence and each information feature vector sequence. The recognition feature vector sequences at different moments can be different. For example, the recognition feature vector sequence at the next moment can include the feature vector of the character recognized at the current moment.

[0051] Specifically, the server can perform feature extraction on each character in the target content to obtain a feature vector corresponding to each character, arrange the feature vectors of the characters according to the order of the characters in the target content, and use the arranged sequence as a content feature vector sequence. The server can perform feature extraction on each character in the task description information to obtain a feature vector corresponding to each character, arrange the feature vectors of the characters according to the order of the characters in the task description information, and use the arranged sequence as an information feature vector sequence. The server can arrange the content feature vector sequence and each information feature vector sequence according to the arrangement strategy to obtain a current recognition feature vector sequence corresponding to the current moment. For example, the content feature vector sequence can be arranged before each information feature vector sequence to obtain a current recognition feature vector sequence corresponding to the current moment.

[0052] In some embodiments, the server can arrange the content feature vector sequence and the information feature vector sequence according to the arrangement strategy, use the arranged sequence as the current recognition feature vector sequence corresponding to the current moment, perform information recognition based on the current recognition feature vector sequence, obtain the character recognized at the current moment, add the feature vector corresponding to the character recognized at the current moment to the current recognition feature vector sequence, obtain the recognition feature vector sequence at the next moment, and then use the recognition feature vector sequence at the next moment to perform information recognition. After multiple cycles, the recognition characters corresponding to each moment can be obtained.

[0053] In some embodiments, the server can obtain the recognition feature vector sequence used in the previous moment as the forward recognition feature vector sequence. The server can obtain the forward feature vector corresponding to the forward recognition character recognized by each task description information at the previous moment, add each forward feature vector to the forward recognition feature vector sequence, and obtain the current recognition feature vector sequence corresponding to the current moment. For the recognition feature vector sequence used in the next moment, the server can obtain the current feature vector corresponding to the current recognition character recognized at the current moment, add each current feature vector to the current recognition feature vector sequence, and obtain the recognition feature vector sequence corresponding to the next moment. The recognition feature vector sequence at each moment is arranged according to the arrangement strategy.

[0054] S206: Based on the current recognition feature vector sequence, a current attention weight sequence corresponding to each task description information at the current moment is obtained.

[0055] Among them, one task description information corresponds to at least one current attention weight sequence. The current attention weight sequence corresponding to the task description information refers to the attention weight sequence corresponding to the task description information at the current moment. The current attention weight sequence corresponding to the task description information includes the attention weights generated by each feature vector in the current recognition feature vector sequence for the task description information. The attention weights in the current attention weight sequence are arranged according to the order of the feature vectors in the current recognition feature vector sequence. The higher the order of the feature vector in the current recognition feature vector sequence, the higher the order of the attention weight generated by the feature vector in the current attention weight sequence.

[0056] The larger the attention weight generated by the feature vector, the greater the influence of the feature vector on the recognition result of the task description information at the current moment. The smaller the attention weight generated by the feature vector, the smaller the influence of the feature vector on the recognition result of the task description information at the current moment.

[0057] Specifically, the current recognition feature vector sequence may include at least one mask feature vector corresponding to each task description information, and the current attention weight sequence corresponding to the task description information refers to the current attention weight sequence corresponding to the mask feature vector of the task description information. Each task description information may correspond to at least one mask feature vector, and each mask feature vector corresponds to a current attention weight sequence, so that the number of current attention weight sequences corresponding to the task description information is the same as the number of mask feature vectors corresponding to the task description information included in the current recognition feature vector sequence. The server can perform an attention weight calculation based on the mask feature vector and the content feature vector to obtain a first attention weight, and perform an attention weight calculation based on the mask feature vector and the information feature vector to obtain a second attention weight. Each content feature vector corresponds to a first attention weight, and each information feature vector corresponds to a second attention weight. The server can arrange each first attention weight and each second attention weight according to the order of the feature vector in the current recognition feature vector sequence to obtain a current attention weight sequence. Among them, the feature vector can be preset or set as needed, and the mask feature vectors corresponding to different task description information can be the same or different.

[0058] In some embodiments, the server may perform a vector product operation on the mask feature vector and the content feature vector, and use the result of the calculation as the first attention weight. The server may perform an attention weight calculation on the mask feature vector and the information feature vector, and use the result of the calculation as the second attention weight.

[0059] In some embodiments, the server can obtain a first linear transformation matrix, use the first linear transformation matrix to perform a linear transformation on each recognition feature vector in the current recognition feature vector sequence, obtain the first transformation feature vector corresponding to each recognition feature vector, arrange the first change feature vector of each recognition feature vector according to the order of the recognition feature vector in the current recognition feature vector sequence, and obtain the first change feature vector sequence corresponding to the current recognition feature vector sequence. The server can obtain a second linear transformation matrix, use the second linear transformation matrix to perform a linear transformation on each recognition feature vector in the current recognition feature vector sequence, obtain the second transformation feature vector corresponding to each recognition feature vector, arrange the second transformation feature vector of each recognition feature vector according to the order of the recognition feature vector in the current recognition feature vector sequence, and obtain the second transformation feature vector sequence corresponding to the current recognition feature vector sequence. The server can obtain the first transformation feature vector corresponding to the mask feature vector from the first transformation feature vector sequence, obtain the second transformation feature vector corresponding to the content feature vector from the second transformation feature vector sequence, perform a vector product operation on the first transformation feature vector corresponding to the mask feature vector and the second transformation feature vector corresponding to the content feature vector, and use the operation result as the first attention weight. The server can obtain the first transformed feature vector corresponding to the mask feature vector from the first transformed feature vector sequence, obtain the second transformed feature vector corresponding to the information feature vector from the second transformed feature vector sequence, perform a vector product operation on the first transformed feature vector corresponding to the mask feature vector and the second transformed feature vector corresponding to the information feature vector, and use the result of the operation as the second attention weight. The first linear transformation matrix and the second linear transformation matrix can be preset or set as needed. For example, the first linear transformation matrix can be the query matrix Q in the trained Transformer model, and the second linear transformation matrix can be the key matrix K in the trained Transformer model.

[0060] In some embodiments, each recognition feature vector in the current recognition feature vector sequence corresponds to a current attention weight sequence. The server can use the first transformed feature vector corresponding to each recognition feature vector in the current recognition feature vector sequence as a row vector in the first matrix, and arrange them according to the order of the recognition feature vector in the current recognition feature vector sequence, that is, the closer the recognition feature vector is in the current recognition feature vector sequence, the closer the row order of the first transformed feature vector corresponding to the recognition feature vector in the first matrix is. Similarly, the server can use the second transformed feature vector corresponding to each recognition feature vector in the current recognition feature vector sequence as a column vector in the second matrix, and arrange them according to the order of the recognition feature vector in the current recognition feature vector sequence, that is, the closer the position of the recognition feature vector is in the current recognition feature vector sequence, the closer the column order of the second transformed feature vector corresponding to the recognition feature vector in the second matrix is. The server can perform a matrix product operation on the first matrix and the second matrix, and use the matrix obtained by the matrix product operation as the weight matrix to be shielded. The server can use each row vector in the weight matrix to be shielded as the current attention weight sequence of each recognition feature vector in the current recognition feature vector sequence. The higher the ranking of the recognition feature vector in the current recognition feature vector sequence, the higher the row ranking of the current attention weight sequence corresponding to the recognition feature vector in the weight matrix to be shielded.

[0061] For example, if the current recognition feature vector sequence is [x1, x2, x3, x4, k00, k01, k10, k11, k12, k20, k21, mask00, mask10, mask20], where "x1, x2, x3, x4" is the content feature vector sequence, "k00, k01" is the information feature vector sequence corresponding to the first task description information, "k10, k11, k12" is the information feature vector sequence corresponding to the second task description information, "k20, k21" is the information feature vector sequence corresponding to the third task description information, mask00 is the mask feature vector corresponding to the first task description information, mask10 is the mask feature vector corresponding to the second task description information, and mask20 is the mask feature vector corresponding to the third task description information. The mask feature vector corresponding to the information, matrix A represents the first matrix, matrix B represents the second matrix, and matrix C represents the matrix to be masked. The first row of matrix A is the first transformed feature vector corresponding to the content feature vector x1, the first column of matrix B is the second transformed feature vector corresponding to the content feature vector x1, the first row of matrix C is the current attention weight sequence corresponding to the content feature vector x1, the 11th row of the first matrix A is the first transformed feature vector corresponding to the mask feature vector mask00, the 11th column of the second matrix B is the second transformed feature vector corresponding to the mask feature vector mask00, and the 11th row of the matrix C to be masked is the current attention weight sequence corresponding to the mask feature vector mask00, that is, the current attention weight sequence corresponding to the task description information corresponding to the mask feature vector mask00. It should be noted that the current attention weight sequence corresponding to the task description information refers to the current attention weight sequence corresponding to the mask feature vector of the task description information, not the current attention weight sequence corresponding to the information feature vector of the task description information. The current attention weight sequence corresponding to the information feature vector of the task description information is, for example, the current attention weight sequence corresponding to the information feature vector k00 in the task description information "k00, k01".

[0062] S208, for each task description information, mask the current attention weight outside the target position in the current attention weight sequence, perform attention processing based on the current attention weight corresponding to the target position and the current recognition feature vector sequence, and obtain an attention feature vector; the target position includes: the feature vector arrangement position of the corresponding task description information in the current recognition feature vector sequence, and the feature vector arrangement position corresponding to the target content in the current recognition feature vector sequence.

[0063] Among them, the target position is determined from the current attention weight sequence according to the arrangement strategy. The feature vector arrangement position refers to the arrangement position of the recognition feature vector in the current recognition feature vector sequence, and the feature vector arrangement position of the task description information includes the arrangement position of each recognition feature vector corresponding to the task description information in the current recognition feature vector sequence. The feature vector arrangement position corresponding to the target content includes the arrangement position of each content feature vector in the current recognition feature vector sequence. For example, the current recognition feature vector sequence is [x1,x2,x3,k00,k01,k10,k11,k12,k20,k21,mask00,mask10,mask20], for the first task description information, the current attention weight sequence corresponding to the mask feature vector mask00 of the first task description information is [wx1,wx2,wx3,wk00,wk01,wk10,wk11,wk12,wk20,wk21,wmask00,wmask10,wmas k20], since the current attention weight sequence includes the attention weights generated by each recognition feature vector in the current recognition feature vector sequence for the task description information, and the generated attention weights are arranged according to the order of the recognition feature vectors in the current recognition feature vector sequence, wx1 represents the attention weight generated by x1 for the first task description information, wx2 represents the attention weight generated by x2 for the first task description information, ..., wmask20 represents the attention weight generated by mask20 for the first task description information. Since the content feature vectors are arranged from the 1st to the 3rd positions and the feature vectors of the first task description information are arranged from the 4th to the 5th positions, the target positions in the current attention weight sequence of the first task description information include the 1st to the 5th positions (including the 1st and the 5th positions).

[0064] Specifically, the server can mask the current attention weights outside the target position in the current attention weight sequence of the task description information, and use the masked current attention weight sequence as the masked attention weight sequence corresponding to the task description information. The server can use the various attention weights in the masked attention weight sequence to perform weighted calculation on the various recognition feature vectors in the current recognition feature vector sequence, and use the vector obtained by the weighted calculation as the attention feature vector corresponding to the task description information. For example, the server can determine the arrangement position of a certain recognition feature vector in the current recognition feature vector sequence, obtain the attention weight at the arrangement position from the masked attention weight sequence, and use it as the weighted weight corresponding to the recognition feature vector. Using the same method, determine the weighted weights corresponding to other recognition feature vectors, perform weighted calculation on each recognition feature vector using the weighted weights corresponding to each recognition feature vector, and use the calculated vector as the attention feature vector corresponding to the task description information.

[0065] In some embodiments, the server can obtain a third linear transformation matrix, use the third linear transformation matrix to perform a linear transformation on each identification feature vector in the current identification feature vector sequence, obtain the third transformation feature vector corresponding to each identification feature vector, and arrange the third transformation feature vectors of each identification feature vector according to the order of the identification feature vector in the current identification feature vector sequence to obtain a third transformation feature vector sequence corresponding to the current identification feature vector sequence. The server can use the attention weights in the masked attention weight sequence corresponding to the task description information to perform weighted calculation on each third transformation feature vector in the third transformation feature vector sequence, and use the vector obtained by the weighted calculation as the attention feature vector corresponding to the task description information. For example, the server can determine the arrangement position of a third transformation feature vector in the third transformation feature vector sequence, obtain the attention weight at the arrangement position from the masked attention weight sequence, and use it as the weighted weight corresponding to the third transformation feature vector. Using the same method, determine the weighted weights corresponding to other third transformation feature vectors, use the weighted weights corresponding to each third transformation feature vector to perform weighted calculation on each third transformation feature vector, and use the calculated vector as the attention feature vector corresponding to the task description information. The third linear transformation matrix may be preset or set as needed, for example, it may be the value matrix K in a trained Transformer model.

[0066] In some embodiments, the server can obtain a trained content information recognition model, which is used to identify the required information in the content, for example, it can identify the answer to the question from the article. Figure 4 As shown in the figure, the architecture of a content information recognition model is shown. The content information recognition model includes a character feature extraction layer (Token embedding), a position feature extraction layer (Position embedding), a sequence feature extraction layer (Segment enbedding), and an attention processing module. The attention processing module includes L attention processing blocks (Transformer block). The attention processing module includes a masked multi-head attention layer (Masked Multi-headAttention), as shown in the figure. Figure 5 As shown in the figure, the dotted box shows the structure of a Transformer block. The content information recognition model includes Nx layers of Transformer blocks. Masked Multi-head Attention uses the Attention Mask matrix to control the dependencies between words.

[0067] Figure 4In the input sequence [input, k0, k1, k2, m00, m10, m20], input is the target content, k0, k1 and k2 are three task description information, m00 is the mask information corresponding to k0, m10 is the mask information corresponding to k1, and m20 is the mask information corresponding to k2, where m refers to mask. Each part in the input sequence can include at least one character. The input sequence is input into the content information recognition model. For each character, the character feature vector, position feature vector and sequence feature vector corresponding to the character are extracted using the character feature extraction layer, position feature extraction layer and sequence feature extraction layer respectively. The three extracted vectors are spliced to obtain a character extraction feature vector. The character extraction feature vectors corresponding to each character are arranged according to the arrangement position of the character in the input sequence, and the arranged sequence is used as the current recognition feature vector sequence. The attention processing module is used to generate the current attention weight sequence corresponding to each task description information at the current moment based on the current recognition feature vector sequence. The attention processing module may also include a weight shielding matrix (Attention Mask matrix), which can be used to shield the current attention weight outside the target position in the current attention weight sequence, and perform attention processing based on the current attention weight corresponding to the target position and the current recognition feature vector sequence to obtain an attention feature vector. For example, the attention processing module may include a query vector Q, a key vector K, and a value vector V. The server can use the query vector Q, the key vector K, and the value vector V to process the current recognition feature vector sequence respectively, and obtain the attention feature vector corresponding to the task description information based on the various results obtained by the processing and the current attention weight sequence. The content information recognition model provided in this embodiment enables the transformer to decode multiple words at the same time when used for information extraction, and reduce the mutual dependence between words, thereby increasing the decoding speed by 10 times and keeping the decoding accuracy unchanged.

[0068] S210, obtaining the current recognized character of the task description information at the current moment based on the attention feature vector, and obtaining the information recognition result corresponding to the task description information based on the recognized characters of the task description information at each moment.

[0069] The "currently recognized character" refers to the character recognized at the current moment. The "information recognition result" refers to the purpose information of the task description. The information recognition result corresponding to the task description includes the characters recognized at different moments in the task description, and the characters are arranged in the order of recognition time. The recognition time refers to the moment when the character is recognized. The earlier the recognition time, the higher the character is ranked in the information recognition result.

[0070] Specifically, the server may perform character conversion processing on the attention feature vector of the task description information. Character conversion processing refers to converting the attention feature vector into characters, and using the converted characters as the current recognition characters corresponding to the task description information. The server may arrange the recognition characters of the task description information at each moment in the order of recognition time, and use the arrangement result as the information recognition result. Since there are multiple task description information, information recognition results corresponding to multiple task description information can be obtained.

[0071] In the above-mentioned content information recognition method, for each task description information, the current attention weight outside the target position in the current attention weight sequence is shielded, and attention processing is performed based on the current attention weight corresponding to the target position and the current recognition feature vector sequence to obtain an attention feature vector. Therefore, the attention feature vector corresponding to each task description information is obtained at the current moment, and the current recognition character of the task description information at the current moment is obtained based on the attention feature vector. Then, the recognition characters corresponding to multiple task description information are obtained at the current moment, that is, multiple recognition characters of multiple task description information at the current moment can be obtained through a current recognition feature vector sequence, thereby improving the efficiency of information recognition. In addition, for each task description information, the current attention weight outside the target position in the current attention weight sequence is shielded, and attention processing is performed based on the current attention weight corresponding to the target position and the current recognition feature vector sequence to obtain an attention feature vector. Since the target position includes the feature vector arrangement position of the corresponding task description information in the current recognition feature vector sequence, and the feature vector arrangement position corresponding to the target content in the current recognition feature vector sequence, information related to the task description information is considered during attention processing while information unrelated to the task description information is ignored, thereby improving the accuracy of attention processing and thus improving the accuracy of information recognition.

[0072] The content information identification method provided in this application can be applied to machine question answering to improve the efficiency of answering. For example, multiple questions raised in the machine question answering can be used as task description information, and the article where the answer is located can be used as the target content. The content information identification method provided in this application is used to obtain the answers corresponding to each question. Since the characters in the answers corresponding to multiple questions can be obtained at the same time in this application, parallel information recognition is achieved, which improves the efficiency of information recognition. In other words, the content information identification method provided in this application implements a parallel decoding method, which improves the decoding speed without affecting the decoding accuracy. The decoding speed can be increased by more than 10 times, and provides an inter-segment parallelization method. For example, an article has N question and answer segments, and N words can be decoded at the same time. The generation time of a single text question and answer is reduced to about 1 / N of the original. Through the transformer attention mask design, the inter-segment parallelization scheme is completely consistent with the current single-word decoding scheme in terms of model decoding accuracy, and the model does not need to be retrained. Therefore, on the basis of keeping the model decoding accuracy consistent with the single-word decoding accuracy, the model decoding speed is increased to more than 10 times the original (the number of question and answer segments is generally greater than 10).

[0073] In some embodiments, the step of obtaining the target position includes: obtaining a first sorting corresponding to the content feature vector sequence in the arrangement strategy, and a second sorting corresponding to the information feature vector sequence; obtaining the feature vector arrangement position corresponding to the target content based on the first sorting and the number of feature vectors corresponding to the content feature vector sequence; obtaining the feature vector arrangement position of the task description information based on the second sorting and the number of feature vectors corresponding to the information feature vector sequence.

[0074] The first sorting refers to the sorting of the content feature vector sequence within the current identification feature vector sequence, and the second sorting refers to the sorting of the information feature vector sequence within the current identification feature vector sequence. The first and second sortings are determined by the permutation strategy. The number of feature vectors refers to the number of feature vectors.

[0075] There are multiple positions of feature vectors corresponding to target content, including the position of each content feature vector of target content in the current recognition feature vector sequence. There are multiple positions of feature vectors of task description information, including the position of each information feature vector of task description information in the current recognition feature vector sequence.

[0076] Specifically, the server can determine the arrangement position of the starting content feature vector in the content feature vector sequence in the current identification feature vector sequence based on the first sorting as the first starting position, and determine the arrangement positions of each feature vector corresponding to the target content based on the first starting position corresponding to the content feature vector sequence and the number of feature vectors corresponding to the content feature vector sequence. For example, the server can determine the first ending position corresponding to the content feature vector sequence based on the first starting position and the number of feature vectors of the target content, and use each position (including the first starting position and the first ending position) in the current identification feature vector sequence from the first starting position to the first ending position as the feature vector arrangement position corresponding to the target content.

[0077] In some embodiments, the server can determine the arrangement position of the starting information feature vector in the information feature vector sequence in the current identification feature vector sequence based on the second sorting, as the second starting position, and determine the arrangement positions of each feature vector corresponding to the task description information based on the second starting position corresponding to the information feature vector sequence and the number of feature vectors corresponding to the information feature vector sequence. For example, the server can determine the second ending position corresponding to the information feature vector sequence based on the second starting position and the number of feature vectors of the task description information, and use each position (including the second starting position and the second ending position) in the current identification feature vector sequence from the second starting position to the second ending position as the feature vector arrangement position corresponding to the task description information.

[0078] In some embodiments, the server may determine a forward sequence that precedes the content feature vector sequence from the current recognition feature vector sequence based on the first sorting, count the number of feature vectors in the forward sequence that precedes the content feature vector sequence, determine the starting point of the content feature vector sequence based on the statistical value obtained, determine the ending point of the content feature vector sequence based on the starting point of the content feature vector sequence and the number of feature vectors in the content feature vector sequence, and use the vector positions from the starting point to the ending point as the feature vector arrangement positions corresponding to the target content. The forward sequence that precedes the content feature vector sequence refers to the sequence that precedes the content feature vector sequence in the current recognition feature vector sequence.

[0079] In this embodiment, based on the first sorting and the number of feature vectors corresponding to the content feature vector sequence, the position of the target content in the current identification feature vector sequence can be quickly obtained. Based on the second sorting and the number of feature vectors corresponding to the information feature vector sequence, the position of the task description information in the current identification feature vector sequence can be quickly obtained, thereby improving the efficiency of determining the target position.

[0080] In some embodiments, obtaining the feature vector arrangement position of the task description information based on the second sorting and the number of feature vectors corresponding to the information feature vector sequence includes: determining the forward sequence arranged before the information feature vector sequence in the current recognition feature vector sequence based on the second sorting; counting the number of feature vectors of each forward sequence, and determining the starting point of the information feature vector sequence based on the statistical value obtained; determining the ending point of the information feature vector sequence based on the starting point and the number of feature vectors of the information feature vector sequence; and using the vector position from the starting point to the ending point as the feature vector arrangement position of the task description information.

[0081] Among them, the forward sequence of the information feature vector sequence refers to the sequence in the current identification feature vector sequence that is arranged before the information feature vector sequence. For example, if the first sorting is 1 and the second sorting is 2, the content feature vector sequence is arranged before the information feature vector sequence, so the content feature vector sequence is the forward sequence of the information feature vector sequence.

[0082] The starting point of an information feature vector sequence refers to the position of the first information feature vector in the information feature vector sequence within the current identification feature vector sequence. The ending point of an information feature vector sequence refers to the position of the last information feature vector in the information feature vector sequence within the current identification feature vector sequence. The number of feature vectors in an information feature vector sequence refers to the number of feature vectors included in the information feature vector sequence.

[0083] Specifically, the server can add the number of feature vectors in the forward sequence before the information feature vector sequence, and use the result of the addition as a quantitative statistical value. The starting point of the information feature vector sequence is determined based on the quantitative statistical value. For example, 1 can be added to the quantitative statistical value to obtain the starting point. For example, if the quantitative statistical value is 10, the starting point of the information feature vector sequence is the 11th bit in the current identification feature vector sequence.

[0084] In some embodiments, the server can determine the positions whose number is equal to the number of feature vectors of the information feature vector starting from the starting point, and use the last position among the determined positions as the end point of the information feature vector sequence. For example, the starting point is the 11th position, and the number of feature vectors of the information feature vector sequence is 3. Then the positions with a length of 3 starting from the 11th position are the 11th, 12th and 13th positions. Therefore, the 13th position is the end point of the information feature vector sequence, and the positions from the 11th to the 13th position are used as the feature vector arrangement positions of the task description information, that is, the 11th, 12th and 13th positions are used as the feature vector arrangement positions of the task description information.

[0085] In this embodiment, based on the second sorting, the forward sequence arranged before the information feature vector sequence in the current identification feature vector sequence is determined, the number of feature vectors in each forward sequence is counted, and the starting point of the information feature vector sequence is determined according to the statistical value of the number obtained. Based on the starting point and the number of feature vectors of the information feature vector sequence, the ending point of the information feature vector sequence is determined, and the vector position from the starting point to the ending point is used as the feature vector arrangement position of the task description information, thereby improving the efficiency and accuracy of determining the feature vector arrangement position of the task description information.

[0086] In some embodiments, the content feature vector sequence and each information feature vector sequence are arranged according to an arrangement strategy to obtain a current recognition feature vector sequence corresponding to the current moment, including: obtaining the forward feature vector corresponding to the forward recognition character obtained by recognizing each task description information at the previous moment; arranging the content feature vector sequence, each information feature vector sequence and each forward feature vector according to the arrangement strategy to obtain the current recognition feature vector sequence corresponding to the current moment.

[0087] The arrangement strategy may further include the arrangement position of the forward feature vector corresponding to the forward recognition character. The forward feature vector is a vector obtained by extracting features from the forward recognition character, and the forward recognition character includes the recognition character corresponding to each task description information at the previous moment.

[0088] Specifically, the server may concatenate the content feature vector sequence, each information feature vector sequence, and each forward feature vector according to the arrangement strategy to obtain the current recognition feature vector sequence. If a forward recognition character was recognized at the previous moment, the server may concatenate the content feature vector sequence, each information feature vector sequence, and each forward feature vector according to the arrangement strategy to obtain the current recognition feature vector sequence. If no character was recognized at the previous moment, the server may concatenate the content feature vector sequence and each information feature vector sequence according to the arrangement strategy to obtain the current recognition feature vector sequence.

[0089] In some embodiments, the recognition feature vector sequence at each moment also includes the mask feature vectors corresponding to each task description information. The mask feature vector is a vector obtained by extracting features from the mask character. The mask characters corresponding to different task description information may be the same or different. The arrangement strategy may also include the arrangement position of the mask feature vector. The server may arrange the content feature vector sequence, each information feature vector sequence, the forward feature vector, and the mask feature vector according to the arrangement strategy to obtain the current recognition feature vector sequence. For example, at the very beginning of recognition, that is, the first recognition, one time corresponds to one moment. Since no characters have been recognized yet, the recognition feature vector sequence can be expressed as [x1, x2, x3, k00, k01, k10, k11, k12, k20, k21, mask00, mask10, mask20]. During the second recognition process, if the feature vectors corresponding to the characters recognized for the first time are respectively 0 00 ,O 10 ,O 20 , then the second recognition feature vector sequence can be expressed as [x1,x2,x3,k00,k01,k10,k11,k12,k20,k21, 00 ,O 10 ,O 20 ,mask01,mask11,mask21], where O 00 is the identification character corresponding to the first task description information, O 10 is the identification character corresponding to the second task description information, O 20 The identification characters corresponding to the third task description information.

[0090] In this embodiment, the current recognition feature vector sequence includes the forward feature vector corresponding to the forward recognition character recognized at the previous moment. Therefore, when obtaining the current attention weight sequence corresponding to the task description information based on the current recognition feature vector sequence, the forward recognition character corresponding to the task description information can be taken into account, thereby improving the accuracy of the attention weight sequence.

[0091] In some embodiments, the content feature vector sequence, each information feature vector sequence and each forward feature vector are arranged according to an arrangement strategy to obtain a current identification feature vector sequence corresponding to the current moment, including: splicing each information feature vector sequence to obtain a first splicing sequence; according to the order of each information feature vector sequence in the first splicing sequence, the forward feature vectors corresponding to each task description information are spliced to obtain a second splicing sequence; the content feature vector sequence, the first splicing sequence and the second splicing sequence are spliced in sequence to generate the current identification feature vector sequence corresponding to the current moment.

[0092] In the arrangement strategy, the arrangement positions of the information feature vector sequences are continuous, for example, the arrangement positions of the information feature vector sequences are 2, 3, and 4. In the arrangement strategy, the arrangement positions of the forward feature vectors obtained at the same time are continuous, and the arrangement position of the forward feature vector corresponding to the task description information can be determined based on the arrangement position of the information feature vector sequence of the task description information.

[0093] Specifically, the server can determine the arrangement positions corresponding to each information feature vector based on the arrangement strategy. In the arrangement strategy, the arrangement positions of each information feature vector sequence are continuous. The information feature vector sequences are spliced according to the arrangement positions to obtain a first spliced sequence. The closer the arrangement position is, the closer the position of the information feature vector sequence is in the first spliced sequence.

[0094] In some embodiments, the server can arrange and splice the forward feature vectors of the task description information based on the order of the information feature vector sequence of the task description information in the first splicing sequence to obtain a second splicing sequence. The higher the order of the information feature vector sequence in the first splicing sequence, the higher the order of the forward feature vector in the second splicing sequence.

[0095] In some embodiments, in the arrangement strategy, the content feature vector sequence is arranged before the first splicing sequence, and the first splicing sequence is arranged before the second splicing sequence. The server can splice the first splicing sequence after the content feature vector sequence, and splice the second splicing sequence after the first splicing sequence, and generate the current identification feature vector sequence based on the spliced sequences.

[0096] In this embodiment, the information feature vector sequence and the forward feature vector are arranged continuously, thereby improving the arrangement regularity of each vector in the current recognition feature vector sequence.

[0097] In some embodiments, the target position also includes the arrangement position of the forward feature vector corresponding to the task description information in the current recognition feature vector sequence, and the step of obtaining the target position includes: obtaining the vector position of the first length starting from the starting position in the current recognition feature vector sequence as the feature vector arrangement position corresponding to the target content; the first length is the number of feature vectors in the content feature vector sequence; for the information feature vector sequence corresponding to the task description information, obtaining the forward information feature vector sequence of the first splicing sequence; counting the number of feature vectors in the forward information feature vector sequence and the number of feature vectors in the content feature vector sequence to obtain the first statistical quantity, and taking the backward position of the position sorted as the first statistical quantity as the first position of the information feature sequence; obtaining the vector position of the second length starting from the first position as the feature vector arrangement position of the task description information; the second length is the number of feature vectors in the information feature vector sequence.

[0098] Among them, the content feature vector sequence, the first splicing sequence and the second splicing sequence in the current recognition feature vector sequence are arranged in sequence, that is, the content feature vector sequence is arranged at the head of the current recognition feature vector sequence, the first splicing sequence is arranged after the content feature vector sequence, and the second splicing sequence is arranged after the first splicing sequence. For example, the current recognition feature vector sequence can be [x1, x2, x3, k00, k01, k10, k11, k12, k20, k21, 00 ,O 10 ,O 20 ], where "x1, x2, x3" is the content feature vector sequence, "k00, k01, k10, k11, k12, k20, k21" is the first splicing sequence, and " 00 ,O 10 ,O 20 " is the second concatenated sequence. The first length is the number of feature vectors included in the content feature vector sequence.

[0099] The forward information feature vector sequence in the first concatenated sequence refers to the information feature vector sequences that precede the information feature vector sequence in the first concatenated sequence. For example, if the first concatenated sequence is [k00, k01, k10, k11, k12, k20, k21], "k00, k01" is the information feature vector sequence corresponding to the first task description, "k10, k11, k12" is the information feature vector sequence corresponding to the second task description, and "k20, k21" is the information feature vector sequence corresponding to the third task description. Therefore, the forward information feature vector sequence corresponding to the information feature vector sequence of the second task description is "k00, k01." The second length is the number of feature vectors included in the information feature vector sequence. For example, if the length of "k00, k01" is 2, the second length corresponding to "k00, k01" is also 2. The first position of the information feature vector sequence is the position of the starting feature vector in the information feature vector sequence within the current recognition feature vector sequence.

[0100] Specifically, the server can determine the portion before the information feature vector sequence from the first splicing sequence as the forward information feature vector sequence corresponding to the information feature vector sequence. The server can count the number of feature vectors in the forward feature vector sequence corresponding to the information feature vector sequence as the forward feature vector number, count the number of feature vectors in the content feature vector sequence as the content feature vector number, sum the number of content feature vectors and the number of forward feature vectors, and use the sum as the first statistical number. The server can determine the position ranked as the first statistical number from the current recognition feature vector sequence, and use the backward position of the determined position as the first position of the information feature vector sequence. Taking the information feature vector sequence corresponding to the second task description information as an example, if the first statistical number is 10, the position ranked 10 can be determined from the current recognition feature vector sequence. The backward position is the position ranked 11, and the position ranked 11 can be used as the first position of the information feature vector sequence corresponding to the second task description information.

[0101] In some embodiments, the arrangement strategy includes the sorting of each information feature vector sequence in the first splicing sequence. The server can determine the information feature vector sequence arranged before the information feature vector sequence in the first splicing sequence based on the sorting of the information feature vector sequence in the first splicing sequence, as the forward information feature vector sequence, count the number of feature vectors of each forward information feature vector sequence to obtain a third length, obtain the vector position of the third length starting from the starting position in the first splicing sequence, and use the part corresponding to the vector position of the third length in the first splicing sequence as the forward information feature vector sequence corresponding to the information feature vector sequence.

[0102] In some embodiments, the server may count the number of feature vectors in the information feature vector sequence, use the counted result as the second length, and obtain the vector position of the second length starting from the first position as the feature vector arrangement position of the task description information corresponding to the information feature vector sequence, where the feature vector arrangement position includes the first position. The feature vector arrangement position of each task description information can be determined separately using the method of this embodiment.

[0103] In some embodiments, the content feature vector sequence is arranged at the head of the current recognition feature vector sequence, the first splicing sequence is arranged after the content feature vector sequence, and the second splicing sequence is arranged after the first splicing sequence. The target position also includes the arrangement position of the forward feature vector corresponding to the task description information in the current recognition feature vector sequence. The arrangement strategy includes the order of the forward feature vectors corresponding to each task description information in the second splicing sequence. The server can count the number of forward feature vectors arranged before the forward feature vector based on the order of the forward feature vector in the second splicing sequence as the second statistical quantity, count the number of content feature vectors and the number of feature vectors in the first splicing sequence as the third statistical quantity, add the second statistical quantity and the third statistical quantity to obtain a fourth statistical quantity, and use the backward position of the position ranked as the fourth statistical quantity in the current recognition feature vector sequence as the arrangement position of the forward feature vector of the task description information.

[0104] In this embodiment, the vector position of the first length is obtained from the starting position in the current recognition feature vector sequence as the feature vector arrangement position corresponding to the target content. For the information feature vector sequence corresponding to the task description information, the forward information feature vector sequence of the first splicing sequence is obtained, the number of feature vectors in the forward information feature vector sequence and the number of feature vectors in the content feature vector sequence are counted to obtain the first statistical quantity, the backward position of the position sorted as the first statistical quantity is used as the first position of the information feature sequence, and the vector position of the second length is obtained from the first position as the feature vector arrangement position of the task description information, thereby accurately determining the position corresponding to the target content in the target position and the position corresponding to the task description information.

[0105] In some embodiments, the current recognition feature vector sequence also includes mask feature vectors corresponding to each task description information at the current moment; the current attention weight outside the target position in the current attention weight sequence is masked, and attention processing is performed based on the current attention weight corresponding to the target position and the current recognition feature vector sequence to obtain the attention feature vector, including: obtaining the mask feature position of the mask feature vector corresponding to the task description information in the current recognition feature vector sequence; obtaining the matrix object corresponding to the mask feature position from the target weight masking matrix as the current weight masking sequence corresponding to the current attention weight sequence; the element value at the target position in the current weight masking sequence is the retention coefficient, and the element value outside the target position is the shielding coefficient; attention processing is performed based on the current weight masking sequence, the current attention weight sequence and the current recognition feature vector sequence to obtain the attention feature vector.

[0106] The mask feature position refers to the position of the mask feature vector in the current recognition feature vector sequence. The mask feature vector corresponding to the task description information can also be referred to as the mask feature vector corresponding to the information feature vector sequence of the task description information. The current recognition feature vector sequence also includes the mask feature vector corresponding to each task description information at the current moment.

[0107] The target weight masking matrix may be pre-set, and the target weight masking matrix includes matrix objects corresponding to each mask eigenvector. The matrix object is a row vector in the matrix.

[0108] The retention coefficient is different from the shielding coefficient. The retention coefficient and the shielding coefficient can be set as needed, for example, the retention coefficient is 1 and the shielding coefficient is 0.

[0109] Specifically, the arrangement strategy may include the arrangement methods corresponding to each mask feature vector. For example, the arrangement strategy may specify that each mask feature vector is arranged continuously and arranged according to the order of the information feature vector sequence corresponding to the mask feature vector in the first splicing sequence. For example, the server may arrange the mask feature vectors corresponding to each information feature vector sequence based on the order of each information feature vector sequence in the first splicing sequence, and use the arranged sequence as the mask feature vector sequence. The higher the order of the information feature vector sequence in the first splicing sequence, the higher the order of the mask feature vector corresponding to the information feature vector in the mask feature vector sequence. For example, the current recognition feature vector sequence may be:

[0110] [x1,x2,x3,k00,k01,k10,k11,k12,k20,k21,O 00 ,O 10 ,O 20 ,mask01,mask11,mask21], where "mask01" is the mask feature vector corresponding to "k00,k01", "mask11" is the mask feature vector corresponding to "k10,k11,k12", "mask21" is the mask feature vector corresponding to "k20,k21", and "mask01,mask11,mask21" is a sequence of mask feature vectors. The method for determining the position of mask features can refer to the method for determining the arrangement position of the forward feature vector corresponding to the task description information, which will not be repeated here.

[0111] In some embodiments, the server can obtain the row vector corresponding to the mask feature position from the target weight masking matrix, and use the row vector as the current weight masking sequence corresponding to the corresponding current attention weight sequence. For example, the mask feature vector corresponding to the first task description information is ranked 15th in the current recognition feature vector sequence, and the mask feature position of the mask feature vector of the first task description information is 15th. The server can obtain the row vector of the 15th row from the target weight masking matrix, and use the row vector of the 15th row as the current weight masking sequence corresponding to the current attention weight sequence of the first task description information.

[0112] In some embodiments, the number of rows of the target weight shielding matrix is greater than a quantity threshold, the number of columns of the target weight shielding matrix is equal to the number of rows, and the quantity threshold is the result obtained by adding the number of feature vectors of the target content, the number of feature vectors of the first splicing sequence, and the number of identification feature vectors. The number of identification feature vectors can be set as needed, and the total number of characters of the purpose information corresponding to each task description information is less than or equal to the number of identification feature vectors. For example, if there are 3 task descriptions and the purpose information is 2, 3, and 4 respectively, then the number of identification feature vectors is greater than 2+3+4=9. The target weight shielding matrix may include matrix elements corresponding to each identification feature vector in the current identification feature vector sequence. The server may determine the arrangement position of the identification feature vector in the current identification feature vector sequence as the identification arrangement position, determine the row vector corresponding to the identification arrangement position from the target weight shielding matrix, and obtain the matrix object corresponding to the identification feature vector. For example, if the current recognition feature vector sequence is [x1,x2,x3,x4,k00,k01,k10,k11,k12,k20,k21,mask00,mask10,mask20], the target weight masking matrix can be as follows Figure 6 As shown, Figure 6 The first to fourth rows in the matrix are the matrix objects corresponding to the content feature vectors, the fifth to eleventh rows are the matrix objects corresponding to the information feature vectors, and the twelfth row and the rows after that are the matrix objects corresponding to the mask feature vectors. Figure 6The gray rectangular box in the figure is the target position, the value corresponding to the gray matrix box is the retention coefficient, and the value corresponding to the white rectangular box is the shielding coefficient. The current weight shielding sequence includes the weight processing values corresponding to each attention weight in the current attention weight sequence. The weight processing value is the element value in the current weight shielding sequence, that is, the retention coefficient or shielding coefficient. The position of the attention weight in the current weight shielding sequence is consistent with the position of the weight processing value corresponding to the attention weight in the current weight shielding sequence. Since the order of the attention weights in the current attention weight sequence is consistent with the order of the recognition feature vectors in the current recognition feature vector sequence, that is, the closer the position of the recognition feature vector in the current recognition feature vector sequence is, the closer the position of the attention weight corresponding to the recognition feature vector in the current attention weight sequence is, the closer the position of the attention weight corresponding to the recognition feature vector in the current attention weight sequence is. Therefore, the arrangement of the weight processing values in the current weight shielding sequence is consistent with the arrangement of the recognition feature vectors. For example, since "mask00" is arranged in the 12th position in the current recognition feature vector sequence, the row vector of the 12th row in the target weight masking matrix can be used as the matrix object corresponding to "mask00", that is, the row vector of the 12th row is used as the current weight masking sequence of the current attention weight sequence corresponding to the task description information corresponding to the information feature vector sequence "k00, k01". Then the element in the 1st column in the 12th row is the weight processing value corresponding to the attention weight arranged in the 1st position in the current attention weight sequence, and is also the weight processing value corresponding to the recognition feature vector arranged in the 1st position in the current recognition feature vector sequence. Since the recognition feature vector arranged in the 1st position in the current recognition feature vector sequence is x1, the element in the 12th row and the 1st column is the weight processing value used to perform weight processing on the attention weight generated by x1. Since the 1st column in the 12th row is the retention coefficient, the result of the weight processing is to retain the attention weight generated by x1. Since the 7th-order recognition feature vector in the current recognition feature vector sequence is k10, the element in the 7th column of the 12th row is the weighted processing value used to weight the attention weight generated by k10. Since the 7th column of the 12th row is the shielding coefficient, the result of the weighting processing is the attention weight generated by shielding k10. Since k10 is the information feature vector corresponding to the second task description information, "mask00" is the mask feature vector corresponding to the first task description information, and the process of obtaining the recognized characters of the first task description information is independent of the second task description information, shielding the attention weight generated by k10 can improve the accuracy of character recognition. The process of recognizing characters is related to the target content, so retaining the attention weight generated by x1 can improve the accuracy of character recognition.

[0113] In some embodiments, the server may use the current weight masking sequence to mask the current attention weight sequence to obtain a processed current attention weight sequence. For example, the server may multiply the current attention weight sequence with the elements at corresponding positions in the current weight masking sequence to obtain a processed current attention weight sequence. For example, if the current weight masking sequence is [1, 0, 1] and the current attention weight sequence is [0.2, 0.3, 0.4], then the processed current attention weight sequence is [0.2, 0, 0.4]. Attention processing may be performed based on the processed current attention weight sequence and the current recognition feature vector sequence to obtain an attention feature vector.

[0114] In this embodiment, the matrix object corresponding to the mask feature position is obtained from the target weight masking matrix as the current weight masking sequence corresponding to the current attention weight sequence. Attention processing is performed based on the current weight masking sequence, the current attention weight sequence and the current recognition feature vector sequence to obtain the attention feature vector, thereby improving the accuracy of the attention feature vector.

[0115] In some embodiments, attention processing is performed based on the current weight masking sequence, the current attention weight sequence and the current recognition feature vector sequence to obtain the attention feature vector, including: using the retention coefficient at the target position in the current weight masking sequence to retain the current attention weight at the target position in the current attention weight sequence, and using the masking coefficient outside the target position in the current weight masking sequence to mask the current attention weight outside the target position in the current attention weight sequence; obtaining the processed current attention weight sequence; performing attention processing on the current recognition feature vector sequence based on the processed current attention weight sequence to obtain the attention feature vector.

[0116] Specifically, the retention processing and the shielding processing can be multiplication operations. For example, the server can multiply the retention coefficient at the target position with the current attention weight at the target position, and use the result of the multiplication operation as the processed current attention weight. The server can multiply the shielding coefficient at the non-target position with the current attention weight at the non-target position, and use the result of the multiplication operation as the processed current attention weight. According to the order of the current attention weight in the current attention weight sequence, the processed current attention weights are arranged to obtain the processed current attention weight sequence. Among them, the non-target position refers to the position outside the target position.

[0117] In this embodiment, the retention coefficient at the target position in the current weight shielding sequence is used to retain the current attention weight at the target position in the current attention weight sequence, and the shielding coefficient outside the target position in the current weight shielding sequence is used to shield the current attention weight outside the target position in the current attention weight sequence; the processed current attention weight sequence is obtained, which improves the shielding efficiency of the attention weight.

[0118] In some embodiments, the step of generating a target weight masking matrix includes: obtaining an initial weight masking matrix; determining the mask arrangement position of the mask feature vector corresponding to the task description information at each recognition moment in the arrangement strategy in the recognition feature vector sequence; determining the mask matrix object corresponding to the mask arrangement position in the initial weight masking matrix; determining the content position corresponding to the target content and the information position corresponding to the task description information in the mask matrix correspondence; adjusting the matrix element values of the initial weight masking matrix based on the content position and the information position to obtain the target weight masking matrix, in which the matrix element values corresponding to the content position and the information position are set as retention coefficients, and the matrix element values other than the content position and the information position are set as masking coefficients.

[0119] The number of rows of the initial weight shielding matrix is equal to the number of columns, and the number of rows of the initial weight shielding matrix is greater than the quantity threshold. The number of rows of the initial weight shielding matrix is greater than the quantity threshold, and the quantity threshold is the result of adding the number of eigenvectors of the target content, the number of eigenvectors of the first splicing sequence, and the number of recognition eigenvectors. The matrix element values in the initial weight shielding matrix can all be shielding coefficients or all be retention coefficients. The recognition moment refers to the moment when the character is recognized, and different recognition moments correspond to different recognition feature vector sequences.

[0120] The mask arrangement position refers to the arrangement position of the mask feature vector in the recognition feature vector sequence. If the recognition process goes through 5 moments, there are 5 recognition feature vector sequences. For each task description information, there is a mask feature vector corresponding to each of the 5 moments, and the mask arrangement position corresponding to the mask feature vector at different moments is different. For example, at the first moment, the recognition feature vector sequence is [x1,x2,x3,x4,k00,k01,k10,k11,k12,k20,k21,mask00,mask10,mask20], then the positions of the mask feature vectors corresponding to the three task description information are 12, 13 and 14 respectively. At the second moment, the recognition feature vector sequence is [x1,x2,x3,x4,k00,k01,k10,k11,k12,k20,k21, 00 ,O 10 ,O 20,mask01,mask11,mask21], the positions of the mask feature vectors corresponding to the three task description information are 15, 16 and 17 respectively.

[0121] The content position is the position in the mask matrix object that corresponds to the target content, and the information position is the position in the mask matrix object that corresponds to the task description information. The content position is consistent with the position of the target content's content feature vector sequence in the recognition feature vector sequence, and the information position is consistent with the position of the task description information's information feature vector sequence in the recognition feature vector sequence.

[0122] Specifically, the server can obtain the row vector corresponding to the mask arrangement position from the initial weight masking matrix as the mask matrix object for the mask feature vector corresponding to the mask arrangement position. For example, at the first moment, the recognition feature vector sequence is [x1, x2, x3, x4, k00, k01, k10, k11, k12, k20, k21, mask00, mask10, mask20]. The server can obtain the row vector of the 12th row from the initial weight masking matrix as the mask matrix object corresponding to mask00. The server can use the position of the content feature vector sequence in the recognition feature vector sequence as the content position corresponding to the target content in the mask matrix object. For example, if the content feature vector sequence "x1, x2, x3, x4" is positioned from bit 1 to bit 4, then bits 1 to 4 in the mask matrix object are the content position. Similarly, the server can use the position of the information feature vector sequence in the recognition feature vector sequence as the information position corresponding to the task description information in the mask matrix object.

[0123] In some embodiments, the matrix element values in the initial weight masking matrix are masking coefficients, and the server may update the matrix element values corresponding to the content position and the information position from masking coefficients to retention coefficients, while keeping the matrix element values outside the content position and the information position unchanged, to obtain a target weight masking matrix. For example, the server may update the matrix element values at the content position and the information position in the mask matrix object from masking coefficients to retention coefficients to obtain a target weight masking matrix.

[0124] In some embodiments, the server can update the matrix element values at the content position, information position and mask arrangement position at the recognition moment in the mask matrix object from the shielding coefficient to the retention coefficient, and keep the matrix element values outside the content position, information position and mask arrangement position unchanged to obtain the target weight shielding matrix.

[0125] In some embodiments, the server can determine the recognition character arrangement position of the recognition character of the task description information at each recognition moment, and the position of the recognition character arrangement position is consistent with the position of the feature vector of the recognition character in the recognition feature vector sequence. The server can update the matrix element values at the content position, information position, mask arrangement position at the recognition moment, and the recognition character arrangement position corresponding to the moment before the recognition moment in the mask matrix object from the shielding coefficient to the retention coefficient, and keep the matrix element values other than the content position, information position, mask arrangement position at the recognition moment, and the recognition character arrangement position corresponding to the moment before the recognition moment unchanged to obtain the target weight shielding matrix. For example, according to the provisions of the arrangement strategy, the recognition feature vector sequence of the second recognition moment is determined to be [x1,x2,x3,x4,k00,k01,k10,k11,k12,k20,k21,0 00 ,O 10 ,O 20 ,mask01,mask11,mask21], that is, the permutation strategy stipulates that, O 00 is the feature vector corresponding to the character recognized by the first task description information at the first recognition moment, O 10 is the feature vector corresponding to the character recognized by the second task description information at the first recognition moment, O 20 is the feature vector corresponding to the character recognized by the first task description information at the third recognition moment. At the second recognition moment, since the mask feature vector of the first task description information is arranged in the 15th position, the vector of the 15th row in the initial weight masking matrix is the mask matrix object corresponding to the mask feature vector of the first task description information at the second moment (referred to as mask matrix object 15). Since the recognition character O of the first task description information recognized at the first moment is 10 It is arranged at the 12th position, so the recognition character is arranged at the 12th position, so the server can update the matrix element value arranged at the 12th position in the mask matrix object 15 from the shielding coefficient to the retaining coefficient.

[0126] In this embodiment, the matrix element values of the initial weight shielding matrix are adjusted based on the content position and the information position to obtain a target weight shielding matrix. In the target weight shielding matrix, the matrix element values corresponding to the content position and the information position are set as retention coefficients, and the matrix element values outside the content position and the information position are set as shielding coefficients, thereby improving the accuracy of the target weight shielding matrix.

[0127] In some embodiments, the step of generating a target weight shielding matrix includes: obtaining an initial weight shielding matrix; determining a first matrix object corresponding to the information feature vector sequence from the initial output shielding matrix based on the arrangement position of the information feature vector sequence in the recognition feature vector sequence in the arrangement strategy; determining the information matrix position corresponding to the information feature vector sequence from the first matrix object based on the arrangement position of the information feature vector sequence in the recognition feature vector sequence in the arrangement strategy; determining the mask matrix position corresponding to the mask feature vector at each recognition moment from the first matrix object based on the mask arrangement position of the mask feature vector of the information feature vector sequence at multiple recognition moments in the recognition feature vector sequence in the arrangement strategy; setting the matrix element values at the information matrix position and each mask matrix position in the first matrix object as retention coefficients, and setting the matrix element values other than the information matrix position and each mask matrix position in the first matrix object as shielding coefficients to obtain a target output shielding matrix; determining the output matrix area from the initial weight shielding matrix based on the mask arrangement position of the mask feature vector of the information feature vector sequence at multiple recognition moments in the recognition feature vector sequence in the arrangement strategy; setting the matrix element values of the output matrix area using the target output shielding matrix to generate a target weight shielding matrix.

[0128] The relevant contents of the initial weight shielding matrix can be referred to the relevant description in the above embodiment and will not be repeated here. The number of columns of the initial output shielding matrix is equal to the number obtained by adding the number of eigenvectors of the first splicing sequence and the number of identification eigenvectors, and the number of rows of the initial output shielding matrix is greater than the number of identification eigenvectors. For example, the number of rows of the initial output shielding matrix can be consistent with the number of columns.

[0129] Specifically, in the arrangement strategy, the sequence of recognition feature vectors is arranged from front to back in the order of content feature vector sequence, each information feature vector sequence, each forward feature vector corresponding to each task description information, and each mask feature vector corresponding to each task description information. For example, the recognition feature vector sequence is [x1,x2,x3,x4,k00,k01,k10,k11,k12,k20,k21, 00 ,O 10 ,O 20 ,mask01,mask11,mask21].

[0130] In some embodiments, the server can count the number of each information feature vector sequence to obtain the number of information sequences, and divide the initial output shielding matrix horizontally according to the number of information sequences to obtain divided matrix regions, each divided matrix region including the row vectors of the initial output shielding matrix of the number of information sequences. For example, if the number of information sequences is 3 and the number of rows of the initial output shielding matrix is 9, then three divided matrix regions are obtained, each of which includes 3 rows. The server can determine the row vector corresponding to the arrangement position of each information feature vector sequence from the divided matrix region as the first matrix object corresponding to the information feature vector sequence based on the arrangement position. The closer the arrangement position of the information feature vector sequence is, the closer the corresponding first matrix object is in the divided matrix region. For example, there are three information feature vector sequences, namely A, B, and C. The rankings of A, B, and C in the arrangement strategy are 2, 3, and 4, respectively. Then, in the divided matrix region, the first row is the first matrix object of A, the second row is the first matrix object of B, and the third row is the first matrix object of C.

[0131] In some embodiments, for each information feature vector sequence, the server can determine each information feature vector sequence arranged before the information feature vector sequence, respectively as the forward information feature vector sequence corresponding to the information feature vector sequence, count the number of feature vectors of each forward information feature vector sequence, use the counted number as the first forward feature vector number, use the backward adjacent position of the position sorted as the first forward feature vector number in the first matrix object as the first matrix position corresponding to the information feature vector sequence, and obtain the position of the second length starting from the first matrix position as the information matrix position corresponding to the information feature vector sequence, where the information matrix position includes the first matrix position.

[0132] In some embodiments, the server may determine the number of eigenvectors arranged before the mask eigenvector, use the counted number as the second forward eigenvector number, and use the backward adjacent position of the position sorted as the second forward eigenvector number in the first matrix object as the mask matrix position of the mask eigenvector.

[0133] In some embodiments, the server may overwrite matrix element values in the output matrix region with matrix element values in the target output mask matrix, thereby making the output matrix region consistent with the target output mask matrix.

[0134] In some embodiments, the server can obtain an initial information shielding matrix, determine the matrix regions corresponding to each information feature vector sequence from the initial information shielding matrix based on the arrangement position of the information feature vector sequence in the arrangement strategy, and set each matrix region to a lower triangular matrix to obtain a target information shielding matrix. The server can determine the matrix regions corresponding to each information feature vector sequence from the initial information shielding matrix based on the arrangement position of the information feature vector sequence in the arrangement strategy and the number of eigenvectors of the information feature vector sequence. For example, there are three information feature vector sequences, namely A, B, and C, with arrangement positions of 2, 3, and 4 respectively, the length of A is 2, the length of B is 3, and the length of C is 2. Then, the region consisting of the 1st row and the 2nd row and the 1st column and the 2nd column in the initial information shielding matrix is used as the matrix region of A, the region consisting of the 3rd row and the 5th row and the 3rd column and the 5th column in the initial information shielding matrix is used as the matrix region of B, and the region consisting of the 1st row and the 2nd row and the 6th column and the 7th column in the initial information shielding matrix is used as the matrix region of C.

[0135] In some embodiments, the server can determine the information matrix area from the initial weight shielding matrix, cover the information matrix area with the target information shielding matrix, and cover the output matrix area with the target output shielding matrix to generate a target weight shielding matrix. After covering the information matrix area and the output matrix area, the server can convert the covered initial weight shielding matrix into a lower triangular matrix, that is, the element values in the upper triangular area of the covered initial weight shielding matrix are all set to shielding coefficients, and the element values in the lower triangular area of the covered initial weight shielding matrix remain unchanged. After converting the covered initial weight shielding matrix into a lower triangular matrix, the server can obtain the number of eigenvectors corresponding to the content eigenvector sequence as the number of content vectors, and use the matrix composed of column vectors corresponding to the number of content vectors in the initial weight shielding matrix after conversion to the lower triangular matrix as the input matrix, set the element values in the input matrix to the retention coefficient, and use the set initial weight shielding matrix as the target weight shielding matrix. The column vector corresponding to the number of content vectors refers to the column vector that is ranked as the number of content vectors and the column vector that is ranked before the number of content vectors. For example, if the number of content vectors is 4, the column vector corresponding to the number of content vectors is the 1st to 4th column, including the 1st and 4th columns.

[0136] In this embodiment, the target output shielding matrix is first generated, and then the initial weight shielding matrix is set using the target output shielding matrix, thereby improving the efficiency of generating the target weight shielding matrix.

[0137] In some embodiments, based on the mask arrangement positions of the mask feature vectors of the information feature vector sequence at multiple recognition moments in the arrangement strategy in the recognition feature vector sequence, determining the output matrix area from the initial weight shielding matrix includes: based on the mask arrangement positions of the mask feature vectors of each information feature vector sequence at each recognition moment in the arrangement strategy in the recognition feature vector sequence, determining the row vectors corresponding to each mask arrangement position from the initial weight shielding matrix; based on each mask arrangement position and the information arrangement position of each information feature vector sequence in the recognition feature vector sequence, determining the column vectors corresponding to each mask arrangement position and each information arrangement position from the initial weight shielding matrix; obtaining the overlapping area of each row vector and each column vector, and using the overlapping area as the output matrix area.

[0138] For example, Figure 7 As shown, at the first recognition moment, the mask arrangement positions of the mask feature vectors of the three keys are 12, 13 and 14 respectively. At the second recognition moment, the mask arrangement positions of the mask feature vectors of the three keys are 15, 16 and 17 respectively. At the third recognition moment, the mask arrangement positions of the mask feature vectors of the three keys are 18, 19 and 20 respectively. Therefore, the 12th to 20th rows in the matrix are the row vectors corresponding to the respective mask arrangement positions. Similarly, Figure 7 The 5th to 20th columns of the matrix are the column vectors corresponding to each mask arrangement position and each information arrangement position, and the output matrix area is the overlapping area between the 12th to 20th rows and the 5th to 20th columns of the matrix.

[0139] In this embodiment, the output matrix region is accurately determined by the position of the mask feature vector and the position of the information feature vector sequence.

[0140] This application also provides an application scenario, which applies the above-mentioned content information identification method. Specifically, Figure 12 As shown, the application of the content information identification method in this application scenario is as follows:

[0141] S1202: Obtain a target text to be recognized, and obtain multiple target question information corresponding to the target text.

[0142] Among them, the target text is a specific form of target content.

[0143] S1204, extracting features from each character in the target text to obtain a text feature vector sequence, and extracting features from each character in the target question information to obtain an information feature vector sequence;

[0144] S1206, obtaining the mask feature vectors corresponding to each target question information at the first recognition moment;

[0145] S1208: sequentially concatenate the text feature vector sequence, each information feature vector sequence, and each mask feature vector sequence to obtain a first recognition feature vector sequence corresponding to the first recognition moment;

[0146] The closer the information feature vector sequence is to the front of the first recognition feature vector sequence, the closer the mask feature vector of the target question information corresponding to the information feature vector sequence is to the front of the first recognition feature vector sequence.

[0147] For example, suppose that the answers to three questions need to be extracted from the target text. The lengths (number of characters) of the three questions are 2, 3, and 2 respectively, and the target text length is 4. Then the first recognition feature vector sequence can be as follows: Figure 8 As shown, input is the target text, the feature vectors corresponding to the four characters of the target text are x1-x4 respectively, key0, key1 and key2 are the information feature vector sequences corresponding to the three target question information respectively, mask00 is the mask feature vector corresponding to key0, mask10 is the mask feature vector corresponding to key1, and mask20 is the mask feature vector corresponding to key2.

[0148] S1210. Obtain a first attention weight sequence corresponding to each target question information at the first recognition moment based on the first recognition feature vector sequence.

[0149] S1212: Obtain an initial output masking matrix, and determine a first matrix object corresponding to the information feature vector sequence from the initial output masking matrix based on the arrangement position of the information feature vector sequence in the recognition feature vector sequence.

[0150] The same information feature vector sequence has the same arrangement position in the recognition feature vector sequence at each recognition moment, and the same text feature vector sequence has the same arrangement position in the recognition feature vector sequence at each recognition moment.

[0151] S1214. Based on the arrangement position of the information feature vector sequence in the identification feature vector sequence, determine the information matrix position corresponding to the information feature vector sequence from the first matrix object.

[0152] S1216. Based on the mask arrangement positions of the mask feature vectors of the information feature vector sequence at multiple recognition moments in the recognition feature vector sequence, determine the mask matrix position corresponding to the mask feature vector at each recognition moment from the first matrix object.

[0153] S1218. Set the matrix element values at the information matrix position and each mask matrix position in the first matrix object as the retention coefficients, and set the matrix element values other than the information matrix position and each mask matrix position in the first matrix object as the shielding coefficients to obtain the target output shielding matrix.

[0154] S1220. Obtain an initial information shielding matrix. Based on the arrangement position of the information feature vector sequence in the arrangement strategy, determine the matrix regions corresponding to each information feature vector sequence from the initial information shielding matrix. Set each matrix region as a lower triangular matrix to obtain a target information shielding matrix.

[0155] S1222. Determine the information matrix area from the initial weight shielding matrix, determine the output matrix area from the initial weight shielding matrix based on the mask arrangement position of the mask feature vector of the information feature vector sequence at multiple recognition moments in the recognition feature vector sequence in the arrangement strategy, cover the information matrix area with the target information shielding matrix, and cover the output matrix area with the target output shielding matrix to obtain the covered initial weight shielding matrix.

[0156] S1224. After converting the covered initial weight shielding matrix into a lower triangular matrix, obtain the number of eigenvectors corresponding to the content eigenvector sequence as the number of content vectors, set the element value in the column vector corresponding to the number of content vectors in the initial weight shielding matrix after conversion into a lower triangular matrix as the retention coefficient, and generate the target weight shielding matrix.

[0157] Among them, the column vector corresponding to the number of content vectors refers to the column vector sorted as the number of content vectors and sorted before the number of content vectors. For example, if the number of content vectors is 4, the column vectors corresponding to the number of content vectors are columns 1 to 4, including columns 1 and 4.

[0158] S1226. For each target question information, obtain the position of the mask feature vector of the target question information in the first recognition feature vector sequence as the mask feature position, and obtain the row vector corresponding to the mask feature position from the target weight masking matrix as the first weight masking sequence corresponding to the target question information.

[0159] S1228. Perform weight processing on the first attention weight sequence of the target question information based on the first weight masking sequence of the target question information to obtain the processed first attention weight sequence, perform attention processing based on the processed first attention weight sequence and the first recognition feature vector sequence to obtain the attention feature vector corresponding to the target question information.

[0160] The server can multiply the first weight masking sequence with the elements at the corresponding positions of the first attention weight sequence, and use the multiplication result as the processed first attention weight sequence. For example, when the target text length is 4, it is necessary to extract the answers corresponding to 3 questions from the target text, and the lengths (number of characters) of the 3 questions are 2, 3, and 2 respectively, the target weight masking matrix can be as follows: Figure 6 As shown, taking the first target question information (key0) as an example, at the first recognition moment, the 12th row in the target weight shielding matrix is the first weight shielding sequence corresponding to the first target question information. It is easy to understand that the 1st-4th elements in the first attention weight sequence respectively represent the attention weights generated by the 4 characters of the target text for the recognition characters of the first target question information, the 5th-6th elements in the first attention weight sequence represent the attention weights generated by the 2 characters in the first target question information for the recognition characters of the first target question information, the 7th-9th elements in the first attention weight sequence represent the attention weights generated by the 3 characters in the second target question information for the recognition characters of the first target question information, the 10th-11th elements in the first attention weight sequence represent the attention weights generated by the 2 characters in the third target question information for the recognition characters of the first target question information, and the 12th element and the elements after the 12th element represent the attention weights generated by the characters recognized at each recognition moment for the recognition characters of the first target question information. Since the 1st-4th element values and the 5th-6th element values in the first weight masking sequence (the 12th row in the matrix) are retention coefficients, when the first attention weight sequence is multiplied by the elements at the corresponding positions in the first weight masking sequence, the 1st-6th attention weights in the first attention weight sequence can be retained, that is, the attention weights generated by the target text and the first target question information are retained, while the attention weights generated by the second target question information, the third target question information, and the characters recognized at each recognition moment after the first recognition moment are masked. Since the values of each element in the first attention weight sequence can be obtained by performing attention calculations on the mask feature vector of the first target question information and each feature vector in the first recognition feature vector sequence, an effect is achieved: Figure 9 As shown in the figure, at the first recognition moment, mask00 has attention with input and key0, mask10 has attention with input and key1, and mask20 has attention with input and key2.

[0161] S1230. Obtain a character feature vector of the target question information at the first recognition moment based on the attention feature vector.

[0162] Among them, the attention feature vector can be used as the character feature vector.

[0163] S1232. Sequentially concatenate the text feature vector sequence, each information feature vector sequence, each mask feature vector, and the character feature vector obtained at the first recognition moment to obtain a second recognition feature vector sequence corresponding to the second recognition moment.

[0164] The closer the information feature vector sequence is to the front of the first recognition feature vector sequence, the closer the character feature vector of the target question information corresponding to the information feature vector sequence is to the front of the first recognition feature vector sequence. The second recognition feature vector sequence is, for example, [x1, x2, x3, x4, k00, k01, k10, k11, k12, k20, k21, 00 ,O 10 ,O 20 ,mask01,mask11,mask21]. Figure 10 As shown, “input,k0,k1,k2,m00,m10,m20” is the first recognition feature vector sequence, input = “x1,x2,x3,x4”, k0 = “k00,k01”, k1 = “k10,k11,k12”, k2 = “k20,k21”, O 00 is the character recognized by k0 at the first recognition moment, O 10 is the character recognized by k1 at the first recognition moment, O 20 is the character recognized by k2 at the first recognition moment. It should be noted that although the input display at the second recognition moment in the figure is “ 00 ,O 10 ,O 20 ,m01,m11,m21", but this is an abbreviation. The actual second recognition feature vector sequence is "input,k0,k1,k2,O 00 ,O 10 ,O 20 ,m01,m11,m21", that is, at the second recognition moment, the recognition feature vector sequence includes the characters recognized at the first recognition moment.

[0165] S1234. Based on the second recognition feature vector sequence, obtain a second attention weight sequence corresponding to each target question information at the second recognition moment.

[0166] S1236. For each target question information, obtain the position of the mask feature vector of the target question information in the second recognition feature vector sequence as the mask feature position, and obtain the row vector corresponding to the mask feature position from the target weight masking matrix as the second weight masking sequence corresponding to the target question information.

[0167] S1238. Perform weight processing on the second attention weight sequence of the target question information based on the second weight masking sequence of the target question information to obtain a processed second attention weight sequence, perform attention processing based on the processed second attention weight sequence and the second recognition feature vector sequence to obtain an attention feature vector corresponding to the target question information.

[0168] For example, take the first target question information as an example. At the second recognition moment, the 15th row in the target weight masking matrix is the second weight masking sequence corresponding to the first target question information. Compared with the 12th row, the 12th bit in the second weight masking sequence is the retention coefficient. Since the 12th bit in the second attention weight sequence is the attention weight generated by the character recognized by the first target question information at the first recognition moment, that is to say, the processed second attention weight sequence retains the attention weight generated by the output result of the first target question information recognized at the first moment, thereby achieving an effect: Figure 11 As shown, mask01 has attention with input and key0, and mask01 has attention with O 00 Attention occurs. Thus, the attention weights generated by the parts of the second attention weight sequence that are not related to the characters to be recognized are shielded, thereby improving the accuracy of the attention weight sequence.

[0169] S1240. Obtain a character feature vector of the target question information at the second recognition moment based on the attention feature vector.

[0170] S1242. Arrange the characters corresponding to the character feature vectors of the target question information obtained at each recognition moment in chronological order as answer information corresponding to the target question information.

[0171] The content information identification method provided by this application can reduce the decoding time of transformer-based information extraction by 90% or more. It has no requirements for text length or key length, making it widely applicable to entity extraction scenarios, such as structured OCR (for documents, receipts, and invoices), as well as machine question answering and other scenarios. A transformer-based parallel decoding solution is provided, enabling parallel decoding of all keys. The optimized model's decoding accuracy is identical to that of a single-key decoding approach, eliminating the need to adjust the model training process. This embodiment proposes a transformer-based attention mask structure specifically for content extraction scenarios. Combined with input permutation and output rearrangement, it enables parallel decoding of all keys in content extraction scenarios, improving transformer-based decoding performance by more than 10 times. While the original decoding method takes approximately 2 seconds or more to process a single text pass, the content information identification method provided by this application can reduce decoding time to less than 200ms (milliseconds) while maintaining the same decoding effect. This reduces user latency, improves the user experience, and reduces hardware costs, saving over 90% of hardware costs.

[0172] It should be understood that although Figure 2-Figure 12 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2-Figure 12 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The order of execution of these steps or stages is not necessarily one by one, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0173] In some embodiments, as Figure 13 As shown, a content information recognition device is provided. The device can be a software module or a hardware module, or a combination of the two to form a part of a computer device. The device specifically includes: a task description information acquisition module 1302, a current recognition feature vector sequence acquisition module 1304, a current attention weight sequence acquisition module 1306, an attention feature vector acquisition module 1308, and an information recognition result acquisition module 1310, wherein:

[0174] The task description information acquisition module 1302 is used to acquire target content to be identified and acquire task description information corresponding to multiple information identification tasks corresponding to the target content;

[0175] The current recognition feature vector sequence obtaining module 1304 is configured to obtain a content feature vector sequence obtained by extracting features from the target content, obtain an information feature vector sequence obtained by extracting features from each task description information, and arrange the content feature vector sequence and each information feature vector sequence according to an arrangement strategy to obtain a current recognition feature vector sequence corresponding to the current moment.

[0176] The current attention weight sequence obtaining module 1306 is used to obtain the current attention weight sequence corresponding to each task description information at the current moment based on the current recognition feature vector sequence;

[0177] An attention feature vector obtaining module 1308 is configured to, for each task description information, mask the current attention weight outside the target position in the current attention weight sequence, perform attention processing based on the current attention weight corresponding to the target position and the current recognition feature vector sequence, and obtain an attention feature vector; the target position includes: the feature vector arrangement position of the corresponding task description information in the current recognition feature vector sequence, and the feature vector arrangement position of the target content in the current recognition feature vector sequence;

[0178] The information recognition result obtaining module 1310 is used to obtain the current recognized characters of the task description information at the current moment based on the attention feature vector, and obtain the information recognition result corresponding to the task description information based on the recognized characters of the task description information at each moment.

[0179] In some embodiments, the device also includes a first target position obtaining module, which includes: a sorting obtaining unit, used to obtain a first sorting corresponding to the content feature vector sequence in the arrangement strategy, and a second sorting corresponding to the information feature vector sequence; a first arrangement position obtaining unit, used to obtain the feature vector arrangement position corresponding to the target content based on the first sorting and the number of feature vectors corresponding to the content feature vector sequence; a second arrangement position obtaining unit, used to obtain the feature vector arrangement position of the task description information based on the second sorting and the number of feature vectors corresponding to the information feature vector sequence.

[0180] In some embodiments, the second arrangement position obtaining unit is also used to determine, based on the second sorting, a forward sequence arranged before the information feature vector sequence in the current identification feature vector sequence; to count the number of feature vectors in each forward sequence, and to determine the starting point of the information feature vector sequence according to the statistical value obtained by counting the number; to determine the ending point of the information feature vector sequence based on the starting point and the number of feature vectors in the information feature vector sequence; and to use the vector position from the starting point to the ending point as the feature vector arrangement position of the task description information.

[0181] In some embodiments, the current recognition feature vector sequence obtaining module includes: a forward feature vector obtaining unit, which is used to obtain the forward feature vector corresponding to the forward recognition character obtained by each task description information at the previous moment; a current recognition feature vector sequence obtaining unit, which is used to arrange the content feature vector sequence, each information feature vector sequence and each forward feature vector according to the arrangement strategy to obtain the current recognition feature vector sequence corresponding to the current moment.

[0182] In some embodiments, the current identification feature vector sequence obtaining unit is also used to splice the various information feature vector sequences to obtain a first spliced sequence; according to the order of the various information feature vector sequences in the first spliced sequence, the forward feature vectors corresponding to the various task description information are spliced to obtain a second spliced sequence; the content feature vector sequence, the first spliced sequence and the second spliced sequence are spliced in sequence to generate the current identification feature vector sequence corresponding to the current moment.

[0183] In some embodiments, the device also includes a second target position obtaining module, which includes: a third arrangement position obtaining unit, which is used to obtain a vector position of a first length starting from the starting position in the current recognition feature vector sequence as the feature vector arrangement position corresponding to the target content; the first length is the number of feature vectors in the content feature vector sequence; a first position obtaining unit, which is used to obtain a forward information feature vector sequence in the first splicing sequence for the information feature vector sequence corresponding to the task description information; count the number of feature vectors in the forward information feature vector sequence and the number of feature vectors in the content feature vector sequence to obtain a first statistical number, and use the backward position of the position sorted as the first statistical number as the first position of the information feature vector sequence; a fourth arrangement position obtaining unit, which is used to obtain a vector position of a second length starting from the first position as the feature vector arrangement position of the task description information; the second length is the number of feature vectors in the information feature vector sequence.

[0184] In some embodiments, the current recognition feature vector sequence also includes the mask feature vectors corresponding to each task description information at the current moment; the attention feature vector obtaining module includes: a mask feature position obtaining unit, which is used to obtain the mask feature position of the mask feature vector corresponding to the task description information in the current recognition feature vector sequence; a masking sequence obtaining unit, which is used to obtain the matrix object corresponding to the mask feature position from the target weight masking matrix as the current weight masking sequence corresponding to the current attention weight sequence; the element value at the target position in the current weight masking sequence is the retention coefficient, and the element value outside the target position is the masking coefficient; the attention feature vector obtaining unit is used to perform attention processing based on the current weight masking sequence, the current attention weight sequence and the current recognition feature vector sequence to obtain the attention feature vector.

[0185] In some embodiments, the attention feature vector obtaining unit is also used to use the retention coefficient at the target position in the current weight masking sequence to retain the current attention weight at the target position in the current attention weight sequence, and to use the masking coefficient outside the target position in the current weight masking sequence to mask the current attention weight outside the target position in the current attention weight sequence; obtain the processed current attention weight sequence; perform attention processing on the current recognition feature vector sequence based on the processed current attention weight sequence to obtain the attention feature vector.

[0186] In some embodiments, the device also includes a first target weight masking matrix generation module, which includes: an initial weight masking matrix acquisition unit for acquiring an initial weight masking matrix; a mask arrangement position acquisition unit for determining the mask arrangement position of the mask feature vector corresponding to the task description information at each recognition moment in the arrangement strategy in the recognition feature vector sequence; a mask matrix object determination unit for determining the mask matrix object corresponding to the mask arrangement position in the initial weight masking matrix; an information position determination unit for determining the content position corresponding to the target content and the information position corresponding to the task description information in the mask matrix object; a target weight masking matrix acquisition unit for adjusting the matrix element values of the initial weight masking matrix based on the content position and the information position to obtain a target weight masking matrix, in which the matrix element values corresponding to the content position and the information position respectively are set as retention coefficients, and the matrix element values other than the content position and the information position are set as masking coefficients.

[0187] In some embodiments, the device also includes a second target weight shielding matrix generation module, and the second target weight shielding matrix generation module includes: a first matrix object obtaining unit, used to obtain an initial output shielding matrix, and based on the arrangement position of the information feature vector sequence in the recognition feature vector sequence in the arrangement strategy, determine the first matrix object corresponding to the information feature vector sequence from the initial output shielding matrix; an information matrix position determining unit, used to determine the information matrix position corresponding to the information feature vector sequence from the first matrix object based on the arrangement position of the information feature vector sequence in the recognition feature vector sequence in the arrangement strategy; a mask matrix position determining unit, used to determine the mask arrangement position of the mask feature vector of the information feature vector sequence at multiple recognition moments in the recognition feature vector sequence from the first matrix object based on the mask arrangement position of the mask feature vector of the information feature vector sequence in the arrangement strategy at multiple recognition moments in the recognition feature vector sequence. The mask matrix position corresponding to the mask eigenvector at each recognition moment is determined in the matrix object; a target output shielding matrix obtaining unit is used to set the matrix element values at the information matrix position and each mask matrix position in the first matrix object as the retention coefficient, and set the matrix element values other than the information matrix position and each mask matrix position in the first matrix object as the shielding coefficient to obtain the target output shielding matrix; an output matrix area determining unit is used to determine the output matrix area from the initial weight shielding matrix based on the mask arrangement position of the mask eigenvector of the information eigenvector sequence at multiple recognition moments in the recognition eigenvector sequence in the arrangement strategy; a target weight shielding matrix generating unit is used to set the matrix element values of the output matrix area using the target output shielding matrix to generate the target weight shielding matrix.

[0188] In some embodiments, the output matrix area determination unit is further used to determine the row vectors corresponding to each mask arrangement position from the initial weight masking matrix based on the mask arrangement position of the mask feature vector of each information feature vector sequence at each recognition moment in the arrangement strategy in the recognition feature vector sequence; determine the column vectors corresponding to each mask arrangement position and each information arrangement position from the initial weight masking matrix based on each mask arrangement position and the information arrangement position of each information feature vector sequence in the recognition feature vector sequence; obtain the overlapping area of each row vector and each column vector, and use the overlapping area as the output matrix area.

[0189] The specific definition of the content information identification device can be found in the definition of the content information identification method above and will not be repeated here. Each module in the above-mentioned content information identification device can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor of the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each of the above modules.

[0190] In some embodiments, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 14 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a content information recognition method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0191] In some embodiments, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 15 As shown. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data involved in the content information identification method. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a content information identification method is implemented.

[0192] Those skilled in the art will understand that Figure 14 and 15 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0193] In some embodiments, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0194] In some embodiments, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0195] In some embodiments, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of each of the above method embodiments.

[0196] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0197] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0198] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A content information identification method, characterized in that: The method comprises: Obtain target content to be identified, and obtain task description information corresponding to a plurality of information identification tasks corresponding to the target content; Obtaining a content feature vector sequence obtained by extracting features from the target content, and obtaining an information feature vector sequence obtained by extracting features from each of the task description information; Obtaining a forward feature vector corresponding to a forward recognized character recognized at a previous moment in each of the task description information; Arrange the content feature vector sequence, each of the information feature vector sequences, and each of the forward feature vectors according to an arrangement strategy to obtain a current recognition feature vector sequence corresponding to the current moment; Based on the current recognition feature vector sequence, a current attention weight sequence corresponding to each of the task description information at the current moment is obtained; For each task description information, shield the current attention weight outside the target position in the current attention weight sequence, perform attention processing based on the current attention weight corresponding to the target position and the current recognition feature vector sequence, and obtain an attention feature vector; the target position includes: the feature vector arrangement position corresponding to the task description information in the current recognition feature vector sequence, and the feature vector arrangement position corresponding to the target content in the current recognition feature vector sequence; The currently recognized characters of the task description information at the current moment are obtained based on the attention feature vector, and the information recognition results corresponding to the task description information are obtained based on the recognized characters of the task description information at each moment.

2. The method according to claim 1, characterized in that The steps of obtaining the target position include: Obtaining a first ranking corresponding to the content feature vector sequence and a second ranking corresponding to the information feature vector sequence in the arrangement strategy; Obtaining an arrangement position of the feature vector corresponding to the target content based on the first sorting and the number of feature vectors corresponding to the content feature vector sequence; The arrangement position of the feature vector of the task description information is obtained based on the second sorting and the number of feature vectors corresponding to the information feature vector sequence.

3. The method according to claim 2, characterized in that The obtaining of the feature vector arrangement position of the task description information based on the second sorting and the number of feature vectors corresponding to the information feature vector sequence includes: Determine, based on the second sorting, a forward sequence in the current recognition feature vector sequence that precedes the information feature vector sequence; Counting the number of feature vectors in each forward sequence, and determining the starting point of the information feature vector sequence according to the statistical value of the number obtained; Determining an end point of the information feature vector sequence based on the starting point and the number of feature vectors in the information feature vector sequence; The vector position from the starting point to the ending point is used as the feature vector arrangement position of the task description information.

4. The method according to claim 1, wherein Arranging the content feature vector sequence, each of the information feature vector sequences, and each of the forward feature vectors according to an arrangement strategy to obtain a current recognition feature vector sequence corresponding to the current moment includes: Splicing the information feature vector sequences to obtain a first spliced sequence; splicing the forward feature vectors corresponding to the task description information according to the order of the information feature vector sequences in the first splicing sequence to obtain a second splicing sequence; The content feature vector sequence, the first splicing sequence, and the second splicing sequence are sequentially spliced together to generate a current recognition feature vector sequence corresponding to the current moment.

5. The method according to claim 4, characterized in that The steps of obtaining the target position include: Obtaining a vector position of a first length from the starting position in the current recognition feature vector sequence as the feature vector arrangement position corresponding to the target content; the first length is the number of feature vectors in the content feature vector sequence; For the information feature vector sequence corresponding to the task description information, obtaining a forward information feature vector sequence in the first concatenated sequence; counting the number of feature vectors in the forward information feature vector sequence and the number of feature vectors in the content feature vector sequence to obtain a first statistical number, and using the backward position of the position ranked by the first statistical number as the first position of the information feature vector sequence; A vector position of a second length is obtained starting from the first position as the position for arranging the feature vector of the task description information; the second length is the number of feature vectors in the information feature vector sequence.

6. The method according to claim 1, characterized in that The current recognition feature vector sequence also includes mask feature vectors corresponding to each of the task description information at the current moment; the current attention weight outside the target position in the current attention weight sequence is masked, and attention processing is performed based on the current attention weight corresponding to the target position and the current recognition feature vector sequence to obtain an attention feature vector including: Obtaining a mask feature position of the mask feature vector corresponding to the task description information in the current recognition feature vector sequence; Obtain the matrix object corresponding to the mask feature position from the target weight masking matrix as the current weight masking sequence corresponding to the current attention weight sequence; the element value at the target position in the current weight masking sequence is the retention coefficient, and the element value outside the target position is the masking coefficient; Attention processing is performed based on the current weight masking sequence, the current attention weight sequence and the current recognition feature vector sequence to obtain an attention feature vector.

7. The method according to claim 6, characterized in that The performing attention processing based on the current weight masking sequence, the current attention weight sequence, and the current recognition feature vector sequence to obtain the attention feature vector includes: Using the retention coefficient at the target position in the current weight masking sequence, the current attention weight at the target position in the current attention weight sequence is retained, and using the masking coefficient outside the target position in the current weight masking sequence, the current attention weight outside the target position in the current attention weight sequence is masked; obtaining a processed current attention weight sequence; Attention processing is performed on the current recognition feature vector sequence based on the processed current attention weight sequence to obtain an attention feature vector.

8. The method according to claim 6, characterized in that The steps of generating the target weight shielding matrix include: Get the initial weight masking matrix; Determining the mask arrangement position of the mask feature vector corresponding to the task description information at each recognition moment in the arrangement strategy in the recognition feature vector sequence; Determine a mask matrix object corresponding to the mask arrangement position in the initial weight mask matrix; Determining a content position corresponding to the target content and an information position corresponding to the task description information in the mask matrix object; The matrix element values of the initial weight shielding matrix are adjusted based on the content position and the information position to obtain a target weight shielding matrix, in which the matrix element values corresponding to the content position and the information position are set as retention coefficients, and the matrix element values other than the content position and the information position are set as shielding coefficients.

9. The method according to claim 6, characterized in that The steps of generating the target weight shielding matrix include: Obtaining an initial output masking matrix, and determining a first matrix object corresponding to the information feature vector sequence from the initial output masking matrix based on the arrangement position of the information feature vector sequence in the identification feature vector sequence in the arrangement strategy; Based on the arrangement position of the information feature vector sequence in the identification feature vector sequence in the arrangement strategy, determining the information matrix position corresponding to the information feature vector sequence from the first matrix object; Based on the mask arrangement positions of the mask feature vectors of the information feature vector sequence at multiple recognition moments in the recognition feature vector sequence in the arrangement strategy, determining the mask matrix position corresponding to the mask feature vector at each recognition moment from the first matrix object; Setting the matrix element values at the information matrix position and each of the mask matrix positions in the first matrix object as retention coefficients, and setting the matrix element values other than the information matrix position and each of the mask matrix positions in the first matrix object as shielding coefficients, to obtain a target output shielding matrix; Determining an output matrix region from an initial weight masking matrix based on mask arrangement positions of mask feature vectors of the information feature vector sequence at multiple recognition moments in the recognition feature vector sequence in the arrangement strategy; The target output shielding matrix is used to set matrix element values in the output matrix region to generate a target weight shielding matrix.

10. The method according to claim 9, characterized in that The step of determining the output matrix region from the initial weight mask matrix based on the mask arrangement positions of the mask feature vectors of the information feature vector sequence at multiple recognition moments in the recognition feature vector sequence in the arrangement strategy comprises: Based on the mask arrangement position of the mask feature vector of each of the information feature vector sequences at each recognition moment in the recognition feature vector sequence in the arrangement strategy, determining the row vector corresponding to each of the mask arrangement positions from the initial weight masking matrix; Based on each of the mask arrangement positions and the information arrangement position of each of the information feature vector sequences in the identification feature vector sequence, determining from the initial weight masking matrix the column vectors corresponding to each of the mask arrangement positions and each of the information arrangement positions; Obtain overlapping regions between each of the row vectors and each of the column vectors, and use the overlapping regions as output matrix regions.

11. A content information identification device, characterized in that: The device comprises: A task description information acquisition module is used to acquire target content to be identified and to acquire task description information corresponding to a plurality of information identification tasks corresponding to the target content; A feature vector acquisition module is used to obtain a content feature vector sequence obtained by extracting features from the target content, obtain an information feature vector sequence obtained by extracting features from each of the task description information, and obtain a forward feature vector corresponding to a forward recognized character obtained at a previous moment in each of the task description information; a current recognition feature vector sequence obtaining module, configured to arrange the content feature vector sequence, each of the information feature vector sequences, and each of the forward feature vectors according to an arrangement strategy to obtain a current recognition feature vector sequence corresponding to the current moment; A current attention weight sequence obtaining module is used to obtain the current attention weight sequence corresponding to each of the task description information at the current moment based on the current recognition feature vector sequence; an attention feature vector obtaining module, configured to, for each of the task description information, mask the current attention weight outside the target position in the current attention weight sequence, perform attention processing based on the current attention weight corresponding to the target position and the current recognition feature vector sequence, and obtain an attention feature vector; the target position includes: the feature vector arrangement position corresponding to the task description information in the current recognition feature vector sequence, and the feature vector arrangement position corresponding to the target content in the current recognition feature vector sequence; The information recognition result obtaining module is used to obtain the current recognition character of the task description information at the current moment based on the attention feature vector, and obtain the information recognition result corresponding to the task description information based on the recognition character of the task description information at each moment.

12. The device according to claim 11, characterized in that The device further includes a target location determination module, which includes: a ranking obtaining unit, configured to obtain a first ranking corresponding to the content feature vector sequence and a second ranking corresponding to the information feature vector sequence in the arrangement strategy; a target content position determining unit, configured to obtain a feature vector arrangement position corresponding to the target content based on the first sorting and the number of feature vectors corresponding to the content feature vector sequence; The task description information position determining unit is configured to obtain the arrangement position of the feature vector of the task description information based on the second sorting and the number of feature vectors corresponding to the information feature vector sequence.

13. The device according to claim 12, characterized in that The task description information position determination unit is also used to determine, based on the second sorting, a forward sequence in the current identification feature vector sequence that is arranged before the information feature vector sequence; count the number of feature vectors in each of the forward sequences, and determine the starting point of the information feature vector sequence based on the statistical value of the counted number; determine the ending point of the information feature vector sequence based on the starting point and the number of feature vectors in the information feature vector sequence; and use the vector position from the starting point to the ending point as the feature vector arrangement position of the task description information.

14. The device according to claim 11, characterized in that The module for obtaining the current recognition feature vector sequence includes: an information feature vector splicing unit, configured to splice each of the information feature vector sequences to obtain a first spliced sequence; a forward feature vector concatenation unit, configured to concatenate the forward feature vectors corresponding to the task description information according to the order of the information feature vector sequences in the first concatenation sequence, to obtain a second concatenation sequence; A sequence splicing unit is used to sequentially splice the content feature vector sequence, the first splicing sequence, and the second splicing sequence to generate a current recognition feature vector sequence corresponding to a current moment.

15. The device according to claim 14, characterized in that The target location determination module also includes: a target content position determination subunit, configured to obtain a vector position of a first length from a starting position in the current recognition feature vector sequence as a feature vector arrangement position corresponding to the target content; the first length being the number of feature vectors in the content feature vector sequence; an information feature vector first position determining unit, configured to obtain, for the information feature vector sequence corresponding to the task description information, a forward information feature vector sequence in the first concatenated sequence; count the number of feature vectors in the forward information feature vector sequence and the number of feature vectors in the content feature vector sequence to obtain a first statistical number, and use the backward position of the position ranked by the first statistical number as the first position of the information feature vector sequence; The task description information position determination subunit is used to obtain a vector position of a second length starting from the first position as the feature vector arrangement position of the task description information; the second length is the number of feature vectors in the information feature vector sequence.

16. The device according to claim 11, characterized in that The current recognition feature vector sequence also includes the mask feature vectors corresponding to each of the task description information at the current moment; the attention feature vector obtaining module includes: a mask feature position acquisition unit, configured to acquire the mask feature position of the mask feature vector corresponding to the task description information in the current recognition feature vector sequence; a current weight masking sequence acquisition unit, configured to acquire a matrix object corresponding to the mask feature position from a target weight masking matrix as a current weight masking sequence corresponding to the current attention weight sequence; the element value at the target position in the current weight masking sequence is a retention coefficient, and the element value outside the target position is a masking coefficient; An attention processing unit is used to perform attention processing based on the current weight masking sequence, the current attention weight sequence and the current recognition feature vector sequence to obtain an attention feature vector.

17. The device according to claim 16, characterized in that The attention processing unit is further configured to perform a retention process on the current attention weight at the target position in the current attention weight sequence using the retention coefficient at the target position in the current weight masking sequence, and to perform a masking process on the current attention weight outside the target position in the current attention weight sequence using the masking coefficient outside the target position in the current weight masking sequence; Obtaining a processed current attention weight sequence; performing attention processing on the current recognition feature vector sequence based on the processed current attention weight sequence to obtain an attention feature vector.

18. The device according to claim 16, characterized in that The device further includes a first target weight shielding matrix generating module, wherein the first target weight shielding matrix generating module includes: An initial weight shielding matrix obtaining unit, used for obtaining an initial weight shielding matrix; a mask arrangement position determination unit, configured to determine the mask arrangement position of the mask feature vector corresponding to the task description information at each recognition moment in the arrangement strategy in the recognition feature vector sequence; a mask matrix object determining unit, configured to determine a mask matrix object corresponding to the mask arrangement position in the initial weight mask matrix; a position determining unit, configured to determine a content position corresponding to the target content and an information position corresponding to the task description information in the mask matrix object; A target weight shielding matrix adjustment unit is used to adjust the matrix element values of the initial weight shielding matrix based on the content position and the information position to obtain a target weight shielding matrix, in which the matrix element values corresponding to the content position and the information position are set as retention coefficients, and the matrix element values other than the content position and the information position are set as shielding coefficients.

19. The device according to claim 16, characterized in that The device further includes a second target weight shielding matrix generating module, wherein the second target weight shielding matrix generating module includes: An initial output shielding matrix obtaining unit, used for obtaining an initial output shielding matrix; a first matrix object determining unit, configured to determine, from the initial output masking matrix, a first matrix object corresponding to the information feature vector sequence based on an arrangement position of the information feature vector sequence in the identification feature vector sequence in the arrangement strategy; an information matrix position determining unit, configured to determine, from the first matrix object, an information matrix position corresponding to the information feature vector sequence based on the arrangement position of the information feature vector sequence in the identification feature vector sequence in the arrangement strategy; a mask matrix position determining unit, configured to determine, from the first matrix object, a mask matrix position corresponding to the mask feature vector at each recognition moment based on the mask arrangement positions of the mask feature vectors of the information feature vector sequence at multiple recognition moments in the recognition feature vector sequence in the arrangement strategy; a target output masking matrix generating unit, configured to set matrix element values at the information matrix position and each of the mask matrix positions in the first matrix object as retention coefficients, and set matrix element values other than the information matrix position and each of the mask matrix positions in the first matrix object as masking coefficients, to obtain a target output masking matrix; an output matrix region determining unit, configured to determine an output matrix region from an initial weight masking matrix based on mask arrangement positions of mask feature vectors of the information feature vector sequence at multiple recognition moments in the recognition feature vector sequence in the arrangement strategy; The target weight shielding matrix generating unit is used to set the matrix element values of the output matrix area using the target output shielding matrix to generate a target weight shielding matrix.

20. The device according to claim 19, characterized in that The output matrix area determination unit is also used to determine the row vectors corresponding to each mask arrangement position from the initial weight masking matrix based on the mask arrangement position of the mask feature vector of each information feature vector sequence at each recognition moment in the arrangement strategy in the recognition feature vector sequence; determine the column vectors corresponding to each mask arrangement position and each information arrangement position from the initial weight masking matrix based on each mask arrangement position and the information arrangement position of each information feature vector sequence in the recognition feature vector sequence; obtain the overlapping area between each row vector and each column vector, and use the overlapping area as the output matrix area.

21. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 10 is implemented.

22. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

23. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images