Method, device, readable medium and electronic device for determining text regional attributes

Abstract

This application belongs to the field of computer technology, specifically relating to a method and apparatus for determining the regional attributes of text. The method includes: encoding the text to be identified to obtain a text encoding vector; encoding the media that published the text to be identified to obtain a media encoding vector; encoding the channel tag of the channel that published the text to be identified to obtain a channel encoding vector; then fusing the text encoding vector, media encoding vector, and channel encoding vector to obtain a fused encoding vector; and finally classifying the text to be identified based on the fused encoding vector to determine whether the text to be identified has regional attributes. This application can comprehensively and accurately identify whether the text to be identified has regional attributes through machine learning and the above steps, thereby improving the accuracy of determining the regional attributes of text.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and specifically relates to a method for determining the geographical attributes of text, a device for determining the geographical attributes of text, a computer-readable medium, and an electronic device. Background Technology

[0002] In intelligent recommendation systems such as personalized news feeds, many articles possess certain regional attributes. These regional attributes typically mean that these articles are only suitable for readers within a specific geographical area, or are even more suitable for readers within a specific geographical area.

[0003] In related technologies, some news feed articles, due to their short length or low frequency of regional terms within a single article, cannot be accurately determined to have regional attributes. If an article merely mentions a place but does not provide information about that place in its entirety, and the algorithm recommends this article to users in that region, it will result in a poor user experience.

[0004] Therefore, improving the accuracy of determining the regional attributes of text is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] The purpose of this application is to provide a method, device, computer-readable medium, and electronic device for determining the geographical attributes of text, which at least partially solves the technical problem of how to improve the accuracy of determining the geographical attributes of text in related technologies.

[0006] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0007] According to one aspect of the embodiments of this application, a method for determining the geographic attribute of text is provided. The method includes:

[0008] Obtain the text to be recognized and encode the text to be recognized to obtain a text encoding vector;

[0009] Obtain the media that published the text to be identified, determine the media text of the text to be identified based on the media name of the media that published the text to be identified, and encode the media text to obtain a media encoding vector, wherein the media is the account entity that published the text on the information platform;

[0010] The channel that publishes the text to be identified and the channel tag of the channel are obtained. The channel tag of the channel is encoded to obtain the channel encoding vector of the channel. The channel is a publishing area on the information platform for publishing texts with similar preset features. The channel encoding vector, the media encoding vector and the text encoding vector are encoding vectors in the same vector space. The similarity between any two encoding vectors in the vector space is used to represent the degree of association between the encoded objects of the two encoding vectors.

[0011] The text encoding vector, the media encoding vector, and the channel encoding vector are fused together to obtain a fused encoding vector;

[0012] The text to be identified is classified according to the fused encoding vector to determine whether the text to be identified is text with regional attributes.

[0013] According to one aspect of the embodiments of this application, a text geographic attribute determination device is provided. The device includes:

[0014] The text encoding module is configured to acquire the text to be recognized and encode the text to be recognized to obtain a text encoding vector;

[0015] The media encoding module is configured to acquire the media that publishes the text to be identified, determine the media text of the text to be identified based on the media name of the media that publishes the text to be identified, and encode the media text to obtain a media encoding vector, wherein the media is the account entity that publishes the text on the information platform;

[0016] The channel encoding module is configured to acquire the channel on which the text to be identified is published and the channel tag of the channel, encode the channel tag of the channel to obtain the channel encoding vector of the channel, wherein the channel is a publishing area on the information platform for publishing text with similar preset features, wherein the channel encoding vector, the media encoding vector and the text encoding vector are encoding vectors in the same vector space, and the similarity between any two encoding vectors in the vector space is used to represent the degree of association between the encoded objects of the two encoding vectors;

[0017] The encoding fusion module is configured to perform encoding fusion on the text encoding vector, the media encoding vector, and the channel encoding vector to obtain a fused encoding vector;

[0018] The regional attribute determination module is configured to classify the text to be identified based on the fused encoding vector to determine whether the text to be identified is text with regional attributes.

[0019] In some embodiments of this application, based on the above technical solutions, the media encoding module includes:

[0020] The media information acquisition unit is configured to acquire media information of the media, including the media name, media introduction, and historical published text of the media;

[0021] The regional attribute media determination unit is configured to determine whether the media is a media with regional attributes based on the regional terms appearing in the media information of the media.

[0022] The media text determination unit is configured to, when the media is a media with regional attributes, obtain the regional description information of the media, and determine the media text based on the media name and the regional description information, wherein the regional description information is information used to describe the regional location of the media;

[0023] The second unit for determining media text is configured to determine the media name as the media text when the media is media without a regional attribute.

[0024] In some embodiments of this application, based on the above technical solutions, the regional attribute media determination unit includes:

[0025] The regional term statistics subunit is configured to count the frequency of regional terms in the historical published texts of the media.

[0026] The first regional term acquisition subunit is configured to use regional terms that appear more frequently than a preset frequency in the historical published text as first regional terms.

[0027] The second regional term acquisition subunit is configured to perform regional term recognition on the introduction of the media to obtain the second regional term;

[0028] The third regional term acquisition subunit is configured to perform regional term recognition on the name of the media to obtain the third regional term;

[0029] The first determination subunit is configured to determine the media as having regional attributes when there are at least two identical regional terms among the first regional term, the second regional term, and the third regional term.

[0030] The second determination subunit is configured to determine the media as media without regional attributes when there are no two identical regional terms among the first regional term, the second regional term, and the third regional term.

[0031] In some embodiments of this application, based on the above technical solutions, the media text determination first unit includes:

[0032] The text template acquisition subunit is configured to acquire a preset regional information enhanced text template, which is a text template used to describe the regional location of the media;

[0033] The associated regional term acquisition subunit is configured to take the same regional term among the first regional term, the second regional term, and the third regional term of the media as associated regional terms that are related to the media.

[0034] The regional description information acquisition subunit is configured to fill the associated regional terms of the media into the regional information enhancement text template to obtain the regional description information;

[0035] The media text acquisition subunit is configured to combine the media name and geographic description information of the media to obtain the media text.

[0036] In some embodiments of this application, based on the above technical solutions, the text encoding module includes:

[0037] The region word recognition unit is configured to perform region word recognition on the text to be recognized, and obtain one or more region words in the text to be recognized;

[0038] The regional word vector acquisition unit is configured to query a preset word list based on the regional words in the text to be identified, and obtain the regional word vectors corresponding to each of the regional words respectively;

[0039] The target word vector acquisition unit is configured to perform vector fusion on one or more of the regional word vectors to obtain the target word vector;

[0040] The semantic encoding vector acquisition unit is configured to perform semantic encoding on the text to be identified to obtain a semantic encoding vector.

[0041] The text encoding vector acquisition unit is configured to concatenate the target word vector and the semantic encoding vector into a text encoding vector.

[0042] In some embodiments of this application, based on the above technical solutions, the semantic encoding vector acquisition unit includes:

[0043] The first semantic encoding subunit is configured to perform semantic encoding on the complete text to be identified when the number of characters in the text to be identified is less than or equal to a preset number of characters, thereby obtaining a semantic encoding vector.

[0044] The second semantic encoding subunit is configured to perform semantic encoding on the merged text consisting of the title of the text to be recognized, the pre-preset paragraph text of the text to be recognized, and the post-preset paragraph text of the text to be recognized when the number of characters in the text to be recognized is greater than the preset number of characters, to obtain a semantic encoding vector. The number of characters in the merged text is less than or equal to the preset number of characters. The pre-preset paragraph text is a first preset number of paragraphs located at the beginning of the text to be recognized, and the post-preset paragraph text is a second preset number of paragraphs located at the end of the text to be recognized.

[0045] In some embodiments of this application, based on the above technical solutions, the text geographic attribute determination device further includes:

[0046] The training dataset acquisition unit is configured to acquire a training dataset, which includes training text, positive media examples of the training text, and negative media examples of the training text, wherein the positive media examples are the media that published the training text, and the negative media examples are other media besides the positive media examples in the training set.

[0047] The encoder acquisition unit is configured to acquire a first encoder and a second encoder; the first encoder is used to encode the text to be recognized and the training text respectively to obtain corresponding text encoding vectors; the second encoder is used to encode the media text to obtain a media encoding vector;

[0048] The first similarity calculation unit is configured to calculate a first similarity between the text encoding vector of the training text and the media encoding vector of the media positive example;

[0049] The second similarity calculation unit is configured to calculate the second similarity between the text encoding vector of the training text and the media encoding vector of each of the media negative examples.

[0050] The first learning loss calculation unit is configured to calculate the first learning loss based on the difference between the first similarity and a preset positive example similarity value, and the difference between each of the second similarities and a preset negative example similarity value.

[0051] The encoder training unit is configured to train the training parameters of the first encoder and the training parameters of the second encoder using the first learning loss.

[0052] In some embodiments of this application, based on the above technical solutions, the training dataset acquisition unit includes:

[0053] The training text acquisition subunit is configured to acquire training texts published by the first media on the first channel of the information platform;

[0054] The media positive example determination subunit is configured to determine the first media that publishes the training text as a media positive example of the training text;

[0055] The first sampling subunit is configured to sample a first preset number of texts to be identified from multiple texts to be identified that are published on any channel of the information platform, other than the training texts.

[0056] The second sampling subunit is configured to sample a second preset number of texts to be identified from multiple texts to be identified, other than the training texts, published on the first channel; the second preset number is less than the first preset number.

[0057] The media negative example determination subunit is configured to use media that are different from the first media in the media of the sampled text to be identified as media negative examples of the training text.

[0058] In some embodiments of this application, based on the above technical solutions, the training dataset further includes positive channel examples and negative channel examples of the training text, wherein the positive channel examples are the channels that publish the training text, and the negative channel examples are other channels besides the channels that publish the training text; the text geographic attribute determination device further includes:

[0059] The third encoder acquisition unit is configured to acquire a third encoder; the third encoder is used to encode the channel tag of the channel to obtain the channel encoding vector of the channel;

[0060] The third similarity calculation unit is configured to calculate the third similarity between the text encoding vector of the training text and the channel encoding vector of the channel positive example;

[0061] The fourth similarity calculation unit is configured to calculate the fourth similarity between the text encoding vector of the training text and the channel encoding vector of each of the channel negative examples.

[0062] The second learning loss calculation unit is configured to calculate the second learning loss based on the difference between the third similarity and the preset positive example similarity value, and the difference between each of the fourth similarities and the preset negative example similarity value.

[0063] The encoder training second unit is configured to train the training parameters of the first encoder and the training parameters of the third encoder using the second learning loss.

[0064] According to one aspect of the embodiments of this application, a computer-readable medium is provided, on which a computer program is stored, which, when executed by a processor, implements the text regional attribute determination method as described in the above technical solutions.

[0065] According to one aspect of the embodiments of this application, an electronic device is provided, the electronic device comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the text regional attribute determination method as described above by executing the executable instructions.

[0066] According to one aspect of the embodiments of this application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions, the processor of a computer device executing the computer instructions, causing the computer device to perform the text regional attribute determination method as in any of the above technical solutions.

[0067] In the technical solution provided in this application embodiment, the text to be identified is encoded to obtain a text encoding vector, the media publishing the text to be identified is encoded to obtain a media encoding vector, and the channel tag of the channel publishing the text to be identified is encoded to obtain a channel encoding vector. Then, the text encoding vector, media encoding vector, and channel encoding vector are fused to obtain a fused encoding vector. Finally, the text to be identified is classified according to the fused encoding vector to determine whether the text to be identified has regional attributes. Therefore, by encoding and fusing the semantic information of the three domains—the text to be identified, the media publishing the text, and the channel publishing the text—and classifying the text, multi-domain information is comprehensively utilized for regional attribute determination. This allows for a more comprehensive and accurate identification of whether the text to be identified has regional attributes, improving the accuracy of regional attribute determination.

[0068] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0069] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0070] Figure 1 An exemplary device architecture block diagram illustrating the application of the present application's technical solutions is shown schematically.

[0071] Figure 2 The flowchart illustrating the steps of a text regional attribute determination method according to certain embodiments of this application is shown in the illustration.

[0072] Figure 3The flowchart illustrating the steps of encoding the text to be recognized to obtain a text encoding vector in a certain embodiment of this application is shown in the illustration.

[0073] Figure 4 The flowchart illustrating the steps of semantically encoding the text to be identified to obtain a semantic encoding vector in a certain embodiment of this application is shown in the illustration.

[0074] Figure 5 The flowchart illustrates the steps before obtaining the text to be identified and encoding the text to be identified to obtain the text encoding vector in a certain embodiment of this application.

[0075] Figure 6 A flowchart illustrating the steps of obtaining a training dataset in a certain embodiment of this application is shown schematically.

[0076] Figure 7 The flowchart illustrates the steps before obtaining the text to be identified and encoding the text to be identified to obtain the text encoding vector in a certain embodiment of this application.

[0077] Figure 8 This illustration schematically shows the heterogeneous and related information between the text published on the information platform, the media publishing the text, and the channel publishing the text, according to a certain embodiment of this application.

[0078] Figure 9 The schematic diagram illustrates the model structure of a machine learning model used for determining the regional attributes of text in a certain embodiment of this application.

[0079] Figure 10 The flowchart illustrating the steps of determining the media text to be identified based on the media name of the media that publishes the text to be identified is shown in a certain embodiment of this application.

[0080] Figure 11 The flowchart illustrating the steps of determining whether a media outlet has regional attributes based on the regional terms appearing in the media information of the media outlet in a certain embodiment of this application is schematically shown.

[0081] Figure 12 The flowchart illustrating the steps of obtaining the geographic description information of media and determining the media text based on the media name and geographic description information in a certain embodiment of this application is shown in the illustration.

[0082] Figure 13 The diagram schematically illustrates the structural block diagram of a text location attribute determination device provided in certain embodiments of this application.

[0083] Figure 14 A schematic block diagram of a computer system architecture for implementing an electronic device according to embodiments of the present application is shown. Detailed Implementation

[0084] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0085] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0086] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0087] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0088] Before providing a detailed description of the text location attribute determination method and text location attribute determination device provided in the embodiments of this application, a brief introduction to the artificial intelligence technology involved in some embodiments of this application will be given first.

[0089] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0090] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0091] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0092] The following describes in detail the text location attribute determination method and text location attribute determination device provided in this application, with reference to specific embodiments.

[0093] Figure 1 An exemplary device architecture block diagram illustrating the application of the present application's technical solutions is shown schematically.

[0094] like Figure 1 As shown, the device architecture 100 may include a terminal device 110, a network 120, and a server 130. The terminal device 110 may include various electronic devices such as smartphones, tablets, laptops, and desktop computers. The server 130 may be a standalone physical server, a server cluster or distributed device composed of multiple physical servers, or a cloud server providing cloud computing services. The network 120 may be a communication medium of various connection types capable of providing a communication link between the terminal device 110 and the server 130, such as a wired communication link or a wireless communication link.

[0095] Depending on the implementation requirements, the device architecture in this application embodiment can have any number of terminal devices, networks, and servers. For example, server 130 can be a server group composed of multiple server devices. In addition, the technical solutions provided in this application embodiment can be applied to terminal device 110, or to server 130, or can be implemented jointly by terminal device 110 and server 130. This application does not impose any special limitations on this.

[0096] For example, server 130 can execute the text regional attribute determination method provided in this application, encoding the text to be identified to obtain a text encoding vector, encoding the media that published the text to be identified to obtain a media encoding vector, encoding the channel tag of the channel that published the text to be identified to obtain a channel encoding vector, and then encoding the text encoding vector, media encoding vector, and channel encoding vector to obtain a fused encoding vector. The text to be identified is then classified according to the fused encoding vector to determine whether the text to be identified has regional attributes. Thus, by encoding, fusing, and classifying the text through three dimensions—the text to be identified, the media to be identified, and the channel that published the text—multi-dimensional information is comprehensively utilized for regional attribute determination, thereby more comprehensively and accurately identifying whether the text to be identified has regional attributes and improving the accuracy of regional attribute determination.

[0097] The text regional attribute determination method provided in this application will be described in detail below with reference to specific implementation methods.

[0098] Figure 2 The flowchart illustrating the steps of the text geographic attribute determination method in certain embodiments of this application is shown in the illustration. It is understood that in specific embodiments of this application, when user-related data is used, and when the above embodiments of this application are applied to specific products or technologies, relevant user permission or consent is required, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0099] like Figure 2 As shown, the text regional attribute determination method can mainly include the following steps S210 to S250.

[0100] S210. Obtain the text to be recognized and encode it to obtain a text encoding vector.

[0101] The text to be identified can be text information published on a news platform. Alternatively, the text to be identified can also be text information obtained by performing text recognition on a video published on a news platform; this application does not impose any special restrictions on this.

[0102] In some implementations, the text to be recognized can be shorter than a preset number of characters. Specifically, the text to be recognized can be shorter than 128 characters. Alternatively, the text to be recognized can be shorter than 256 characters.

[0103] It is understandable that when the text content is relatively short, the features that the classification model can learn are very limited, and the reliability of determining the regional attributes of the text based on the text content is low, resulting in a low accuracy rate for determining the regional attributes of the text.

[0104] The implementation method of this application encodes, integrates, and classifies texts based on three dimensions: the text to be identified, the media in which the text is to be identified, and the channel in which the text is published. It comprehensively utilizes multi-dimensional information to determine regional attributes, thereby enabling a more comprehensive and accurate identification of whether the text to be identified has regional attributes. Even for texts with a small number of characters, the regional attributes of the text can be accurately determined.

[0105] Figure 3 The flowchart illustrating the steps of encoding the text to be recognized to obtain a text encoding vector in a certain embodiment of this application is shown schematically. Figure 3 As shown, based on the above embodiments, step S210, which encodes the text to be recognized to obtain a text encoding vector, may further include the following steps S310 to S350.

[0106] S310. Perform regional word recognition on the text to be recognized to obtain one or more regional words in the text to be recognized;

[0107] S320. Query the preset word list based on the regional words in the text to be identified, and obtain the regional word vectors corresponding to each regional word;

[0108] S330. Perform vector fusion on one or more regional word vectors to obtain the target word vector;

[0109] S340. Perform semantic encoding on the text to be recognized to obtain a semantic encoding vector;

[0110] S350. Concatenate the target word vector and the semantic encoding vector into a text encoding vector.

[0111] The pre-defined vocabulary can store the initial mapping relationship between regional words and regional word vectors. During the training process of the encoding, the initial mapping relationship between regional words and regional word vectors can be continuously changed as training progresses. This allows the regional word vectors to effectively represent the features of regional words, reducing the learning loss in the final regional attribute determination result and improving the accuracy of the regional attribute determination of the text.

[0112] Vector fusion is performed on one or more regional word vectors to obtain the target word vector. Specifically, when multiple regional words are identified in the text to be recognized, and each corresponds to a regional word vector, the average of the multiple regional word vectors can be calculated to obtain the target word vector. For example, the average of regional word vectors A = (a1, a2, a3...an), B = (b1, b2, b3...bn), and C = (c1, c2, c3...cn) can be calculated to obtain the target word vector S = (s1, s2, s3...sn), where s1 = (a1+b1+c1) / 3, s1 = (a2+b2+c2) / 3, and s1 = (a3+b3+c3) / 3.

[0113] Semantic encoding of the text to be recognized can be performed using the BERT encoding model.

[0114] In some implementations, an n-dimensional target word vector and an m-dimensional semantic encoding vector can be concatenated to obtain an n+m-dimensional semantic encoding vector. In other implementations, an n+m-dimensional intermediate encoding vector can be concatenated with the n-dimensional target word vector and the m-dimensional semantic encoding vector, and then the dimensionality of the n+m-dimensional intermediate encoding vector can be reduced to obtain an x-dimensional semantic encoding vector. In some implementations, other feasible methods can also be used to concatenate the target word vector and the semantic encoding vector.

[0115] Figure 4 The flowchart illustrating the steps of semantically encoding the text to be recognized to obtain a semantic encoded vector in a certain embodiment of this application is shown in the illustration. Figure 4 As shown, based on the above embodiments, step S340, which performs semantic encoding on the text to be recognized to obtain a semantic encoding vector, may further include the following steps S410 and S420.

[0116] S410. When the number of characters in the text to be recognized is less than or equal to the preset number of characters, semantic encoding is performed on the complete text to be recognized to obtain a semantic encoding vector;

[0117] S420. When the number of characters in the text to be identified is greater than the preset number of characters, semantic encoding is performed on the merged text consisting of the title of the text to be identified, the pre-preset paragraph text of the text to be identified, and the post-preset paragraph text of the text to be identified, to obtain a semantic encoding vector. The number of characters in the merged text is less than or equal to the preset number of characters. The pre-preset paragraph text is the first preset number of paragraphs of text located at the beginning of the text to be identified, and the post-preset paragraph text is the second preset number of paragraphs of text located at the end of the text to be identified.

[0118] Therefore, when the number of characters in the text to be identified is less than or equal to the preset number of characters, semantic encoding is performed on the complete text to be identified to obtain a semantic encoding vector. This enables semantic encoding of the complete text to be identified when the number of characters is small, and can achieve complete extraction of the regional features of the text to be identified, resulting in an encoding vector with complete text features.

[0119] When the number of characters in the text to be recognized is greater than the preset number of characters, semantic encoding is performed on the merged text consisting of the title of the text to be recognized, the preset preceding paragraph of the text to be recognized, and the preset following paragraph of the text to be recognized. This can improve the efficiency of encoding the text to be recognized when the number of characters in the text is large, while ensuring that the regional features at the key parts of the text are fully encoded and extracted.

[0120] Figure 5 This schematically illustrates a flowchart of the steps preceding the acquisition of the text to be recognized and the encoding of the text to be recognized to obtain a text encoding vector in a certain embodiment of this application. For example... Figure 5 As shown, based on the above embodiments, before obtaining the text to be recognized in step S210 and encoding the text to be recognized to obtain the text encoding vector, the following steps S510 to S560 may be further included.

[0121] S510. Obtain the training dataset, which includes training text, positive media examples of the training text, and negative media examples of the training text. Positive media examples are the media that published the training text, and negative media examples are other media besides the positive media examples in the training set.

[0122] S520. Obtain the first encoder and the second encoder; the first encoder is used to encode the text to be recognized and the training text respectively to obtain the corresponding text encoding vectors; the second encoder is used to encode the media text to obtain the media encoding vector;

[0123] S530. Calculate the first similarity between the text encoding vector of the training text and the media encoding vector of the media positive example;

[0124] S540. Calculate the second similarity between the text encoding vector of the training text and the media encoding vector of each media negative example;

[0125] S550. Calculate the first learning loss based on the difference between the first similarity and the preset positive example similarity value, and the difference between each second similarity and the preset negative example similarity value;

[0126] S560. Train the training parameters of the first encoder and the training parameters of the second encoder using the first learning loss.

[0127] In a specific implementation, the preset positive example similarity value can be 1, and the preset negative example similarity value can be 0. Alternatively, the preset positive example similarity value can be 100%, and the preset negative example similarity value can be 0. Or, the preset positive example similarity value can be 100%, and the preset negative example similarity value can be 0.

[0128] In some implementations, the preset positive example similarity value can be 1, and the preset negative example similarity value can be 0. This allows the encoding similarity of the multi-domain information of the text domain, media domain, and channel domain in a training text to be fitted to 1, and the encoding similarity between the text domain in the training text and the multi-domain information of other media domains and channel domains in other training texts to be fitted to 0. This integrates the text encoding vector, media encoding vector, and channel encoding vector into a single encoding space. Through the calculation and training fitting of the similarity between the text encoding vector and the media encoding vector, and between the text encoding vector and the channel encoding vector, the encoded text encoding vector, media encoding vector, and channel encoding vector can all possess rich regional semantic information, which can greatly improve the accuracy of the regional attribute determination of the text in the embodiments of this application.

[0129] In some implementations, the first learning loss can be calculated using the cross-entropy loss function based on the difference between the first similarity and a preset positive example similarity value, and the difference between each second similarity and a preset negative example similarity value.

[0130] Figure 6 A flowchart illustrating the steps for obtaining a training dataset in a certain embodiment of this application is shown schematically. Figure 6 As shown, based on the above embodiments, step S510 of obtaining the training dataset may further include the following steps S610 to S650.

[0131] S610. Obtain the training text published by the first media on the first channel of the information platform;

[0132] S620. The first media in which the training text was published shall be identified as a positive media example of the training text;

[0133] S630. From multiple texts to be identified published on any channel of the information platform, other than the training text, sample a first preset number of texts to be identified;

[0134] S640. From multiple texts to be identified that are published on the first channel, other than the training text, sample a second preset number of texts to be identified; the second preset number is less than the first preset number;

[0135] S650. Use media that are different from the first media in the sampled text to be identified as media negative examples of the training text.

[0136] Therefore, it is possible to sample a first preset number of texts to be identified from any channel on the information platform, and to sample a second preset number of texts to be identified from multiple texts to be identified published on the first channel other than the training text. This allows for global sampling to ensure good sample comprehensiveness, while also increasing the sampling of texts within the same channel. This results in some media negative examples being related to the training text, thereby increasing the difficulty of distinguishing the similarity between the training text and the media negative examples. This ensures good stability of the training results and can still distinguish the training text and the media negative examples based on encoding similarity even when encountering highly interfering media negative examples.

[0137] Figure 7 This schematically illustrates a flowchart of the steps preceding the acquisition of the text to be recognized and the encoding of the text to be recognized to obtain a text encoding vector in a certain embodiment of this application. For example... Figure 7 As shown, based on the above embodiments, the training dataset also includes positive channel examples and negative channel examples of the training text. The positive channel examples are the channels that publish the training text, and the negative channel examples are other channels besides the channels that publish the training text. Before obtaining the text to be recognized in step S210 and encoding the text to be recognized to obtain the text encoding vector, the following steps S710 to S750 may be further included.

[0138] S710. Obtain the third encoder; the third encoder is used to encode the channel label of the channel to obtain the channel encoding vector of the channel;

[0139] S720. Calculate the third similarity between the text encoding vector of the training text and the channel encoding vector of the channel positive example;

[0140] S730. Calculate the fourth similarity between the text encoding vector of the training text and the channel encoding vector of each channel negative example;

[0141] S740. The second learning loss is calculated based on the difference between the third similarity and the preset positive example similarity value, and the difference between each fourth similarity and the preset negative example similarity value.

[0142] S750. The training parameters of the first encoder and the training parameters of the third encoder are trained using the second learning loss.

[0143] In a specific implementation, the preset positive example similarity value can be 1, and the preset negative example similarity value can be 0. Alternatively, the preset positive example similarity value can be 100%, and the preset negative example similarity value can be 0. Or, the preset positive example similarity value can be 100%, and the preset negative example similarity value can be 0.

[0144] In some implementations, the preset positive example similarity value can be 1, and the preset negative example similarity value can be 0. This allows the encoding similarity of the multi-domain information of the text domain, media domain, and channel domain in a training text to be fitted to 1, and the encoding similarity between the text domain in the training text and the multi-domain information of other media domains and channel domains in other training texts to be fitted to 0. This integrates the text encoding vector, media encoding vector, and channel encoding vector into a single encoding space. Through the calculation and training fitting of the similarity between the text encoding vector and the media encoding vector, and between the text encoding vector and the channel encoding vector, the encoded text encoding vector, media encoding vector, and channel encoding vector can all possess rich regional semantic information, which can greatly improve the accuracy of the regional attribute determination of the text in the embodiments of this application.

[0145] In some implementations, the second learning loss can be calculated using the cross-entropy loss function based on the difference between the third similarity and the preset positive example similarity value, and the difference between each fourth similarity and the preset negative example similarity value.

[0146] In a specific implementation, a similarity vector can be obtained by combining the difference between the third similarity and the preset positive example similarity value, and the differences between each fourth similarity and the preset negative example similarity value. Then, the cross-entropy loss between the similarity vector and the target similarity vector is calculated. The target similarity vector can be a one-hot vector where only the dimension corresponding to the positive example is 1, and the dimension corresponding to the other negative examples is 0.

[0147] It is understandable that the training data source is manual annotation, which is very costly, thus limiting the amount of training data. The present application, as described above, can sample and acquire text published on an information platform to obtain training text, media positive examples of the training text, media negative examples of the training text, channel positive examples of the training text, and channel negative examples of the training text, thereby obtaining a training dataset. This allows for the acquisition of a training dataset rich in information without manual data annotation. Therefore, the training dataset can be used to pre-train the training parameters of the first encoder, second encoder, and third encoder, thereby improving the encoding performance of the first encoder, second encoder, and third encoder, enhancing the representational power of the encoding, and ultimately improving the accuracy of subsequent predictions regarding whether the text to be identified has regional attributes using text encoding vectors, media encoding vectors, and channel encoding vectors.

[0148] Figure 8 This illustration schematically depicts the heterogeneous and related information between text in an information platform published in a certain embodiment of this application, the media publishing the text, and the channel publishing the text. For example... Figure 8The texts published on various channels and media outlets on the information platform naturally contain related and heterogeneous information, possessing rich semantic connections without the need for manual data annotation. For example, the media outlet "A City TV" published the text "Our city recently held XXXX event" on its social channel and "A City can be so beautiful!" on its tourism channel. The media outlet "B City Micro Life" published "It is rumored online that XXX phenomenon has recently occurred in B City" on its social channel, and the media outlet "C City Weather" published "City C XX Lake is shrouded in mist like a fairyland" on its tourism channel. This forms a structure like... Figure 8 The complex relationships shown include four sets of texts with both related and heterogeneous information. These four sets of texts possess rich semantic association information without the need for manual data annotation. Using multiple sets of texts with related and heterogeneous information as training datasets can enable pre-training of the training parameters for the first encoder, the second encoder, and the third encoder.

[0149] The above-described embodiments of this application obtain training text, media positive examples of training text, media negative examples of training text, channel positive examples of training text, and channel negative examples of training text by sampling and acquiring text published on the information platform, thereby obtaining a training dataset. This fully utilizes the naturally existing related and heterogeneous information in the text published on various channels and media on the information platform, which can improve the encoding representation capabilities of the first encoder, second encoder, and third encoder, and overcome the defect of low regional attribute prediction accuracy caused by the difficulty of manual annotation.

[0150] Figure 9 This diagram schematically illustrates the model structure of a machine learning model for determining text geographic attributes according to a certain embodiment of this application. The text geographic attribute determination method of this application can be implemented by a machine learning model for determining text geographic attributes. This machine learning model may include a first encoder, a second encoder, a third encoder, a feedforward neural network, and a classifier.

[0151] Specifically, the first encoder can encode the text to be recognized to obtain a text encoding vector T. d The first encoder can be a BERT encoding model. The second encoder can encode the media text to obtain the media encoding vector T. m The second encoder can be a preset encoding model w&d-cls (wide&deep-cls). The third encoder can encode the channel label of the channel to obtain the channel encoding vector T. chl The third encoder can be a mean coding model (Mean). A feedforward neural network (FFN) can encode the text vector T. d Media encoding vector T m and channel coding vector Tchl The encoding is fused to obtain the fused encoding vector T. final The classifier can be based on the fused encoding vector T. final The text to be identified is classified into texts with regional attributes and texts without regional attributes, thereby determining whether the text to be identified is a text with regional attributes.

[0152] The preset encoding model w&d-cls can execute the steps shown in steps S310 to S350 in the above embodiment, thereby encoding the text to be recognized to obtain a text encoding vector.

[0153] In specific implementations, the classifier can be a softmax or sigmoid classifier, or other classifiers, and this application does not impose any special restrictions on it.

[0154] In some implementations, a model training dataset can be obtained, which includes multiple labeled training texts. Texts with geographic attributes are labeled as 1, and those without geographic attributes are labeled as 0. This allows the machine learning model for determining text geographic attributes to be trained using the model training dataset. It is understood that the training parameters of the first encoder, second encoder, and third encoder in this machine learning model have been pre-trained as shown in the embodiments above, exhibiting good encoding and representation capabilities. Therefore, during the training phase using the model training dataset, only a small amount of labeled training text data is needed to complete the fine-tuning training of the machine learning model for determining text geographic attributes, reducing the required amount of labeled data and improving model training efficiency.

[0155] It is understandable that after the pre-training phase, which involves training the parameters of the first encoder, the second encoder, and the third encoder, the first learning loss and the second loss no longer decrease. At this point, in some implementations, sample media can be sampled, and the portion of the media encoded by the second encoder that has the highest similarity to the sample media can be obtained. This sample media and the portion of the media with the highest similarity to the sample media can then be displayed on an interactive interface. This provides intuitive feedback on the encoding performance of the second encoder to technical personnel, facilitating the evaluation of the model's encoding performance and enabling continuous improvement of the encoding scheme.

[0156] As shown in Table 1, the sample media "Micro City A" was obtained, and the media encoding vectors with the highest similarity to the sample media's media encoding vector, namely "Today's City A," "City A Release," and "Beautiful City A," were acquired. The sample media and the media with the highest similarity to the sample media are then displayed on the backend interactive interface as shown in Table 1. It can be understood that "Micro City A," "Today's City A," "City A Release," and "Beautiful City A" are all media related to "City A," and the similarity between the sample media "Micro City A" and "Today's City A," "City A Release," and "Beautiful City A" is higher than 0.9, approaching 1. Furthermore, in the vector space composed of encoding vectors obtained by the trained encoder, the similarity between two vectors represents the degree of association between the encoded objects of the two encoding vectors. Therefore, after encoding the media texts "Micro City A", "Today City A", "City A Release", and "Beautiful City A", the encoder can represent the strong correlation between "Micro City A", "Today City A", "City A Release", and "Beautiful City A" with a similarity score higher than 0.9. This intuitively shows that the trained encoder performs well.

[0157]

[0158] Table 1

[0159] In some implementations, sample channels can be sampled, and the media with the highest similarity to the sample channel obtained by the second encoder can be acquired. The sample channel and the media with the highest similarity to the sample channel can be displayed on the interactive interface to intuitively provide feedback on the encoding effects of the second encoder and the third encoder to the technicians, thereby facilitating the evaluation of the model's encoding effect and continuously improving the encoding scheme.

[0160] As shown in Table 2, the sample channel "Oral Health" was obtained, and the media corresponding to the media encoding vectors with the highest similarity to the channel encoding vector of this sample channel, namely "Y City XX Oral Clinic", "Z City Oral Clinic", and "K City XX Oral Clinic", were acquired. This sample channel and the media corresponding to it with the highest similarity were then displayed on the backend interactive interface. It can be understood that "Y City XX Oral Clinic", "Z City Oral Clinic", and "K City XX Oral Clinic" are all media related to "oral health", and the similarity between the sample channel "Oral Health" and "Y City XX Oral Clinic", "Z City Oral Clinic", and "K City XX Oral Clinic" are all higher than 0.6. Considering that the calculation is based on the similarity between the channel encoding vector and the media encoding vector, the similarity between the channel and the media cannot be too high. Furthermore, in the vector space composed of the encoding vectors obtained by the trained encoder, the similarity between two vectors is used to represent the degree of association between the encoded objects of the two encoding vectors. Therefore, after training, the third encoder encodes the channel label for the channel "Oral Health," and the second encoder, after training, encodes the media text for "Y City XX Dental Clinic," "Z City Dental Clinic," and "K City XX Dental Clinic." This demonstrates a strong correlation between the sample channel "Oral Health" and "Y City XX Dental Clinic," "Z City Dental Clinic," and "K City XX Dental Clinic" through a similarity score higher than 0.6. This clearly shows that the encoder trained above performs well.

[0161]

[0162] Table 2

[0163] In some implementations, after sampling the encoded object and obtaining the target encoded object corresponding to the encoded vector with the highest similarity to the encoded vector of the encoded object obtained by the encoder, the encoded object and the similarity to the target encoded object can be displayed on the development interface through a view, which makes it easy for developers to intuitively see the encoding effect of the encoder.

[0164] S220. Obtain the media that published the text to be identified, determine the media text of the text to be identified based on the media name of the media that published the text to be identified, and encode the media text to obtain the media encoding vector. The media is the account entity that published the text on the information platform.

[0165] Figure 10 This schematically illustrates a flowchart of the steps in a certain embodiment of this application to determine the media text to be identified based on the media name of the media that publishes the text to be identified. For example... Figure 10As shown, based on the above embodiments, step S220, which determines the media text to be identified based on the media name of the media that published the text to be identified, may further include the following steps S1010 to S1040.

[0166] S1010. Obtain media information, including the media's name, a brief introduction, and historical publications;

[0167] S1020. Determine whether a media outlet has regional attributes based on the regional terms appearing in its media information;

[0168] S1030. When the media is a media with regional attributes, obtain the regional description information of the media, and determine the media text based on the media name and the regional description information. The regional description information is information used to describe the regional location of the media.

[0169] S1040. When the media is not geographically specific, the media name shall be determined as the media text.

[0170] Therefore, it is possible to determine whether a media outlet has regional attributes by identifying regional terms appearing in the media information. For media outlets with regional attributes, the regional description information of the media can be obtained, and the media text can be determined based on the media name and regional description information, thereby enhancing the information of the media and obtaining the final media text.

[0171] Figure 11 This schematically illustrates a flowchart of the steps in a certain embodiment of this application to determine whether a media outlet has regional attributes based on the regional terms appearing in its media information. For example... Figure 11 As shown, based on the above embodiments, step S1020, which determines whether a media has regional attributes based on the regional terms appearing in the media information, may further include the following steps S1110 to S1160.

[0172] S1110. Analyze the frequency of regional terms in historical published texts of statistical media;

[0173] S1120. Regional terms that appear more frequently than a preset frequency in historical published texts will be used as the first regional terms;

[0174] S1130. Perform regional term identification on the media introduction to obtain the second regional term;

[0175] S1140. Perform regional term identification on the media name to obtain the third regional term;

[0176] S1150. When there are at least two identical regional terms among the first, second, and third regional terms, the media is determined to be media with regional attributes;

[0177] S1160. When there are no two identical regional terms among the first, second, and third regional terms, the media is determined to be media without regional attributes.

[0178] In a specific embodiment, the frequency of regional terms in the historical published texts of a media outlet can be statistically analyzed. If all historical published texts of the media outlet "A City TV" contain the regional term "A City" with a preset probability, then the first regional term "A City" for that media outlet is identified. A media outlet may correspond to zero or one or more of the first regional terms. Then, regional term identification is performed on the media outlet's concise text to obtain zero or one or more second regional terms. For example, performing regional term identification on the media outlet's concise text "A City TV mainly focuses on the social welfare, news, and information of the entire city of A" yields the second regional term "A City." Next, regional term identification is performed on the media outlet's name to obtain zero or one or more third regional terms. For example, performing regional term identification on the media outlet name "A City TV" yields the third regional term "A City." Since there are three identical regional terms "A City" among the first, second, and third regional terms, the media outlet "A City TV" is determined to be a media outlet with regional attributes. Therefore, by acquiring the first, second, and third regional terms of a media outlet, it is possible to accurately determine whether a media outlet has regional attributes.

[0179] Figure 12 This illustration schematically depicts a flowchart illustrating the steps of obtaining geographic description information of media and determining media text based on the media name and geographic description information in a certain embodiment of this application. Figure 12 As shown, based on the above embodiments, step S1030, which involves obtaining the regional description information of the media and determining the media text based on the media name and regional description information, may further include the following steps S1210 to S1230.

[0180] S1210. Obtain a preset regional information enhanced text template, which is a text template used to describe the regional location of the media;

[0181] S1220. The same regional terms among the first, second, and third regional terms of a media outlet shall be regarded as related regional terms that are associated with the media outlet;

[0182] S1230. Fill in the relevant regional terms of the media into the regional information enhancement text template to obtain regional description information;

[0183] S1240. Combine the media name and geographic description information to obtain the media text.

[0184] In a specific embodiment, as shown in Table 3 below, for media with a regional attribute, "City A TV", a preset regional information enhancement text template "() is a regional media of ()" can be obtained. Then, the associated regional term "City A" of media "City A TV" is filled into the regional information enhancement text template to obtain the regional description information "() is a regional media of City A." Then, the media name "City A TV" and the regional description information "() is a regional media of City A" are combined to obtain the media text "City A TV is a regional media of City A." For media without a regional attribute, "XX Ham Association", its media name "XX Ham Association" is determined as the media text.

[0185]

[0186] Table 3

[0187] S230. Obtain the channel where the text to be identified is published and the channel's channel tag. Encode the channel tag to obtain the channel's channel encoding vector. The channel is the publishing area on the information platform used to publish texts with similar preset features. The channel encoding vector, media encoding vector, and text encoding vector are encoding vectors in the same vector space. The similarity between any two encoding vectors in this vector space is used to represent the degree of association between the encoded objects of the two encoding vectors.

[0188] Specifically, a channel can have one or more channel tags. For example, the channel "Barbecue Food Recommendations" can have the channel tags "Food," "Barbecue," and "Food Recommendations." For example, the channel "20th Century History Knowledge Popularization" can have the channel tags "20th Century," "History Knowledge," and "Popularization."

[0189] In a specific implementation, the channel label of a channel is encoded in the following ways: when a channel has only one channel label, the channel label is encoded to obtain a channel encoding vector; when a channel has multiple channel labels, the multiple channel labels are encoded separately to obtain corresponding multiple channel label codes, and then the average of the encoding vectors of the multiple channel label codes is calculated to obtain the channel encoding vector.

[0190] S240. The text encoding vector, media encoding vector, and channel encoding vector are encoded and fused to obtain a fused encoding vector.

[0191] Therefore, the multi-domain information of the text domain, media domain, and channel domain in the text to be identified can be fused into a fused coding vector through the text coding vector, media coding vector, and channel coding vector, so that the fused coding vector has the multi-domain information of the text domain, media domain, and channel domain in the text to be identified.

[0192] It's understandable that in search scenarios, if a user's search keywords have a clear regional bias—for example, if a user searches for "swimming pool" and their location is in "City A"—then articles from "City A" should be displayed with increased weight. In some implementations of information recommendation, information from the user's location can be recommended, while articles from areas outside the user's location are avoided, thereby improving the accuracy of recommendations and meeting the user's reading needs.

[0193] To recommend relevant information based on a user's location, it's essential to first acquire information with regional attributes. Understandably, while some articles may include the keyword "City A," they may only mention City A without focusing on it. If such text is identified as having the "City A" regional attribute, the accuracy of regional information recommendations for users located in City A will be lower, resulting in a poorer user experience.

[0194] In this embodiment, the text to be identified is encoded to obtain a text encoding vector, the media publishing the text to be identified is encoded to obtain a media encoding vector, and the channel tag of the channel publishing the text to be identified is encoded to obtain a channel encoding vector. The text encoding vector, media encoding vector, and channel encoding vector are then fused to obtain a fused encoding vector. The text to be identified is then classified based on the fused encoding vector to determine whether it possesses regional attributes. Therefore, this method comprehensively utilizes multi-domain information from the text, media, and channel domains of the text to be identified, improving the accuracy of regional attribute determination compared to relying solely on the title and body of the text for regional attribute identification.

[0195] S250. Classify the text to be identified based on the fused coding vector to determine whether the text to be identified is text with regional attributes.

[0196] Therefore, by fusing encoding vectors, it is possible to accurately determine whether the text to be identified has regional attributes. This facilitates the delivery of regionally relevant information to users in the corresponding region, thereby improving the accuracy of information recommendations and enhancing the user experience. Specifically, text with regional attributes can include: local news, information related to local food, clothing, housing, transportation, events, conferences, tourism introductions, etc.

[0197] It should be noted that although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0198] The following describes an embodiment of the apparatus of this application, which can be used to execute the corresponding text regional attribute determination method in the above embodiments of this application.

[0199] Figure 13 The diagram schematically illustrates the structural block diagram of a text geographic attribute determination device provided in certain embodiments of this application. For example... Figure 13 As shown, the text geographic attribute determination device 1300 includes:

[0200] The text encoding module 1310 is configured to acquire the text to be recognized and encode the text to be recognized to obtain a text encoding vector;

[0201] The media encoding module 1320 is configured to acquire the media that publishes the text to be identified, determine the media text of the text to be identified based on the media name of the media that publishes the text to be identified, and encode the media text to obtain the media encoding vector. The media is the account entity that publishes the text on the information platform.

[0202] The channel encoding module 1330 is configured to acquire the channel where the text to be identified is published and the channel tag of the channel, encode the channel tag of the channel to obtain the channel encoding vector of the channel. The channel is the publishing area on the information platform for publishing text with similar preset features. The channel encoding vector, media encoding vector and text encoding vector are encoding vectors in the same vector space. The similarity between any two encoding vectors in the vector space is used to represent the degree of association between the encoded objects of the two encoding vectors.

[0203] The encoding fusion module 1340 is configured to fuse the text encoding vector, media encoding vector and channel encoding vector to obtain a fused encoding vector;

[0204] The regional attribute determination module 1350 is configured to classify the text to be identified based on the fused coding vector in order to determine whether the text to be identified is text with regional attributes.

[0205] In some embodiments of this application, based on the above embodiments, the media encoding module includes:

[0206] The media information acquisition unit is configured to acquire media information, including the media's name, a brief introduction, and historical published texts.

[0207] The regional attribute media determination unit is configured to determine whether a media has regional attributes based on the regional terms appearing in the media information of the media.

[0208] The first unit for determining media text is configured to, when the media is a media with regional attributes, obtain the regional description information of the media and determine the media text based on the media name and the regional description information, whereby the regional description information is used to describe the regional location of the media.

[0209] The second unit for determining media text is configured to determine the media name as the media text when the media is media that does not have a regional attribute.

[0210] In some embodiments of this application, based on the above embodiments, the regional attribute media determination unit includes:

[0211] The regional term statistics subunit is configured to count the frequency of regional terms in historically published texts of media outlets;

[0212] The first regional term acquisition subunit is configured to use regional terms that appear more frequently than a preset frequency in historical published texts as the first regional terms.

[0213] The second regional term acquisition subunit is configured to perform regional term recognition on the media's description to obtain the second regional term;

[0214] The third regional term acquisition subunit is configured to perform regional term recognition on the media name to obtain the third regional term;

[0215] The first determination subunit is configured to determine the media as having regional attributes when there are at least two identical regional terms among the first regional term, the second regional term, and the third regional term.

[0216] The second determination subunit is configured to determine the media as having no regional attributes when there are no two identical regional terms among the first, second, and third regional terms.

[0217] In some embodiments of this application, based on the above embodiments, the media text determination first unit includes:

[0218] The text template acquisition subunit is configured to acquire a preset geographic information enhanced text template, which is a text template used to describe the geographic location of the media.

[0219] The associated regional term acquisition subunit is configured to take the same regional term among the first, second, and third regional terms of the media as associated regional terms that are related to the media;

[0220] The regional description information acquisition subunit is configured to fill the media's associated regional terms into the regional information enhancement text template to obtain regional description information;

[0221] The media text acquisition subunit is configured to combine the media name and geographic description information to obtain the media text.

[0222] In some embodiments of this application, based on the above embodiments, the text encoding module includes:

[0223] The region word recognition unit is configured to perform region word recognition on the text to be recognized, and obtain one or more region words in the text to be recognized.

[0224] The regional word vector acquisition unit is configured to query a preset word list based on the regional words in the text to be identified, and obtain the regional word vectors corresponding to each regional word.

[0225] The target word vector acquisition unit is configured to perform vector fusion on one or more regional word vectors to obtain the target word vector;

[0226] The semantic encoding vector acquisition unit is configured to perform semantic encoding on the text to be recognized to obtain a semantic encoding vector.

[0227] The text encoding vector acquisition unit is configured to concatenate the target word vector and the semantic encoding vector into a text encoding vector.

[0228] In some embodiments of this application, based on the above embodiments, the semantic encoding vector acquisition unit includes:

[0229] The first semantic coding subunit is configured to perform semantic coding on the complete text to be recognized when the number of characters in the text to be recognized is less than or equal to the preset number of characters, and obtain a semantic coding vector.

[0230] The second semantic encoding subunit is configured to perform semantic encoding on the merged text consisting of the title of the text to be recognized, the pre-preset paragraph text of the text to be recognized, and the post-preset paragraph text of the text to be recognized when the number of characters in the text to be recognized is greater than the preset number of characters, to obtain a semantic encoding vector. The number of characters in the merged text is less than or equal to the preset number of characters. The pre-preset paragraph text is the first preset number of paragraphs of text located at the beginning of the text to be recognized, and the post-preset paragraph text is the second preset number of paragraphs of text located at the end of the text to be recognized.

[0231] In some embodiments of this application, based on the above embodiments, the text geographic attribute determination device further includes:

[0232] The training dataset acquisition unit is configured to acquire a training dataset, which includes training text, positive media examples of the training text, and negative media examples of the training text. The positive media examples are the media that published the training text, and the negative media examples are other media besides the positive media examples in the training set.

[0233] The encoder acquisition unit is configured to acquire a first encoder and a second encoder; the first encoder is used to encode the text to be recognized and the training text respectively to obtain the corresponding text encoding vectors; the second encoder is used to encode the media text to obtain the media encoding vector.

[0234] The first similarity calculation unit is configured to calculate the first similarity between the text encoding vector of the training text and the media encoding vector of the media positive example;

[0235] The second similarity calculation unit is configured to calculate the second similarity between the text encoding vector of the training text and the media encoding vector of each media negative example.

[0236] The first learning loss calculation unit is configured to calculate the first learning loss based on the difference between the first similarity and the preset positive example similarity value, and the difference between each second similarity and the preset negative example similarity value.

[0237] The encoder trains the first unit, which is configured to train the training parameters of the first encoder and the training parameters of the second encoder using a first learning loss.

[0238] In some embodiments of this application, based on the above embodiments, the training dataset acquisition unit includes:

[0239] The training text acquisition subunit is configured to acquire training texts published by the first media on the first channel of the information platform;

[0240] The media positive example determination subunit is configured to determine the first media that published the training text as the media positive example of the training text;

[0241] The first sampling subunit is configured to sample a first preset number of texts to be identified from multiple texts to be identified published on any channel of the information platform, other than the training texts.

[0242] The second sampling subunit is configured to sample a second preset number of texts to be identified from multiple texts to be identified other than the training texts published on the first channel; the second preset number is less than the first preset number.

[0243] The media negative example determination subunit is configured to use media that are different from the first media in the media of the sampled text to be identified as media negative examples of the training text.

[0244] In some embodiments of this application, based on the above embodiments, the training dataset further includes positive channel examples and negative channel examples of the training text, where positive channel examples are the channels that publish the training text, and negative channel examples are other channels besides the channels that publish the training text; the text geographic attribute determination device further includes:

[0245] The third encoder acquisition unit is configured to acquire the third encoder; the third encoder is used to encode the channel label of the channel to obtain the channel encoding vector of the channel.

[0246] The third similarity calculation unit is configured to calculate the third similarity between the text encoding vector of the training text and the channel encoding vector of the channel positive example;

[0247] The fourth similarity calculation unit is configured to calculate the fourth similarity between the text encoding vector of the training text and the channel encoding vector of each channel negative example.

[0248] The second learning loss calculation unit is configured to calculate the second learning loss based on the difference between the third similarity and the preset positive example similarity value, and the difference between each fourth similarity and the preset negative example similarity value.

[0249] The encoder trains the second unit, which is configured to train the training parameters of the first encoder and the training parameters of the third encoder using a second learning loss.

[0250] The specific details of the text regional attribute determination device provided in the various embodiments of this application have been described in detail in the corresponding related method embodiments, and will not be repeated here.

[0251] Figure 14 A schematic block diagram of a computer system architecture for implementing an electronic device according to embodiments of the present application is shown.

[0252] It should be noted that, Figure 14 The computer system 1400 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0253] like Figure 14As shown, the computer system 1400 includes a central processing unit (CPU) 1401, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 1402 or programs loaded from storage section 1408 into random access memory (RAM). The RAM 1403 also stores various programs and data required for system operation. The CPU 1401, ROM 1402, and RAM 1403 are interconnected via a bus 1404. An input / output interface 1405 (I / O interface) is also connected to the bus 1404.

[0254] The following components are connected to the input / output interface 1405: an input section 1406 including a keyboard, mouse, etc.; an output section 1407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1408 including a hard disk, etc.; and a communication section 1409 including a network interface card such as a local area network card, modem, etc. The communication section 1409 performs communication processing via a network such as the Internet. A drive 1410 is also connected to the input / output interface 1405 as needed. A removable medium 1411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1410 as needed so that computer programs read from it can be installed into the storage section 1408 as needed.

[0255] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1409, and / or installed from removable medium 1411. When the computer program is executed by central processing unit 1401, it performs various functions defined in the system of this application.

[0256] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0257] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0258] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0259] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0260] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0261] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for determining the regional attribute of text, characterized in that, The method includes: Obtain the text to be recognized and encode the text to be recognized to obtain a text encoding vector; Obtain the media that published the text to be identified, determine the media text of the text to be identified based on the media name of the media that published the text to be identified, and encode the media text to obtain a media encoding vector, wherein the media is the account entity that published the text on the information platform; The channel that publishes the text to be identified and the channel tag of the channel are obtained. The channel tag of the channel is encoded to obtain the channel encoding vector of the channel. The channel is a publishing area on the information platform for publishing texts with similar preset features. The channel encoding vector, the media encoding vector and the text encoding vector are encoding vectors in the same vector space. The similarity between any two encoding vectors in the vector space is used to represent the degree of association between the encoded objects of the two encoding vectors. The text encoding vector, the media encoding vector, and the channel encoding vector are fused together to obtain a fused encoding vector; The text to be identified is classified according to the fused encoding vector to determine whether the text to be identified is text with regional attributes.

2. The method according to claim 1, characterized in that, The media text of the text to be identified is determined based on the media name of the media in which the text to be identified is published, including: Obtain the media information of the media, which includes the media name, media introduction, and historical published texts of the media; Determine whether the media is a media with regional attributes based on the regional terms appearing in the media information of the media. When the media is a media with regional attributes, the regional description information of the media is obtained, and the media text is determined according to the media name and the regional description information. The regional description information is information used to describe the regional location of the media. When the media is media without regional attributes, the media name is determined as the media text.

3. The method according to claim 2, characterized in that, Determining whether a media outlet has regional attributes based on the regional terms appearing in its media information includes: Analyze the frequency of regional terms in historical published texts of the aforementioned media. The regional words that appear more frequently than a preset frequency in the historical published text will be used as the first regional words. The description of the media is analyzed for regional terms to obtain a second regional term; The name of the media is subjected to regional term identification to obtain a third regional term; When at least two of the first, second, and third regional terms are the same, the media is determined to be media with regional attributes. If there are no two identical regional terms among the first, second, and third regional terms, the media is determined to be media without regional attributes.

4. The method according to claim 3, characterized in that, Obtaining the geographic description information of the media, and determining the media text based on the media name and geographic description information, including: Obtain a preset text template with enhanced geographic information, wherein the text template with enhanced geographic information is a text template used to describe the geographic location of the media; The same regional terms among the first, second, and third regional terms of the media are regarded as associated regional terms that are related to the media. The associated regional terms of the media are filled into the regional information enhancement text template to obtain the regional description information; The media text is obtained by combining the media name and the geographical description information.

5. The method according to claim 1, characterized in that, The text to be identified is encoded to obtain a text encoding vector, including: The text to be identified is subjected to regional word recognition to obtain one or more regional words in the text to be identified. Based on the region words in the text to be identified, a preset word list is queried to obtain the region word vectors corresponding to each region word; Vector fusion is performed on one or more of the regional word vectors to obtain the target word vector; The text to be identified is semantically encoded to obtain a semantic encoding vector; The target word vector and the semantic encoding vector are concatenated to form a text encoding vector.

6. The method according to claim 1, characterized in that, The text to be identified is semantically encoded to obtain a semantic encoding vector, including: When the number of characters in the text to be identified is less than or equal to the preset number of characters, the complete text to be identified is semantically encoded to obtain a semantic encoding vector; When the number of characters in the text to be identified is greater than the preset number of characters, semantic encoding is performed on the merged text consisting of the title of the text to be identified, the pre-preset paragraph text of the text to be identified, and the post-preset paragraph text of the text to be identified, to obtain a semantic encoding vector. The number of characters in the merged text is less than or equal to the preset number of characters. The pre-preset paragraph text is the text of a first preset number of paragraphs located at the beginning of the text to be identified, and the post-preset paragraph text is the text of a second preset number of paragraphs located at the end of the text to be identified.

7. The method according to claim 1, characterized in that, Before encoding the text to be recognized to obtain a text encoding vector, the method further includes: Obtain a training dataset, which includes training text, positive media examples of the training text, and negative media examples of the training text, wherein the positive media examples are the media that published the training text, and the negative media examples are other media besides the positive media examples in the training set; A first encoder and a second encoder are obtained; the first encoder is used to encode the text to be identified and the training text respectively to obtain corresponding text encoding vectors; the second encoder is used to encode the media text to obtain media encoding vectors; Calculate the first similarity between the text encoding vector of the training text and the media encoding vector of the media example; Calculate the second similarity between the text encoding vector of the training text and the media encoding vector of each of the media negative examples; The first learning loss is calculated based on the difference between the first similarity and the preset positive example similarity value, and the difference between each of the second similarities and the preset negative example similarity value. The training parameters of the first encoder and the training parameters of the second encoder are trained using the first learning loss.

8. The method according to claim 7, characterized in that, Obtain the training dataset, including: Retrieve training texts published by leading media outlets on the first channel of information platforms; The first media in which the training text is published is identified as a positive media example of the training text; From multiple texts to be identified that are published on any channel of the information platform, other than the training text, a first preset number of texts to be identified are sampled. From multiple texts to be identified published on the first channel, excluding the training text, a second preset number of texts to be identified are sampled; the second preset number is less than the first preset number. Media that are different from the first media in the sampled text to be identified are used as media negative examples of the training text.

9. The method according to claim 7, characterized in that, The training dataset also includes positive channel examples and negative channel examples of the training text, wherein the positive channel examples are the channels through which the training text is published, and the negative channel examples are other channels besides the channels through which the training text is published; before encoding the text to be identified to obtain a text encoding vector, the method further includes: Obtain a third encoder; the third encoder is used to encode the channel tag of the channel to obtain the channel encoding vector of the channel; Calculate the third similarity between the text encoding vector of the training text and the channel encoding vector of the positive channel example; Calculate the fourth similarity between the text encoding vector of the training text and the channel encoding vector of each of the channel negative examples; The second learning loss is calculated based on the difference between the third similarity and the preset positive example similarity value, and the difference between each of the fourth similarities and the preset negative example similarity value. The training parameters of the first encoder and the training parameters of the third encoder are trained using the second learning loss.

10. A text geographic attribute determination device, characterized in that, The device includes: The text encoding module is configured to acquire the text to be recognized and encode the text to be recognized to obtain a text encoding vector; The media encoding module is configured to acquire the media that publishes the text to be identified, determine the media text of the text to be identified based on the media name of the media that publishes the text to be identified, and encode the media text to obtain a media encoding vector, wherein the media is the account entity that publishes the text on the information platform; The channel encoding module is configured to acquire the channel on which the text to be identified is published and the channel tag of the channel, encode the channel tag of the channel to obtain the channel encoding vector of the channel, wherein the channel is a publishing area on the information platform for publishing text with similar preset features, wherein the channel encoding vector, the media encoding vector and the text encoding vector are encoding vectors in the same vector space, and the similarity between any two encoding vectors in the vector space is used to represent the degree of association between the encoded objects of the two encoding vectors; The encoding fusion module is configured to perform encoding fusion on the text encoding vector, the media encoding vector, and the channel encoding vector to obtain a fused encoding vector; The regional attribute determination module is configured to classify the text to be identified based on the fused encoding vector to determine whether the text to be identified is text with regional attributes.

11. A computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the text geographic attribute determination method according to any one of claims 1 to 9.

12. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the text geographic attribute determination method according to any one of claims 1 to 9 by executing the executable instructions.

Images