Text semantic similarity calculation method, device, equipment and storage medium

Abstract

The present invention relates to the technical field of text analysis, and discloses a method for calculating text semantic similarity, comprising: obtaining a pair of texts to be compared, and extracting the text structural features of each text in the pair of texts to be compared; performing feature scaling and feature standardization on each text structural feature to obtain the optimized text structural features of each text; obtaining the text plane features of each text in the pair of texts to be compared, combining the optimized text structural features of each text with the corresponding text plane features to obtain the structure-plane features of each text; using a kernel function to learn the structure-plane features of each text in a preset support vector regression model to obtain the text similarity of the pair of texts to be compared, and scoring the text similarity through a scoring system to obtain a text similarity score. The present invention also proposes a text semantic similarity calculation device, an electronic device, and a storage medium. The present invention can improve the accuracy of text similarity calculation.

Description

Technical Field

[0001] The present invention relates to the field of text analysis technology, and in particular to a method, device, electronic device, and computer-readable storage medium for calculating text semantic similarity. Background Art

[0002] With the rapid development of internet technology, text similarity calculations have found widespread application in many fields, such as search and question-answering systems. In practical applications, high accuracy is required for text similarity matching. Existing techniques typically convert text into vectors and calculate text similarity based on the similarity between the vectors.

[0003] However, calculating text similarity based solely on text vectors has serious flaws. For example, the texts "I like to eat vegetables" and "I don't like to eat vegetables" differ by only one character, but their vector representations may be very similar due to the large number of shared characters. However, from a semantic perspective, the two meanings are completely different. Existing methods for calculating text semantic similarity mostly use flat similarity features to represent the similarity of a pair of texts, which are relatively weak in representation. Therefore, it is now necessary to further use structured features to represent the syntactic and semantic information of the texts to calculate text semantic similarity. Summary of the Invention

[0004] The present invention provides a method and device for calculating text semantic similarity and a computer-readable storage medium, the main purpose of which is to improve the accuracy of text similarity calculation.

[0005] To achieve the above object, the present invention provides a method for calculating text semantic similarity, comprising:

[0006] Obtaining a pair of texts to be compared, and extracting text structure features of each text in the pair of texts to be compared;

[0007] Performing feature scaling and feature standardization on the text structure features of each of the texts to obtain optimized text structure features of each of the texts;

[0008] Obtaining text plane features of each text in the text pair to be compared, and combining the optimized text structural features of each text with the corresponding text plane features to obtain structure-plane features of each text;

[0009] The structure-plane features of each text are learned in a preset support vector regression model using a kernel function to obtain the text similarity of the text pairs to be compared, and the text similarity is scored using a preset scoring system to obtain a text similarity score.

[0010] Optionally, extracting the text structure features of each text in the text pair to be compared includes:

[0011] Extracting a first text and a second text from the pair of texts to be compared;

[0012] Performing word segmentation processing on the first text and the second text respectively to obtain a first text word segmentation set and a second text word segmentation set;

[0013] Calculating the distance between each word in the first text segmentation set and the second text segmentation set using a preset distance calculation method, and pairing the words in the first text segmentation set and the second text segmentation set according to the distance to obtain a word correspondence relationship;

[0014] Analyzing dependency relationships of the first text and the second text respectively, and constructing a first dependency tree of the first text and a second dependency tree of the second text according to the dependency relationships;

[0015] According to the first dependency tree, the second dependency tree, and the word correspondence, text structure features of the first text and text structure features of the second text are obtained.

[0016] Optionally, the respectively analyzing the dependency relationships of the first text and the second text, and constructing a first dependency tree of the first text and a second dependency tree of the second text according to the dependency relationships, includes:

[0017] Performing grammatical analysis on the first text and the second text, marking the parts of speech and phrase structures of the first text and the second text;

[0018] Constructing dependency arcs between a first text segmentation set of the first text and a second text segmentation set of the second text according to the parts of speech and phrase structures of the first text and the second text;

[0019] A first dependency tree of the first text is constructed based on dependency arcs between the first text word sets, and a second dependency tree of the second text is constructed based on dependency arcs between the second text word sets.

[0020] Optionally, performing feature scaling and feature standardization on the text structure features of each of the texts includes:

[0021] A preset standard feature range is obtained, and the text structure feature of each text is scaled to the minimum and maximum until the scaled text structure feature is within the standard feature range, thereby obtaining the optimized text structure feature of each text.

[0022] Optionally, obtaining the text plane features of each text in the text pair to be compared includes:

[0023] Performing noise reduction and standardization processing on the first text and the second text in the text pair to be compared, respectively, to obtain a first standard text and a second standard text;

[0024] The first standard text is segmented to obtain a first text string, and the second standard text is segmented to obtain a second text string, and plane features of the first text string and the second text string are extracted to obtain text plane features of the first text and the second text.

[0025] Optionally, combining the optimized text structure feature of each text with the corresponding text plane feature to obtain the structure-plane feature of each text includes:

[0026] Obtain a preset multi-layer perceptron, and concatenate the optimized text structure features and the corresponding text plane features of each text at the input layer of the multi-layer perceptron to obtain two input vectors;

[0027] The two input vectors are subjected to nonlinear transformation and feature synthesis through the hidden layer of the multi-layer perceptron to obtain the structural-plane features of each of the texts.

[0028] Optionally, the kernel function is used to learn the structure-plane features of each text in a preset support vector regression model to obtain the text similarity of the text pair to be compared, including:

[0029] Calculating the Euclidean distance between the structure-plane features of two texts, and calculating the kernel function value between the structure-plane features of the two texts;

[0030] Calculating the similarity of the two structure-plane features according to the Euclidean distance and the kernel function value between the structure-plane features of the two texts;

[0031] Based on the similarity between the structure-plane features, regression prediction is performed using the support vector regression model to obtain the text similarity of the text pair to be compared.

[0032] In order to solve the above problems, the present invention further provides a text semantic similarity calculation device, the device comprising:

[0033] A structural feature extraction module is used to perform feature scaling and feature standardization on the text structural features of each of the texts to obtain optimized text structural features of each of the texts;

[0034] A structural feature optimization module, configured to perform feature scaling and feature standardization on the text structural features of each of the texts to obtain optimized text structural features of each of the texts;

[0035] a feature combining module, configured to obtain text plane features of each text in the text pair to be compared, and combine the optimized text structural features of each text with the corresponding text plane features to obtain structure-plane features of each text;

[0036] The similarity calculation module is used to use a kernel function to learn the structure-plane features of each text in a preset support vector regression model to obtain the text similarity of the text pairs to be compared, and to score the text similarity using a preset scoring system to obtain a text similarity score.

[0037] In order to solve the above problem, the present invention further provides an electronic device, comprising:

[0038] at least one processor; and,

[0039] a memory communicatively connected to the at least one processor; wherein,

[0040] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can perform the above-mentioned text semantic similarity calculation method.

[0041] In order to solve the above problem, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned text semantic similarity calculation method.

[0042] After obtaining the text pairs to be compared, the embodiment of the present invention extracts the text structure features and text plane features of each text respectively, and after optimizing the text structure features to obtain optimized text structure features, the text plane features and the optimized text structure features are combined to obtain structure-plane features to represent the text pairs to be compared, thereby realizing the calculation of the compared text pairs based on the advantages of the two features. Finally, the two structure plane features are calculated using the kernel function and the support vector regression model to obtain the text similarity between the text pairs to be compared. The present invention calculates the similarity of the text pairs to be compared by combining the two features, and quantifies it by scoring, so as to improve the accuracy of the text similarity calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flowchart of a method for calculating text semantic similarity provided by one embodiment of the present invention;

[0044] Figure 2 for Figure 1 A schematic diagram of a detailed implementation flow of one of the steps in the text semantic similarity calculation method shown;

[0045] Figure 3 for Figure 1 A schematic diagram of a detailed implementation flow of one of the steps in the text semantic similarity calculation method shown;

[0046] Figure 4 A functional module diagram of a text semantic similarity calculation device provided by one embodiment of the present invention;

[0047] Figure 5 A schematic diagram of the structure of an electronic device for implementing the text semantic similarity calculation method provided by one embodiment of the present invention.

[0048] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0050] The embodiment of the present application provides a method for calculating text semantic similarity. The execution subject of the text semantic similarity calculation method includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the text semantic similarity calculation method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0051] Reference Figure 1 FIG. 1 is a flow chart of a method for calculating text semantic similarity provided by an embodiment of the present invention.

[0052] In this embodiment, the text semantic similarity calculation method includes:

[0053] S1. Obtain a text pair to be compared, and extract text structure features of each text in the text pair to be compared.

[0054] In the embodiment of the present invention, the text pair to be compared is a text pair for which text semantic similarity needs to be calculated. The text pair to be compared includes a first text and a second text. For example, the first text is "Today's weather is very good" and the second text is "Today's weather is very beautiful."

[0055] In an embodiment of the present invention, the text structure features are mainly features representing the text structure, such as the syntactic structure features of the text. For example, although the two texts "tiger attacks lion" and "lion attacks tiger" have the same vocabulary, their syntactic structures are different. From the perspective of the master-slave relationship, the agents and patients in the two texts are completely opposite.

[0056] The agent is the person who performs the action and is proactive, while the patient is the person who receives the action and is passive. For example, in the text "a tiger attacks a lion", the tiger is the agent and the lion is the patient.

[0057] See Figure 2 As shown, further, extracting the text structure features of each text in the text pair to be compared includes:

[0058] S101, extracting a first text and a second text from the text pair to be compared;

[0059] S102: Perform word segmentation processing on the first text and the second text respectively to obtain a first text word segmentation set and a second text word segmentation set;

[0060] S103, using a preset distance calculation method to calculate the distance between each word in the first text segmentation set and the second text segmentation set, and pairing the words in the first text segmentation set and the second text segmentation set according to the distance to obtain a word correspondence relationship;

[0061] S104: Analyze the dependency relationships of the first text and the second text respectively, and construct a first dependency tree of the first text and a second dependency tree of the second text according to the dependency relationships;

[0062] S105 : Obtain text structure features of the first text and text structure features of the second text according to the first dependency tree, the second dependency tree, and the word correspondence.

[0063] In the embodiment of the present invention, the Euclidean distance calculation formula may be used to calculate the distance between each word in the first text word segmentation set and the second text word segmentation set.

[0064] In the embodiment of the present invention, the dependency relationship refers to the grammatical relationship between a group of words in a sentence. This relationship can be represented as a tree structure, which is the dependency tree. In a given text, each word has only one parent node, which can form the dependency tree.

[0065] Specifically, in natural language processing, the representation of text is often related to its content and context, representing the information and meaning it carries at the semantic level. Using flat features to represent the similarity between a pair of texts is relatively weak, so structured features are needed to further represent the similarity relationship between texts.

[0066] Furthermore, the respectively analyzing the dependency relationships of the first text and the second text, and constructing a first dependency tree of the first text and a second dependency tree of the second text according to the dependency relationships, includes:

[0067] Performing grammatical analysis on the first text and the second text, marking the parts of speech and phrase structures of the first text and the second text;

[0068] Constructing dependency arcs between a first text segmentation set of the first text and a second text segmentation set of the second text according to the parts of speech and phrase structures of the first text and the second text;

[0069] A first dependency tree of the first text is constructed based on dependency arcs between the first text word sets, and a second dependency tree of the second text is constructed based on dependency arcs between the second text word sets.

[0070] In an embodiment of the present invention, the dependency arc represents a directed arc of a specific syntactic relationship between two words, and the direction of the dependency arc is from the subordinate word to the dominant word. For example, in "staff need to calculate text similarity", "need" is a subordinate word, "staff" is the dominant word of "need", "calculate" is a subordinate word, and "text similarity" is the dominant word of "calculate".

[0071] In the embodiment of the present invention, before extracting the text structure features of each text in the text pair to be compared, the method further includes:

[0072] Performing symbol screening on each text in the text pair to be compared, removing redundant symbols in the text pair to be compared, and obtaining a cleaned text pair;

[0073] Stop words in the cleaned text pair are removed by a preset text screening method, and the text language in the cleaned text pair is unified to obtain a cleaned text pair to be compared.

[0074] The embodiment of the present invention can avoid the appearance of unrecognizable symbols in the text when the text is subsequently processed by screening the text for symbols, and unify the text language in the cleaned text pair, thereby avoiding the need to call multiple language processing methods when processing the text pair.

[0075] S2. Perform feature scaling and feature standardization on the text structure features of each of the texts to obtain optimized text structure features of each of the texts.

[0076] In the embodiments of the present invention, feature scaling and feature standardization are performed on text structure features to improve the accuracy and effectiveness of features applied to tasks such as text classification and sentiment analysis.

[0077] In detail, the feature scaling and feature standardization processing of the text structure features of each of the texts includes:

[0078] A preset standard feature range is obtained, and the text structure feature of each text is scaled to the minimum and maximum until the scaled text structure feature is within the standard feature range, thereby obtaining the optimized text structure feature of each text.

[0079] In the embodiment of the present invention, the standard feature range is the value range of a specific feature, and the feature standardization of the text structure feature of the text is the process of making the text structure feature fall into the value range.

[0080] Furthermore, the present invention performs feature scaling by converting the text structure features of the text into a vector form, wherein scaling the text structure features of the text to within a preset standard feature range can facilitate feature combination and calculation.

[0081] S3. Obtain text plane features of each text in the text pair to be compared, and combine the optimized text structural features of each text with the corresponding text plane features to obtain structure-plane features of each text.

[0082] In the embodiment of the present invention, the text plane feature refers to the visual two-dimensional representation of the text, such as a character string feature, a text extension feature, and the like.

[0083] See Figure 3 As shown, in the embodiment of the present invention, obtaining the text plane features of each text in the text pair to be compared includes:

[0084] S301, performing noise reduction and standardization processing on the first text and the second text in the text pair to be compared, respectively, to obtain a first standard text and a second standard text;

[0085] S302 , segmenting the first standard text to obtain a first text string, segmenting the second standard text to obtain a second text string, and extracting plane features of the first text string and the second text string to obtain text plane features of the first text and the second text.

[0086] Specifically, the standardization is to unify the language format of the texts to be compared, remove stop words, etc.

[0087] In the embodiment of the present invention, regular expression matching may be used to segment the first and second standard texts to obtain first and second text strings.

[0088] In an embodiment of the present invention, the text plane features include string features and text extension features. Therefore, the embodiment of the present invention separately extracts the string features and text extension features of the text string and combines them to obtain the text plane features of each text. Specifically, the embodiment of the present invention can obtain string features by extracting word frequency and part of speech from the text string, and obtain text extension features by extracting text length, text type, and sentence complexity from the text string.

[0089] Furthermore, in the embodiment of the present invention, the optimized text structure features and the corresponding text plane features of the text can be directly combined to form a set feature as a structure-plane feature. i Represents the text plane feature, v i When optimizing text structure features, you can directly <t i ,v i > as structural-plane features of the text pairs to be compared.

[0090] In another embodiment of the present invention, combining the optimized text structural features of each text with the corresponding text plane features to obtain the structural-plane features of each text includes:

[0091] Obtain a preset multi-layer perceptron, and concatenate the optimized text structure features and the corresponding text plane features of each text at the input layer of the multi-layer perceptron to obtain two input vectors;

[0092] The two input vectors are subjected to nonlinear transformation and feature synthesis through the hidden layer of the multi-layer perceptron to obtain the structural-plane features of each of the texts.

[0093] The Multilayer Perceptron (MLP) is a basic model in artificial neural networks, which is usually used to solve classification and regression problems. Compared with traditional perception models, the MLP model adds one or more hidden layers containing multiple neurons, which can fuse features to make the features have stronger representation capabilities.

[0094] In an embodiment of the present invention, before the input layer of the multilayer perceptron splices the optimized text structure features and the corresponding text plane features of each text, and after the two input vectors are nonlinearly transformed and feature synthesized through the hidden layer of the multilayer perceptron, a step of converting the features and the vectors into each other may be included, wherein, since the features are text features, the features and the vectors can be converted into each other by digital coding.

[0095] S4. Using a kernel function, the structure-plane features of each text are learned in a preset support vector regression model to obtain the text similarity of the text pair to be compared, and the text similarity is scored using a preset scoring system to obtain a text similarity score.

[0096] In the embodiment of the present invention, the support vector regression (SVR) model is a non-parametric regression model based on the support vector machine (SVM), which is used in this solution to calculate the text similarity of the text pairs to be compared.

[0097] Among them, the kernel function is a common function in machine learning, which is usually used to map data in the original space to a high-dimensional space, and is used in support vector machines to expand the capabilities of linear models and improve the performance of classification and regression tasks, such as calculating text similarity.

[0098] In an embodiment of the present invention, the scoring system may return a similarity calculation score for a given pair of texts, for example, the score ranges from 0 to 10, where 0 indicates that the meanings of the pair of texts are completely unrelated, and 10 indicates that the meanings of the pair of texts are exactly the same.

[0099] In addition, the performance with the same score can also be evaluated by the Pearson correlation coefficient of the similarity score calculated by the system and the manual judgment score, where the Pearson correlation coefficient is used to measure the correlation between two variables X and Y, and its value is between -1 and 1. When the Pearson correlation coefficient is close to -1, it indicates negative correlation, and when it is close to 0, it indicates no correlation.

[0100] Furthermore, the kernel function is used to learn the structural-plane features of each text in a preset support vector regression model to obtain the text similarity of the text pairs to be compared, including:

[0101] Calculating the Euclidean distance between the structure-plane features of two texts, and calculating the kernel function value between the structure-plane features of the two texts;

[0102] Calculating the similarity of the two structure-plane features according to the Euclidean distance and the kernel function value between the structure-plane features of the two texts;

[0103] Based on the similarity between the structure-plane features, regression prediction is performed using the support vector regression model to obtain the text similarity of the text pair to be compared.

[0104] Furthermore, the kernel function formula for calculating the kernel function value between the structure-plane features of the two texts is:

[0105] K(x i ,x j )=K TK (t i ,t j )+K fvec (v i ,v j )

[0106] Among them, K TK K is the kernel function for calculating structural features, fvec is the kernel function for calculating plane features, t i and v i Composed of a structure-plane feature, t j and v j Composed of a structure-plane feature, K(x i ,x j ) is the kernel function value.

[0107] In the embodiment of the present invention, the calculation formula for calculating the similarity of the two structure-plane features based on the Euclidean distance between the structure-plane features of the two texts and the kernel function value is:

[0108] S ij =K(x i ,x j )×exp(-γd ij )

[0109] Among them, γ is a hyperparameter that controls the relationship between the Euclidean distance and the kernel function value, K(x i ,x j ) is the structural-plane feature x i and structure-plane feature x j The kernel function value between ij is the similarity between two structure-plane features, d ij is the Euclidean distance between two structural-planar features.

[0110] After obtaining the text pairs to be compared, the embodiment of the present invention extracts the text structure features and text plane features of each text respectively, and after optimizing the text structure features to obtain optimized text structure features, the text plane features and the optimized text structure features are combined to obtain structure-plane features to represent the text pairs to be compared, thereby realizing the calculation of the compared text pairs based on the advantages of the two features. Finally, the two structure plane features are calculated using the kernel function and the support vector regression model to obtain the text similarity between the text pairs to be compared. The present invention calculates the similarity of the text pairs to be compared by combining the two features, and quantifies it by scoring, so as to improve the accuracy of the text similarity calculation.

[0111] like Figure 4 , which is a functional module diagram of a text semantic similarity calculation device provided by an embodiment of the present invention.

[0112] The text semantic similarity calculation device 100 described in the present invention can be installed in an electronic device. Depending on the functionality implemented, the text semantic similarity calculation device 100 may include a structural feature extraction module 101, a structural feature optimization module 102, a feature combination module 103, and a text similarity calculation module 104. A module described in the present invention, also referred to as a unit, refers to a series of computer program segments that can be executed by an electronic device processor and perform a fixed function, and is stored in the memory of the electronic device.

[0113] In this embodiment, the functions of each module / unit are as follows:

[0114] The structural feature extraction module 101 is used to perform feature scaling and feature standardization on the text structural features of each of the texts to obtain optimized text structural features of each of the texts;

[0115] The structural feature optimization module 102 is used to perform feature scaling and feature standardization on the text structural features of each of the texts to obtain optimized text structural features of each of the texts;

[0116] A feature combining module 103 is configured to obtain text plane features of each text in the text pair to be compared, and combine the optimized text structural features of each text with the corresponding text plane features to obtain a structural-plane feature of each text;

[0117] The similarity calculation module 104 is used to use a kernel function to learn the structure-plane features of each text in a preset support vector regression model to obtain the text similarity of the text pair to be compared, and to score the text similarity using a preset scoring system to obtain a text similarity score.

[0118] In detail, each module in the text semantic similarity calculation device 100 according to the embodiment of the present invention adopts the same method as above when in use. Figures 1 to 3 The text semantic similarity calculation method described in the text semantic similarity calculation method is the same technical means and can produce the same technical effect, so it will not be repeated here.

[0119] like Figure 5 FIG. 1 is a schematic diagram of the structure of an electronic device for implementing a method for calculating text semantic similarity provided by an embodiment of the present invention.

[0120] The electronic device 1 may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13. It may also include a computer program stored in the memory 11 and executable on the processor 10, such as a text semantic similarity calculation program.

[0121] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing or executing programs or modules stored in the memory 11 (for example, executing a text semantic similarity calculation program, etc.), as well as calling data stored in the memory 11, to perform various functions of the electronic device and process data.

[0122] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 can also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the memory 11 can also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of a text semantic similarity calculation program, but can also be used to temporarily store data that has been output or is to be output.

[0123] The communication bus 12 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0124] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0125] Figure 5 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 5 The structure shown does not constitute a limitation on the electronic device 1 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0126] For example, although not shown, the electronic device may further include a power source (such as a battery) for powering various components. Preferably, the power source may be logically connected to the at least one processor 10 via a power management device, thereby implementing functions such as charge management, discharge management, and power consumption management through the power management device. The power source may further include any components such as one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, etc. The electronic device may further include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0127] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0128] The text semantic similarity calculation program stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve the following:

[0129] Obtaining a pair of texts to be compared, and extracting text structure features of each text in the pair of texts to be compared;

[0130] Performing feature scaling and feature standardization on the text structure features of each of the texts to obtain optimized text structure features of each of the texts;

[0131] Obtaining text plane features of each text in the text pair to be compared, and combining the optimized text structural features of each text with the corresponding text plane features to obtain structure-plane features of each text;

[0132] The structure-plane features of each text are learned in a preset support vector regression model using a kernel function to obtain the text similarity of the text pairs to be compared, and the text similarity is scored using a preset scoring system to obtain a text similarity score.

[0133] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, which will not be repeated here.

[0134] Furthermore, if the modules / units integrated into the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0135] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor of an electronic device, the computer program can implement:

[0136] Obtaining a pair of texts to be compared, and extracting text structure features of each text in the pair of texts to be compared;

[0137] Performing feature scaling and feature standardization on the text structure features of each of the texts to obtain optimized text structure features of each of the texts;

[0138] Obtaining text plane features of each text in the text pair to be compared, and combining the optimized text structural features of each text with the corresponding text plane features to obtain structure-plane features of each text;

[0139] The structure-plane features of each text are learned in a preset support vector regression model using a kernel function to obtain the text similarity of the text pairs to be compared, and the text similarity is scored using a preset scoring system to obtain a text similarity score.

[0140] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and other division methods may be used in actual implementation.

[0141] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0142] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0143] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0144] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0145] Blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, verification mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0146] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0147] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. Terms such as "first" and "second" are used to indicate names and do not imply any particular order.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for calculating text semantic similarity, characterized in that: The method comprises: Obtaining a text pair to be compared, extracting a first text and a second text from the text pair to be compared, pairing words in a first text segmentation set of the first text with words in a second text segmentation set of the second text to obtain word correspondences, analyzing dependency relationships between the first text and the second text, constructing a first dependency tree for the first text and a second dependency tree for the second text based on the dependency relationships, and obtaining text structural features of the first text and the second text based on the first dependency tree and the second dependency tree, as well as the word correspondences; Performing feature scaling and feature standardization on the text structure features of each of the texts to obtain optimized text structure features of each of the texts; Combining the string features and text extension features of the first text to obtain text plane features of the first text, combining the string features and text extension features of the second text to obtain text plane features of the second text, combining the optimized text structure features and text plane features of the first text to obtain structure-plane features of the first text, and combining the optimized text structure features and text plane features of the second text to obtain structure-plane features of the second text; Using a kernel function, the structural-plane features of the first text and the structural-plane features of the second text are learned in a preset support vector regression model to obtain the text similarity of the text pair to be compared, and the text similarity is scored using a preset scoring system to obtain a text similarity score.

2. The text semantic similarity calculation method according to claim 1, wherein: The pairing of words in the first text segmentation set of the first text with words in the second text segmentation set of the second text to obtain a word correspondence relationship includes: Performing word segmentation processing on the first text and the second text respectively to obtain a first text word segmentation set and a second text word segmentation set; The distance between each word in the first text segmentation set and the second text segmentation set is calculated using a preset distance calculation method, and the words in the first text segmentation set and the second text segmentation set are paired according to the distance to obtain a word correspondence relationship.

3. The text semantic similarity calculation method as described in claim 1, characterized in that: The respectively analyzing the dependency relationships of the first text and the second text, and constructing a first dependency tree of the first text and a second dependency tree of the second text according to the dependency relationships, includes: Performing grammatical analysis on the first text and the second text, marking the parts of speech and phrase structures of the first text and the second text; Constructing dependency arcs between a first text segmentation set of the first text and a second text segmentation set of the second text according to the parts of speech and phrase structures of the first text and the second text; A first dependency tree of the first text is constructed based on dependency arcs between the first text word sets, and a second dependency tree of the second text is constructed based on dependency arcs between the second text word sets.

4. The text semantic similarity calculation according to claim 1, wherein: The feature scaling and feature standardization processing of the text structure features of each of the texts includes: A preset standard feature range is obtained, and the text structure feature of each text is scaled to the minimum and maximum until the scaled text structure feature is within the standard feature range, thereby obtaining the optimized text structure feature of each text.

5. The text semantic similarity calculation method as described in claim 1, characterized in that: Before combining the character string feature and the text extension feature of the first text to obtain the text plane feature of the first text, the method further includes: Performing noise reduction and standardization processing on the first text and the second text in the text pair to be compared, respectively, to obtain a first standard text and a second standard text; The first standard text is segmented to obtain a first text string, and the second standard text is segmented to obtain a second text string.

6. The text semantic similarity calculation according to claim 1, wherein: The step of combining the optimized text structural features and text plane features of the first text to obtain the structural-plane features of the first text, and combining the optimized text structural features and text plane features of the second text to obtain the structural-plane features of the second text includes: Obtain a preset multi-layer perceptron, and concatenate the optimized text structure features and the corresponding text plane features of each text at the input layer of the multi-layer perceptron to obtain two input vectors; The two input vectors are subjected to nonlinear transformation and feature synthesis through the hidden layer of the multi-layer perceptron to obtain the structural-plane features of each of the texts.

7. The text semantic similarity calculation method as claimed in claim 1, characterized in that: The method of using a kernel function to learn the structure-plane features of the first text and the structure-plane features of the second text in a preset support vector regression model to obtain the text similarity of the text pair to be compared includes: Calculating the Euclidean distance between the structure-plane features of two texts, and calculating the kernel function value between the structure-plane features of the two texts; Calculating the similarity of the two structure-plane features according to the Euclidean distance and the kernel function value between the structure-plane features of the two texts; Based on the similarity between the structure-plane features, regression prediction is performed using the support vector regression model to obtain the text similarity of the text pair to be compared.

8. A text semantic similarity calculation device, characterized in that: The device comprises: a structural feature extraction module, configured to obtain a pair of texts to be compared, extract a first text and a second text from the pair of texts to be compared, pair words in a first text segmentation set of the first text with words in a second text segmentation set of the second text to obtain word correspondences, analyze dependency relationships between the first text and the second text, construct a first dependency tree for the first text and a second dependency tree for the second text based on the dependency relationships, and obtain text structural features of the first text and text structural features of the second text based on the first dependency tree and the second dependency tree, as well as the word correspondences; A structural feature optimization module, configured to perform feature scaling and feature standardization on the text structural features of each of the texts to obtain optimized text structural features of each of the texts; a feature combining module, configured to combine the string features and text extension features of the first text to obtain text plane features of the first text, combine the string features and text extension features of the second text to obtain text plane features of the second text, combine the optimized text structure features and text plane features of the first text to obtain structure-plane features of the first text, and combine the optimized text structure features and text plane features of the second text to obtain structure-plane features of the second text; The similarity calculation module is used to use a kernel function to learn the structure-plane features of the first text and the structure-plane features of the second text in a preset support vector regression model to obtain the text similarity of the text pair to be compared, and to score the text similarity using a preset scoring system to obtain a text similarity score.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the text semantic similarity calculation method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for calculating text semantic similarity according to any one of claims 1 to 7 is implemented.

Images