Translation result determination method and device, equipment, storage medium and program product

Abstract

The invention relates to the technical field of data processing, and discloses a translation result determination method and device, equipment, a storage medium and a program product. The method comprises the following steps: acquiring initial service text data of an online car-hailing service; performing preprocessing and feature engineering processing on the service text data to obtain target service text data; and processing the target service text data by using a pre-trained neural network model to obtain a translation result corresponding to the initial service text data. According to the method, the pre-trained neural network model is utilized, so that complex language phenomena and context differences can be processed, and a translation result which is more accurate than that of a traditional translation method based on rules and vocabularies is provided. The neural network model can learn internal rules and modes of languages from a large amount of data, so that service text data can be more accurately understood and translated.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method, device, equipment, storage medium and program product for determining a translation result. Background Art

[0002] With the accelerated advancement of globalization and the rapid development of Internet technology, online car-hailing services, as an important part of the sharing economy, have rapidly penetrated into many countries and regions around the world. This trend has not only greatly facilitated people's travel methods, but also put forward higher requirements for the international operation capabilities of online car-hailing companies. Among them, multilingual support, as a key link in improving user experience and service quality, has become an important challenge that online car-hailing companies must face.

[0003] In the internationalization process of online ride-hailing services, users come from all over the world and communicate in different languages. In order to meet the diverse needs of these users, online ride-hailing companies must build an efficient and accurate multilingual support system to ensure the consistency and accuracy of services. This system not only needs to be able to handle requests and feedback from users in different languages, but also needs to be able to accurately understand and respond to these requests to provide personalized travel services.

[0004] Currently, the implementation of multilingual support systems for online ride-hailing services mainly relies on translation technology, which uses a large vocabulary (i.e., dictionary) to look up words in user input and translate them into corresponding words in the target language.

[0005] However, since traditional translation technology mainly relies on predefined rules and vocabularies, these rules and vocabularies often fail to fully cover the diversity and complexity of languages. Therefore, when dealing with some special contexts or complex expressions, inaccurate translation or misunderstanding often occurs, thus affecting the user experience. Summary of the invention

[0006] In view of this, the present invention provides a method, device, equipment, storage medium and program product for determining translation results, so as to solve the problem that traditional translation technology mainly relies on predefined rules and vocabularies, which often cannot fully cover the diversity and complexity of languages. Therefore, when dealing with some special contexts or complex expressions, inaccurate translation or misunderstanding often occurs, thus affecting the user experience.

[0007] In a first aspect, the present invention provides a method for determining a translation result, the method comprising: obtaining initial service text data of an online car-hailing service; performing preprocessing and feature engineering on the service text data to obtain target service text data; and processing the target service text data using a pre-trained neural network model to obtain a translation result corresponding to the initial service text data.

[0008] The method for determining the translation result provided in this embodiment can handle complex language phenomena and contextual differences by utilizing a pre-trained neural network model, and provide more accurate translation results than traditional rule-based and vocabulary-based translation methods. The neural network model can learn the inherent laws and patterns of the language from a large amount of data, thereby more accurately understanding and translating the service text data. Moreover, the neural network model is not limited to fixed rules and vocabularies, so it can handle a variety of new and unseen language phenomena and expressions. This flexibility and adaptability enables the process to adapt to the ever-changing language environment and provide users with consistent high-quality translation services.

[0009] In addition, the preprocessing and feature engineering steps can optimize the service text data, making it more suitable for processing by the neural network model. At the same time, with the advancement of technology and the increase of computing resources, the training and processing speed of neural network models are also increasing. This enables the process to efficiently process large amounts of data and support a wider range of application scenarios.

[0010] In an optional embodiment, a method for constructing a pre-trained neural network model includes: obtaining a training data set; wherein the training data set includes historical online car-hailing multilingual service text data, and the historical online car-hailing multilingual service text data includes service texts and translation labels in different languages, different expressions and different themes; processing the training data set to obtain a target data set; wherein the target data set includes: word frequency, word order, grammatical structure and semantic information of the service text; using the target data set to train a first model, and during the training process, tuning the first model until the neural network prediction model meets the target conditions, thereby obtaining a pre-trained neural network model.

[0011] The method for determining the translation result provided in this embodiment uses historical online car-hailing multilingual service text data as a training data set, which covers service texts and translation labels in different languages, different expressions and different themes. This diversity of data enables the trained model to understand language phenomena more comprehensively and improve the accuracy and generalization ability of translation. In addition, by processing the training data set, rich features such as word frequency, word order, grammatical structure and semantic information of the service text are extracted. These features not only help the model to better capture the inherent laws and patterns of the language, but also improve the model's ability to understand complex contexts and expressions. During the training process, the first model is continuously tuned, including adjusting model parameters, optimizing algorithm selection, etc., to ensure that the model gradually approaches the target conditions during the training process. This tuning process can significantly improve the predictive performance and generalization ability of the model, making it more reliable and effective in practical applications.

[0012] In an optional embodiment, a training data set is processed to obtain a target data set, including: preprocessing the training data set to obtain first target data; performing text tokenization on the first target data to obtain second target data and an authentication identifier corresponding to the second target data; wherein the second target data is a word or a term; using an embedding matrix to convert the authentication identifier into a vector of fixed dimension; and adding a corresponding position vector to each vector of fixed dimension to obtain a target data set.

[0013] The method for determining the translation result provided in the present embodiment can significantly reduce the complexity of the data through preprocessing steps, such as removing noise, standardizing the format, etc., so that subsequent processing is more efficient and accurate. This helps to reduce the computational burden during model training and may improve the generalization ability of the model. Text tokenization (such as word segmentation, part-of-speech tagging, etc.) converts the original text into more fine-grained words or words, which are easier to be understood and processed by the model. At the same time, assigning an authentication identifier (such as a unique ID) to each unit helps the model establish a mapping relationship between vocabulary and semantics during the training process. An embedding matrix is ​​a technique for mapping vocabulary into a high-dimensional space, in which similar words are close in position in space. Using an embedding matrix to convert the authentication identifier into a vector of fixed dimension can capture the semantic relationship between words, so that the model can better understand the meaning of the text.

[0014] In an optional embodiment, the above method also includes: providing a voice prompt or displaying the translation result in text on the user terminal; when the translation result is displayed in text on the user terminal, in response to a trigger operation of the user, performing a processing operation on the translation result; wherein the processing operation includes: at least one of a copy operation, a share operation, and an edit operation.

[0015] The method for determining the translation result provided by the present embodiment allows the user to intuitively obtain the translation result through voice prompts or text display, and understand the service content without additional operations. This instant feedback mechanism enhances the user's interactive experience and makes the entire service process smoother and more efficient. By providing processing operations such as copying, sharing, and editing, the user can further process and utilize the translation result according to his or her needs. This interactive method not only enhances the user's participation, but also makes the translation result more practical and valuable.

[0016] In an optional embodiment, the service text data is preprocessed and feature-engineered to obtain target service text data, including: removing noise, removing stop words, processing missing values ​​and outliers from the service text data to obtain first data; converting the first data into uppercase and lowercase to obtain second data; wherein the second data is used to represent data with the same text size; performing digital conversion on the second data to obtain third data; wherein the third data is used to represent data with the same format; performing vocabulary normalization on the third data to obtain fourth data; converting the fourth data into data in numerical form, and determining feature data that meets preset conditions from the data in numerical form; scaling the feature data so that each feature data is within the same preset numerical range; and combining each feature data to obtain target service text data.

[0017] The method for determining the translation result provided in the present embodiment can significantly improve the quality of data by removing noise, stop words, and processing missing values ​​and outliers. These steps help to reduce irrelevant information and errors in the data, making subsequent processing more accurate and efficient. The format consistency of the data is ensured by case conversion and digital conversion steps. This helps to eliminate processing errors caused by format differences, so that the model can understand and process data more accurately. The vocabulary in the text data is converted into a unified form through vocabulary normalization, which helps to reduce the impact of vocabulary diversity on model training. At the same time, the text data is converted into data in numerical form, so that the model can process and analyze data more conveniently. By determining the feature data that meets the preset conditions from the data in numerical form and scaling it, the most critical features for model training can be extracted. These features can more accurately reflect the inherent laws and patterns of the text data, thereby improving the prediction performance of the model. The feature data is combined and scaled so that each feature data is within the same preset numerical range. This helps to reduce the amount of calculation during model training and improve training efficiency. At the same time, due to the consistent dimensions between the feature data, the model can converge to the optimal solution more quickly. By preprocessing the service text data comprehensively, the trained model can be made more robust and generalized. The model can better adapt to different types of service text data and improve its accuracy and reliability in practical applications.

[0018] In an optional embodiment, the above method also includes: obtaining a test data set; inputting the test data set into a pre-trained neural network model to obtain a test result; detecting whether the test result meets the preset requirements; if the test result meets the preset requirements, determining that the pre-trained neural network model is qualified; if the test result does not meet the preset requirements, determining that the pre-trained neural network model is unqualified.

[0019] In the method for determining the translation result provided in this embodiment, the test data set is a sample set independent of the training data set, which can objectively evaluate the performance of the neural network model. By inputting the test data set into the model and observing the output results, the performance of the model in practical applications can be intuitively understood, thereby verifying whether it meets the expected requirements.

[0020] In a second aspect, the present invention provides a device for determining a translation result, the device comprising: an acquisition module for acquiring initial service text data of an online car-hailing service; a first processing module for preprocessing and feature engineering the service text data to obtain target service text data; and a second processing module for processing the target service text data using a pre-trained neural network model to obtain a translation result corresponding to the initial service text data.

[0021] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method for determining the translation result of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0022] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for determining a translation result of the first aspect or any corresponding embodiment thereof.

[0023] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the method for determining a translation result of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 is a flow chart of a method for determining a translation result according to an embodiment of the present invention;

[0026] Figure 2 is a flow chart of another method for determining a translation result according to an embodiment of the present invention;

[0027] Figure 3 is a structural block diagram of a device for determining a translation result according to an embodiment of the present invention;

[0028] Figure 4 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0030] Based on relevant technologies, we know that with the accelerated advancement of globalization and the rapid development of Internet technology, online car-hailing services, as an important part of the sharing economy, have rapidly penetrated into many countries and regions around the world. This trend has not only greatly facilitated people's travel methods, but also put forward higher requirements for the international operation capabilities of online car-hailing companies. Among them, multilingual support, as a key link in improving user experience and service quality, has become an important challenge that online car-hailing companies must face.

[0031] In the internationalization process of online ride-hailing services, users come from all over the world and communicate in different languages. In order to meet the diverse needs of these users, online ride-hailing companies must build an efficient and accurate multilingual support system to ensure the consistency and accuracy of services. This system not only needs to be able to handle requests and feedback from users in different languages, but also needs to be able to accurately understand and respond to these requests to provide personalized travel services.

[0032] Currently, the implementation of multilingual support systems for online ride-hailing services mainly relies on translation technology, which uses a large vocabulary (i.e., dictionary) to look up words in user input and translate them into corresponding words in the target language.

[0033] However, since traditional translation technology mainly relies on predefined rules and vocabularies, these rules and vocabularies often fail to fully cover the diversity and complexity of languages. Therefore, when dealing with some special contexts or complex expressions, inaccurate translation or misunderstanding often occurs, thus affecting the user experience.

[0034] Based on this, the present invention provides a method for determining translation results. By utilizing a pre-trained neural network model, it is able to handle complex language phenomena and contextual differences, and provide more accurate translation results than traditional rule-based and vocabulary-based translation methods. The neural network model can learn the inherent laws and patterns of the language from a large amount of data, thereby more accurately understanding and translating service text data. Moreover, the neural network model is not limited to fixed rules and vocabularies, so it can handle a variety of new and unseen language phenomena and expressions. This flexibility and adaptability enables the process to adapt to the ever-changing language environment and provide users with consistent high-quality translation services.

[0035] In addition, the preprocessing and feature engineering steps can optimize the service text data, making it more suitable for processing by the neural network model. At the same time, with the advancement of technology and the increase of computing resources, the training and processing speed of neural network models are also increasing. This enables the process to efficiently process large amounts of data and support a wider range of application scenarios.

[0036] According to an embodiment of the present invention, an embodiment of a method for determining a translation result is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] In this embodiment, a method for determining a translation result is provided, which can be used in computer devices, such as computers, servers, etc. Figure 1 is a flow chart of a method for determining a translation result according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0038] Step S101, obtaining the initial service text data of the online car-hailing service.

[0039] The initial service text data is used to characterize the various text information generated during the online car-hailing service. The original text information generated during the online car-hailing service has not been processed or converted in any way. Among them, the initial service text data includes but is not limited to the passenger's order information (such as starting point, end point, travel time), communication records between the driver and the passenger, service evaluation, etc. Online car-hailing service: a new mode of transportation that provides travel services to passengers by reserving and dispatching vehicles through an online platform. Specifically, the initial service text data can be directly obtained through the database of the online car-hailing service platform, or it can be imported from a third-party system through an API interface, etc., which is not specifically limited here and can be implemented by technical personnel in this field.

[0040] Step S102: Preprocess and perform feature engineering on the service text data to obtain the target service text data.

[0041] Preprocessing may include removing noise (such as invalid characters, garbled codes), removing stop words (such as meaningless words like "de", "le", etc.), handling missing values and outliers (such as null values, abnormally long texts, etc.), and case conversion. Feature engineering is used to transform the preprocessed text data into a format suitable for processing by a neural network model, including text vectorization (such as bag-of-words model, TF-IDF, Word2Vec, etc.), vocabulary normalization (such as unifying vocabulary spellings, removing synonyms, etc.), and numericalization (converting text into numerical form).

[0042] In this embodiment, preprocessing can be used to represent the process of cleaning and transforming the initial service text data before data analysis or machine learning. Feature engineering can be used to represent the process of transforming the cleaned and transformed initial service text data into a form more suitable for processing by a machine learning model. Specifically, the pandas library in Python is used for data cleaning and preprocessing. Natural language processing libraries (such as NLTK, jieba) can be used for text tokenization and stop word removal. Word embedding techniques (such as Word2Vec, BERT) can be used to convert text into vector representations. Feature selection or dimensionality reduction techniques (such as PCA, SVD) can be used to optimize feature representations. For example: Preprocess the order information in the initial service text data, remove invalid characters and stop words, uniformly convert the starting and ending addresses into longitude and latitude formats, and perform numericalization. Use the Word2Vec model to convert the communication records between drivers and passengers into vector representations as input features for the neural network model.

[0043] Step S103: Process the target service text data using a pre-trained neural network model to obtain the translation result corresponding to the initial service text data.

[0044] Select a neural network model suitable for processing the target text data (such as RNN, LSTM, Transformer, etc.). This model should have been pre-trained on a relevant dataset and have a certain generalization ability. Use the target service text data as the input of the model, and calculate the output result through the forward propagation process of the model. The output result is the translation result corresponding to the initial service text data or other forms of processing results (such as sentiment analysis, intent recognition, etc.). For example: When it is necessary to translate Chinese evaluations in online car-hailing services into English, a pre-trained machine translation model (such as a model based on Transformer) can be used. Take the target service text data (Chinese evaluations) as the input and obtain the English translation result as the output.

[0045] The method for determining the translation result provided in this embodiment can handle complex language phenomena and contextual differences by utilizing a pre-trained neural network model, and provide more accurate translation results than traditional rule-based and vocabulary-based translation methods. The neural network model can learn the inherent laws and patterns of the language from a large amount of data, thereby more accurately understanding and translating the service text data. Moreover, the neural network model is not limited to fixed rules and vocabularies, so it can handle a variety of new and unseen language phenomena and expressions. This flexibility and adaptability enables the process to adapt to the ever-changing language environment and provide users with consistent high-quality translation services.

[0046] In addition, the preprocessing and feature engineering steps can optimize the service text data, making it more suitable for processing by the neural network model. At the same time, with the advancement of technology and the increase of computing resources, the training and processing speed of neural network models are also increasing. This enables the process to efficiently process large amounts of data and support a wider range of application scenarios.

[0047] In an optional embodiment, a method of constructing a pre-trained neural network model includes:

[0048] Step a1, obtaining a training data set; wherein the training data set includes historical online car-hailing multilingual service text data, and the historical online car-hailing multilingual service text data includes service texts and translation labels in different languages, different expressions and different themes.

[0049] Specifically, multilingual service text data is collected from the historical records of the online ride-hailing platform. These data may come from the chat records between drivers and passengers, passenger feedback, service requests, etc. Data screening: Ensure that the collected data is valid and remove duplicate, incomplete or irrelevant data. Data annotation: Provide translation labels for text data in each language. These labels can be manually translated or generated with the assistance of machine translation tools, but they need to be manually verified to improve accuracy. The translation label can be the translated target language text corresponding to the original text, which is used as a label for supervised learning. Specifically, crawler technology is used to crawl data from the website or APP of the online ride-hailing platform. It is also possible to cooperate with the online ride-hailing platform to directly obtain historical data in its database. It is also possible to use open source data sets or third-party data providers to obtain relevant data. For example: 100,000 Chinese-English online ride-hailing service text data were collected, including passengers' inquiries (such as "How much does it cost to go to the airport?") and drivers' answers (such as "It's about 50 yuan, sir." and its corresponding Chinese translation).

[0050] Step a2, processing the training data set to obtain a target data set; wherein the target data set includes: word frequency, word order, grammatical structure and semantic information of the service text.

[0051] Word frequency: the frequency of a word appearing in a text. Word order: the order in which words are arranged in a sentence. Grammatical structure: the grammatical relationship between words in a sentence. Semantic information: the meaning expressed by a word or sentence. Specifically, processing the training data set may include: Text preprocessing: including word segmentation, removal of stop words, stemming, etc., to simplify the text and reduce noise. Feature extraction: calculating word frequency, word order, grammatical structure and semantic information. This can be achieved through natural language processing technologies such as TF-IDF, Word2Vec, BERT, etc. Data formatting: converting the processed data into the format required for model training, such as serialization, vectorization, etc., which is not specifically limited here and can be implemented by those skilled in the art.

[0052] Specifically, the above step a2 includes:

[0053] Step a21, preprocessing the training data set to obtain first target data.

[0054] The training data set is processed in a series of ways to remove noise, standardize the format, and prepare the data for subsequent steps. This usually includes text cleaning (such as removing HTML tags, special characters, extra spaces, etc.), text normalization (such as converting all text to lowercase or uppercase), word segmentation (splitting sentences into words or terms), etc., to obtain the first target data. For example: the text "How much does it cost to go to the airport?" is cleaned into "how much does it cost to go to the airport". Convert the cleaned text to lowercase: "how much does it cost to go to the airport". Use the word segmentation tool to split it into words: "how", "much", "does", "it", "cost", "to", "go", "to", "the", "airport".

[0055] Step a22, performing text tokenization processing on the first target data to obtain second target data and an authentication identifier corresponding to the second target data; wherein the second target data is a word or a character.

[0056] Convert the segmented text (i.e., the first target data) into a format that can be understood and processed by a machine. This usually involves assigning a unique identifier (such as an index) to each word or term and creating a mapping table (or vocabulary) to record these identifiers. At the same time, generate an authentication identifier corresponding to each word or term (although in this context, "authentication identifier" may not be a standard term, it can be understood as an identifier or index of a word or term). For example: if the vocabulary is {"how":1,"much":2}, then the authentication identifier of the word "how" is 1, and the authentication identifier of "much" is 2.

[0057] Step a23, using the embedding matrix to convert the authentication identifier into a vector of fixed dimension.

[0058] Use a pre-trained word embedding model (such as Word2Vec, GloVe, BERT, etc.) or a custom embedding layer to convert the authentication identifier (index) of each word into a vector of fixed dimension. These vectors are usually able to capture the semantic relationship between words. For example: the dimension of the embedding matrix is ​​[vocabulary size, embedding dimension], and the embedding dimension is 300, then each word will be converted into a 300-dimensional vector.

[0059] Step a24, adding a corresponding position vector to each vector of fixed dimension to obtain a target data set.

[0060] When dealing with sequential data (such as text), the order of words is an important information. To capture this sequential information, a position vector can be added to the embedding vector of each word. The position vector is a vector of the same dimension as the embedding vector, and its values ​​vary depending on the position of the word in the sentence. In this way, even if two words have the same embedding vector, their final representation will be different due to their different positions. For example: the dimension of the embedding vector is 300, then for the first word in the sentence, its position vector may be a 300-dimensional vector composed of specific sine and cosine function values. For the second word in the sentence, different sine and cosine function values ​​will be used.

[0061] Step a3, using the target data set to train the first model, and in the process of training, tuning the first model until the neural network prediction model reaches the target condition, thereby obtaining a pre-trained neural network model.

[0062] The target condition can be used to characterize the convergence of the loss function. Specifically, select a suitable neural network model according to the task requirements, such as a sequence-to-sequence (Seq2Seq) model, Transformer, etc. Use the target dataset to train the model and adjust the model parameters through the back-propagation algorithm. Improve the model performance by adjusting hyperparameters (such as learning rate, batch size, number of iterations, etc.), using regularization techniques, early stopping methods, etc. to prevent overfitting. Use the validation set to evaluate the model performance and adjust the model structure and parameters based on the evaluation results. For example: Select the Transformer model as the translation model and use the Adam optimizer for training. Use grid search to find the best learning rate of 0.001 and batch size of 32. During the training process, use the validation set to monitor the model performance and stop training when the loss on the validation set no longer decreases.

[0063] The method for determining the translation result provided in this embodiment uses historical online car-hailing multilingual service text data as a training data set, which covers service texts and translation labels in different languages, different expressions and different themes. This diversity of data enables the trained model to understand language phenomena more comprehensively and improve the accuracy and generalization ability of translation. In addition, by processing the training data set, rich features such as word frequency, word order, grammatical structure and semantic information of the service text are extracted. These features not only help the model to better capture the inherent laws and patterns of the language, but also improve the model's ability to understand complex contexts and expressions. During the training process, the first model is continuously tuned, including adjusting model parameters, optimizing algorithm selection, etc., to ensure that the model gradually approaches the target conditions during the training process. This tuning process can significantly improve the predictive performance and generalization ability of the model, making it more reliable and effective in practical applications.

[0064] In an optional embodiment, the above method further includes:

[0065] Step b1, the translation result is displayed on the user terminal in a voice prompt or in text.

[0066] The user terminal can be a user's mobile phone, tablet computer, etc., which is not specifically limited here. Specifically, after completing the translation task, the translation result is presented in a user-perceivable way. This can be achieved in two ways: one is to convert the translation result into speech through speech synthesis technology, and play it through the speaker of the user terminal to achieve voice prompts; the other is to display the translation result directly on the screen of the user terminal in text form for the user to read, etc. For example: when using the translation application, the user chooses to translate the English sentence "Hello, how are you?" into Chinese. After the translation is completed, the application plays "Hello, how are you?" through voice, and displays "Hello, how are you?" on the screen at the same time.

[0067] Step b2: When the translation result is displayed as text on the user terminal, in response to the user's triggering operation, perform a processing operation on the translation result; wherein, the processing operation includes at least one of: a copy operation, a sharing operation, and an editing operation.

[0068] The processing operation is an operation generated by the user through mouse clicking, keyboard selection, or touch screen. When the user issues a processing operation, correspondingly, the computer device can respond to the processing operation issued by the user and perform a processing operation on the translation result. Among them, when the translation result is displayed as text on the user terminal, corresponding processing operations are executed according to the user's triggering operation, and these operations include, but are not limited to, copying the translation result to the clipboard, sharing the translation result with others or applications, editing the translation result to modify the content therein, etc. For example: After the user sees the translation result "Hello, how are you?", clicks the "Copy" button next to it, and the translation result is copied to the clipboard. The user long presses the translation result, a sharing menu pops up, and the user selects to share it with a friend via WeChat, and the translation result is sent to the friend's WeChat chat window. When the user clicks the "Edit" button next to the translation result or enters the editing mode, the translation result can be modified, such as adding punctuation marks, correcting translation errors, etc.

[0069] The method for determining the translation result provided in this embodiment enables the user to intuitively obtain the translation result through voice prompts or text displays, and can understand the service content without additional operations. This instant feedback mechanism enhances the user's interactive experience and makes the entire service process smoother and more efficient. By providing processing operations such as copying, sharing, and editing, the user can further process and utilize the translation result according to their own needs. This interaction method not only enhances the user's participation but also makes the translation result more practical and valuable.

[0070] In an alternative embodiment, the above step S102 includes:

[0071] Step c1: Perform noise removal, stop word removal, missing value processing, and outlier processing on the service text data to obtain the first data.

[0072] Noise: Interference information in the data, which is not conducive to data analysis. Stop words: Common words that need to be ignored in text processing. Missing values: Values that are not specified or missing in the dataset. Outliers: Values that are significantly different from other data. Specifically, preprocess the original service text data to improve data quality. Noise may include irrelevant characters, garbled codes, etc.; stop words refer to those words that frequently appear in the text but contribute little to the meaning of the text, such as "de" (的), "le" (了), etc.; missing values need to be filled or deleted; outliers refer to those values that deviate significantly from other data, which may be caused by input errors or abnormal events.

[0073] Preferably, regular expressions are used to remove noise. A predefined stop word list is loaded for matching and removal. Missing values are filled (such as filling with mean, median, mode or specific values), or records containing missing values are directly deleted. Outliers are identified and processed, such as by statistical methods (such as the IQR method) or model-based methods (such as Isolation Forest) for detection and processing. For example: Removing HTML tags and special symbols in the text as noise processing. Removing stop words such as "of" and "already" from the text. Filling the missing numerical fields with the mean of the field. Identifying and deleting outliers that deviate significantly from other data.

[0074] Step c2, perform case conversion on the first data to obtain second data; wherein, the second data is used to represent data with the same case.

[0075] Convert all characters in the text data to uppercase or lowercase to eliminate text inconsistencies caused by case differences. For example: Convert the text "Hello,World!" to all lowercase "hello,world!".

[0076] Step c3, perform digital conversion on the second data to obtain third data; wherein, the third data is used to represent data with the same format.

[0077] Convert the content in the second data in specific formats such as numbers, dates, and times to a unified digital format for subsequent processing. In addition, it may also include mapping certain words or phrases in the text to numerical IDs, such as through hashing or lookup tables. For example: Convert the date "2023-04-01" in the text to a unified digital format (such as a timestamp).

[0078] Step c4, perform lexical normalization on the third data to obtain fourth data.

[0079] Convert the words in the text to a unified form to eliminate inconsistencies caused by spelling mistakes, abbreviations, inflections, etc. This may include operations such as stemming and lemmatization. Specifically, it can be achieved by using the stemmer or lemmatizer in natural language processing libraries (such as NLTK, spaCy), or by other means, which are not specifically limited here and shall be determined by those skilled in the art. For example: Through lemmatization, words such as "running", "ran", and "run" are processed into a unified form "run".

[0080] Step c5, convert the fourth data into numerical form data, and determine the feature data that meets the preset conditions from the numerical form data.

[0081] The text data (or data that has been converted to digital format) is further converted into numerical features, which can be used in machine learning models. At the same time, according to business needs or data characteristics, feature data that meets specific conditions can be filtered out.

[0082] Step c6, scaling the characteristic data so that each characteristic data is within the same preset value range.

[0083] During the data preprocessing stage, especially when the dataset contains multiple features and the numerical ranges of these features vary greatly, direct machine learning or data analysis may cause some features to be overemphasized or neglected during model training. Therefore, the purpose of this step is to scale all feature data to a uniform numerical range, usually [0,1] or [-1,1], to ensure that each feature contributes fairly to model training. Commonly used scaling methods include Min-Max Scaling and Z-score normalization. For example: if there is a feature data range of [100,1000], after using Min-Max normalization, the range of the feature data will become [0,1].

[0084] Step c7, combining various feature data to obtain target service text data.

[0085] Combine the processed feature data (including scaled numerical data) into a complete data set, namely the target service text data. This data set will be used for subsequent data analysis, machine learning model training and other tasks. The combination method may be to arrange multiple feature columns in a certain order into a two-dimensional array (or DataFrame), where each row represents a sample and each column represents a feature. For example: there are three feature data: age (scaled to [0,1]), gender (encoded as 0 and 1), and annual income (scaled to [0,1]). Combine these feature data into a DataFrame, where each row represents a person's information and each column represents age, gender and annual income. This DataFrame is the target service text data and can be used for subsequent analysis or model training.

[0086] The method for determining the translation result provided in the present embodiment can significantly improve the quality of data by removing noise, stop words, and processing missing values ​​and outliers. These steps help to reduce irrelevant information and errors in the data, making subsequent processing more accurate and efficient. The format consistency of the data is ensured by case conversion and digital conversion steps. This helps to eliminate processing errors caused by format differences, so that the model can understand and process data more accurately. The vocabulary in the text data is converted into a unified form through vocabulary normalization, which helps to reduce the impact of vocabulary diversity on model training. At the same time, the text data is converted into data in numerical form, so that the model can process and analyze data more conveniently. By determining the feature data that meets the preset conditions from the data in numerical form and scaling it, the most critical features for model training can be extracted. These features can more accurately reflect the inherent laws and patterns of the text data, thereby improving the prediction performance of the model. The feature data is combined and scaled so that each feature data is within the same preset numerical range. This helps to reduce the amount of calculation during model training and improve training efficiency. At the same time, due to the consistent dimensions between the feature data, the model can converge to the optimal solution more quickly. By preprocessing the service text data comprehensively, the trained model can be made more robust and generalized. The model can better adapt to different types of service text data and improve its accuracy and reliability in practical applications.

[0087] In an optional embodiment, the above method further includes:

[0088] Step d1, obtain the test data set.

[0089] Prepare a test data set that is independent of the training data set. This test data set is used to evaluate the performance of the neural network model on unseen data to ensure that the model has generalization capabilities. The test data set should be similar to the training data set in terms of data distribution, feature space, etc., but should not contain any samples in the training data set. Among them, the test data set may include: historical online car-hailing multilingual service text data including service texts and translation labels in different languages, different expressions and different themes. Specifically, a part of the existing data set can be divided as a test data set to ensure that this part of the data has no intersection with the training data set. It is also possible to specially collect or generate new data that is similar to but completely different from the training data set as a test data set, etc., which is not specifically limited here and can be implemented by technical personnel in this field.

[0090] Step d2, input the test data set into the pre-trained neural network model to obtain the test results.

[0091] Each sample in the test data set is input into the pre-trained neural network model in turn, and the model calculates and outputs the prediction results based on the input data. These prediction results will be used for subsequent performance evaluation.

[0092] Step d3, checking whether the test results meet the preset requirements.

[0093] Preset requirements can be used to characterize the accuracy of evaluation results. Specifically, according to business needs or evaluation criteria, a series of preset requirements (such as accuracy, etc.) are set, and the test results are compared with these requirements. The purpose of this step is to evaluate whether the performance of the neural network model on the test data set meets expectations.

[0094] Step d4: If the test result meets the preset requirements, the pre-trained neural network model is determined to be qualified.

[0095] If the test results meet all preset requirements, the neural network model is considered to perform well on the test data set and has sufficient generalization ability, and can be judged as a qualified model.

[0096] Step d5: If the test result does not meet the preset requirements, the pre-trained neural network model is determined to be unqualified.

[0097] If the test results do not meet the preset requirements, it is considered that the neural network model performs poorly on the test data set and may have overfitting, underfitting or other problems, requiring further optimization or retraining, and is therefore judged to be an unqualified model.

[0098] In the method for determining the translation result provided in this embodiment, the test data set is a sample set independent of the training data set, which can objectively evaluate the performance of the neural network model. By inputting the test data set into the model and observing the output results, the performance of the model in practical applications can be intuitively understood, thereby verifying whether it meets the expected requirements.

[0099] Combination Figure 2 , Figure 3 and Figure 4 As shown, in an optional implementation, the method for determining the translation result may further include:

[0100] Step 1: Randomly select a large amount of real online car-hailing multilingual service text data from the online database, including service texts in different languages, different expressions and different topics and their corresponding translation labels.

[0101] Step 2: Preprocess and feature engineer the data to generate training sets, test sets, and validation sets; feature description and examples: including word frequency, word order, grammatical structure, semantic information, etc. of the service text.

[0102] Step 3, repeating steps 1 and 2, and generating multiple sets of service text data (ie, the above-mentioned training data sets) based on the processed data.

[0103] Step 4: Use the self-attention mechanism (transformer) model for model training. This task can be regarded as a kind of seq2seq model. The specific steps are as follows:

[0104] Step 4.1, Data Preparation: First, you need to prepare a fine-tuning dataset. This dataset is cleaned and preprocessed so that the model can learn from it. Tokenize the text and convert it into a format that the transformer model can understand. This usually involves splitting the text into words or subwords (tokens).

[0105] Step 4.2, word embedding: As with common NLP tasks, we first use a word embedding algorithm to convert each word in the input text sequence into a word vector. The vectors in practical applications are generally 256 or 512 dimensions.

[0106] Step 4.3, Position Embedding: The Transformer model adds a position vector to each input word vector. These vectors help determine the position characteristics of each word, or the distance characteristics between different words in a sentence. The intuition behind adding position vectors to word vectors is that adding these position vectors to word vectors can provide more meaningful information to the model, such as the position of words, the distance between words, etc.

[0107] Step 4.4, Loss Function: During fine-tuning, you need to define a loss function to measure the difference between the model's prediction and the actual label. The loss function is usually the cross entropy loss.

[0108] Step 4.4.1, load the training set and test set, generate a data iterator for step-by-step calling of the iterative model.

[0109] Step 4.4.2, initialize the model and related hyperparameters, including the input layer, multi-layer bidirectional Transformer encoder layer, decoder layer, output layer, loss function, number of training rounds and the size of each training data.

[0110] Step 4.4.3, start training until the loss function converges and generate a deep learning model.

[0111] Step 5: Read the validation set data and input the data into the model of step 4 to verify the effect of step 4 on the validation set.

[0112] Step 6: Repeat steps 3, 4, and 5 to generate the optimal deep learning model (that is, the pre-trained neural network model mentioned above). The evaluation indicators of the model are mainly:

[0113] Cross-Entropy Loss is a commonly used loss function that measures the difference between the predicted distribution and the true distribution of the model in classification problems. It is based on the concept of cross entropy in information theory and is used to quantify the difference between two probability distributions.

[0114] Step 7: preprocess and feature engineer the real-time service text data;

[0115] Step 8: The real-time data from step 7 (i.e. Figure 2 The sentences to be translated and the target language in the text are input into the model (the model identifies the sentences to be translated and the target language, obtains the original language, and translates it through the online car-hailing exclusive translation model), and obtains the translation calculation results based on each service text (the translated sentences). The main online evaluation indicators are: the evaluation results are given in real time using the evaluation indicators in step 5 above; the output of the model is the translation calculation results of each service text.

[0116] Step 9, apply the model: Feedback the model's output results to the downstream in real time, such as confirming the translation of service texts through system prompts, providing multi-language service support, etc.; provide real-time multi-language translation capabilities for other systems on the platform, and build a diversified platform system.

[0117] In the present embodiment, a device for determining a translation result is also provided, and the device is used to implement the above-mentioned embodiment and the preferred implementation mode, and the description thereof will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and contemplated.

[0118] This embodiment provides a translation result determination device, such as Figure 3 As shown, it includes: an acquisition module 301, which is used to obtain the initial service text data of the online car-hailing service. A first processing module 302, which is used to pre-process and feature-engineer the service text data to obtain target service text data. A second processing module 303, which is used to process the target service text data using a pre-trained neural network model to obtain the translation result corresponding to the initial service text data.

[0119] In an optional embodiment, the above-mentioned device also includes: an acquisition processing module, used to acquire a training data set; wherein the training data set includes historical online car-hailing multilingual service text data, and the historical online car-hailing multilingual service text data includes service texts and translation labels in different languages, different expressions and different themes; processing the training data set to obtain a target data set; wherein the target data set includes: word frequency, word order, grammatical structure and semantic information of the service text; using the target data set to train the first model, and during the training process, tuning the first model until the neural network prediction model meets the target conditions, thereby obtaining a pre-trained neural network model.

[0120] In an optional embodiment, a processing module is obtained to preprocess the training data set to obtain first target data; perform text tokenization on the first target data to obtain second target data and an authentication identifier corresponding to the second target data; wherein the second target data is a word or a term; use an embedding matrix to convert the authentication identifier into a vector of fixed dimension; and add a corresponding position vector to each vector of fixed dimension to obtain a target data set.

[0121] In an optional embodiment, the above-mentioned device also includes: a display processing module, which is used to give a voice prompt or display the translation result in text on the user terminal; when the translation result is displayed in text on the user terminal, in response to the user's trigger operation, the translation result is processed; wherein the processing operation includes: at least one of: a copy operation, a sharing operation, and an editing operation.

[0122] In an optional embodiment, the first processing module 302 is used to remove noise, stop words, missing values ​​and outliers from the service text data to obtain first data; convert the first data into uppercase and lowercase to obtain second data; wherein the second data is used to represent data with the same text size; perform digital conversion on the second data to obtain third data; wherein the third data is used to represent data with the same format; perform vocabulary normalization on the third data to obtain fourth data; convert the fourth data into data in numerical form, and determine feature data that meets preset conditions from the data in numerical form; scale the feature data so that each feature data is within the same preset numerical range; and combine each feature data to obtain target service text data.

[0123] In an optional embodiment, the above-mentioned device also includes: obtaining a verification module, which is used to obtain a test data set; inputting the test data set into a pre-trained neural network model to obtain a test result; detecting whether the test result meets the preset requirements; if the test result meets the preset requirements, the pre-trained neural network model is determined to be qualified; if the test result does not meet the preset requirements, the pre-trained neural network model is determined to be unqualified.

[0124] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0125] The translation result determination device in this embodiment is presented in the form of a functional unit, where the functional unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0126] The embodiment of the present invention also provides a computer device having the above Figure 3 The translation result is determined by the device shown.

[0127] See also Figure 4 , Figure 4 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 4 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0128] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0129] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0130] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0131] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0132] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0133] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0134] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0135] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for determining a translation result, characterized in that: The method comprises: Obtain the initial service text data of the online car-hailing service; Preprocessing and feature engineering the service text data to obtain target service text data; The target service text data is processed using a pre-trained neural network model to obtain a translation result corresponding to the initial service text data.

2. The method for determining a translation result according to claim 1, characterized in that: Ways to build pre-trained neural network models, including: Obtain a training data set; wherein the training data set includes historical online car-hailing multilingual service text data, and the historical online car-hailing multilingual service text data includes service texts and translation labels in different languages, different expressions and different themes; Processing the training data set to obtain a target data set; wherein the target data set includes: word frequency, word order, grammatical structure and semantic information of the service text; The target data set is used to perform model training on the first model, and during the training process, the first model is tuned until the neural network prediction model meets the target conditions, thereby obtaining a pre-trained neural network model.

3. The method for determining the translation result according to claim 2, characterized in that: The processing of the training data set to obtain a target data set includes: Preprocessing the training data set to obtain first target data; Performing text tokenization processing on the first target data to obtain second target data and an authentication identifier corresponding to the second target data; wherein the second target data is a word or a character; Converting the authentication identifier into a vector of fixed dimension using an embedding matrix; Add the corresponding position vector to each fixed-dimensional vector to obtain the target data set.

4. The method for determining a translation result according to claim 2, characterized in that: The method further comprises: Prompt the translation result in voice or display it in text on the user terminal; When the translation result is displayed on the user terminal in the form of text, in response to a trigger operation of the user, a processing operation is performed on the translation result; wherein the processing operation includes: at least one of a copy operation, a share operation, and an edit operation.

5. The method for determining a translation result according to claim 1, characterized in that: The service text data is preprocessed and feature-engineered to obtain target service text data, including: Remove noise, stop words, missing values ​​and outliers from the service text data to obtain first data; Convert the first data into uppercase and lowercase data to obtain second data; wherein the second data is used to represent data with the same character size; Performing digital conversion on the second data to obtain third data; wherein the third data is used to represent data with the same format; Performing vocabulary normalization processing on the third data to obtain fourth data; Converting the fourth data into data in numerical form, and determining characteristic data satisfying a preset condition from the data in numerical form; Scaling the characteristic data so that each of the characteristic data is within the same preset value range; The target service text data is obtained by combining the characteristic data.

6. The method for determining a translation result according to claim 1, characterized in that: The method further comprises: Get the test dataset; Inputting the test data set into a pre-trained neural network model to obtain a test result; Detecting whether the test results meet preset requirements; If the test result meets the preset requirement, the pre-trained neural network model is determined to be qualified; If the test result does not meet the preset requirement, the pre-trained neural network model is determined to be unqualified.

7. A device for determining a translation result, characterized in that: The device comprises: An acquisition module, used to acquire initial service text data of an online car-hailing service; A first processing module, used for performing preprocessing and feature engineering on the service text data to obtain target service text data; The second processing module is used to process the target service text data using a pre-trained neural network model to obtain a translation result corresponding to the initial service text data.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for determining the translation result according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for determining a translation result according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to cause a computer to execute the method for determining a translation result according to any one of claims 1 to 6.

Images