Multi-modal translation method in cross-border e-commerce scene

Abstract

The invention provides a multi-modal translation method in a cross-border e-commerce scene, which comprises the following steps: acquiring e-commerce platform page content, identifying commodity links, classification links and activity links from the page content, and extracting absolute paths, relative paths and dynamic parameter information of the links; aiming at the identified link, analyzing the context of the link by adopting a natural language processing technology, and judging whether a target page type pointed by the link is a commodity detail page, a commodity list page or an activity special topic page; according to the judged target page type, obtaining a corresponding page address linked in a target language version from a pre-established multi-language page mapping table, and generating link mapping relation data; according to the generated link mapping relation data, processing rules of different types of links are formulated, the processing rules are adopted to conduct batch automatic conversion on the links, and target language version links are obtained.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a multimodal translation method in a cross-border e-commerce scenario. Background Art

[0002] In the cross-border e-commerce scenario, multimodal translation methods face the technical challenge of how to handle various links in the pages of e-commerce platforms. These links include product links, category links, and activity links. They are often deeply embedded in various parts of the page and are tightly coupled with other elements of the page. When performing multimodal translation on a page, it is necessary not only to translate the text content in the page, but also to ensure that these links are still valid after translation and can correctly jump to the corresponding page of the target language version. The processing of links involves considerations at multiple technical levels. First, there is an intricate correspondence between pages in different language versions. A source language page may correspond to multiple target language pages, and vice versa. Second, links come in a variety of forms, which may be absolute paths, relative paths, and may also contain dynamic parameters. Furthermore, links may change over time. For example, a link to a product may become invalid due to the removal of the product. These factors have brought difficulties to the processing of links in the multimodal translation process. If links are not handled properly, a large number of dead links or wrong links may appear in the translated page, seriously affecting the user experience. At the same time, the efficiency of link processing also directly affects the performance of the entire translation system. How to ensure the correctness of links while efficiently replacing links, achieve accurate link processing, and provide users with a smooth cross-language browsing experience is a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present invention provides a multimodal translation method in a cross-border e-commerce scenario, which mainly includes:

[0004] Obtain the page content of the e-commerce platform, identify product links, category links and activity links from the page content, and extract the absolute path, relative path and dynamic parameter information of the link; for the identified links, use natural language processing technology to analyze the link context and determine whether the target page type pointed to by the link is a product details page, a product list page or an activity topic page; according to the determined target page type, obtain the corresponding page address of the link in the target language version from the pre-established multi-language page mapping table, and generate link mapping relationship data; for the generated link mapping relationship data, formulate processing rules for different types of links, use the processing rules to automatically convert the links in batches, and obtain the target language version links; monitor the validity of the obtained target language version links, and if an invalid link is detected, analyze the user behavior log and product information update record, and map the invalid link to the latest valid link; use automated testing tools to simulate user click behavior, perform access tests on the mapped links, check whether they jump to the target page correctly, and determine whether they are the correct target language version pages based on the similarity of the page content.

[0005] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:

[0006] The invention discloses a multimodal translation method in a cross-border e-commerce scenario. The method first identifies commodities, categories and active links from the content of an e-commerce platform page, and extracts the path and parameter information of the link. Then, a natural language processing technology is used to analyze the link context and determine the type of target page pointed to by the link. According to the judgment result, the corresponding page address of the link in the target language version is obtained from a pre-established multilingual page mapping table, and link mapping relationship data is generated. Then, processing rules for different types of links are formulated, and the links are automatically converted in batches to obtain target language version links. The validity of the converted links is monitored. If an invalid link is detected, the user behavior log and the commodity information update record are analyzed, and the invalid link is mapped to the latest valid link. Finally, an automated testing tool is used to simulate the user's click behavior, and an access test is performed on the mapped link to check whether the jump is correct and determine the similarity of the page content. The invention realizes the cross-language automatic mapping and validity maintenance of the e-commerce platform link, and improves the user experience of the multilingual version website. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 The present invention is a flowchart of a multimodal translation method in a cross-border e-commerce scenario.

[0008] Figure 2 A schematic diagram of a multimodal translation method in a cross-border e-commerce scenario of the present invention.

[0009] Figure 3This is another schematic diagram of a multimodal translation method in a cross-border e-commerce scenario of the present invention. DETAILED DESCRIPTION

[0010] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0011] like Figure 1-3 In this embodiment, a multimodal translation method in a cross-border e-commerce scenario may specifically include:

[0012] Step S101, obtain the e-commerce platform page content, identify product links, category links and activity links from the page content, and extract the absolute path, relative path and dynamic parameter information of the link.

[0013] Obtain the page content of the e-commerce platform and parse the page content into a DOM tree structure; identify the corresponding link nodes from the DOM tree according to the pre-defined feature rules of product links, category links and active links; for each identified link node, determine whether it is an absolute path or a relative path; if the link node starts with "http: / / " or "https: / / ", it is determined to be an absolute path; otherwise, it is determined to be a relative path; for the link node determined to be a relative path, convert the relative path to an absolute path according to the URL of the current page; from the URL of the link node, match and extract the dynamic parameter part through a pre-defined regular expression pattern to obtain the parameter name and parameter value; store the absolute path, relative path and dynamic parameter information of the extracted link node in a structured manner to build a link information database; the table structure of the link information database includes link ID, link URL, link type and dynamic parameter fields; According to the manually annotated training data set, the features of the link nodes are extracted, and one of the naive Bayes or decision tree algorithms is trained to obtain a link classification model; the features of the newly collected link nodes are extracted, and the extracted features are input into the link classification model to obtain the classification results of the link nodes; the classification results are stored in the link information database to improve the metadata information of the link nodes; based on the link information database, link analysis and application are performed.

[0014] Exemplarily, taking a well-known cross-border e-commerce platform as an example, the crawler program can simulate a browser to access the home page, obtain the HTML source code, and then use a parsing library such as Beautiful Soup to convert it into a DOM tree. This can conveniently extract various elements on the page. Identifying link nodes according to predefined rules is the key. For example, product links usually contain keywords such as "product" or "item", category links may have the word "category", and activity links often contain "activity" or "promotion". Target links can be quickly located through these features. It is important to determine whether the link is an absolute path. Taking "https: / / www.example.com / product / 12345" as an example, it starts with "https: / / " and is an absolute path. While " / category / electronics" is a relative path and needs to be concatenated with the current page URL to form a complete link. This can ensure that a complete and valid URL is used in subsequent processing. Extracting dynamic parameters from the link can obtain more information. For example, in "https: / / www.example.com / search?keyword=手机&page=2", the two parameters "keyword" and "page" and their values can be extracted. These parameters often contain important business meanings, such as search keywords, page numbers, etc. Storing the extracted link information in a database is convenient for subsequent use. A table named "links" can be designed, containing fields such as id, url, type, params, etc. This can efficiently store and query a large amount of link data and provide a basis for subsequent analysis. Using machine learning algorithms to classify links is an effective method to improve accuracy. A batch of training data can be manually labeled. For example, " / product / 12345" is labeled as a product link, " / category / electronics" is labeled as a category link, etc. Then extract the features of the link, such as URL length, keyword occurrence times, etc., and train a machine learning algorithm such as the Naive Bayes or Decision Tree model to obtain a link classification model. This can automatically classify new unknown links. Analyzing based on the link database can obtain a lot of valuable information. For example, by analyzing the quantity and distribution of product links, the product structure of the platform can be understood; by tracking the changes of activity links, the platform's promotion strategy can be grasped; by analyzing the link sequence accessed by users, the browsing path and interest preferences of users can be mined. These analysis results can provide more auxiliary information for multimodal translation in the cross-border e-commerce web page scenario, thus obtaining a more accurate and refined translation effect.

[0015] Step S102: for the identified link, natural language processing technology is used to analyze the link context to determine whether the target page type pointed to by the link is a product details page, a product list page or an activity topic page.

[0016] Get the target web page link to be analyzed, and get the HTML content of the target web page through the requests library; pre-process the HTML content, use regular expressions to remove JavaScript and CSS noise, use the jieba library for Chinese word segmentation, and use the THULAC tool for part-of-speech tagging; use the TextRank algorithm to extract keywords and key phrases from the pre-processed page content; obtain the feature word library of product details page, product list page and event topic page pre-built using TF-IDF; calculate the cosine similarity between the target page content and the feature words of the three types of pages, and determine the page type with the highest similarity as the type of the target page; if the target page is determined to be a product details page, use Regular expressions are used to match HTML tags and their contents corresponding to the key attributes of product name, price, and specification parameters, and structured product information fields are extracted. If the target page is judged to be a product list page, XPath is used to parse the HTML node tree to obtain the link, thumbnail URL, name, and price information of each product in the product list, and the steps are recursively called for each product link to obtain product information. If the target page is judged to be an event topic page, regular expressions are used to match the HTML content corresponding to the key attributes of event theme, time, and participation method, and structured event information fields are extracted. The parsed structured product information or event information is stored in the MongoDB database with URL as the primary key, and the page type is recorded at the same time.

[0017] For example, taking a well-known cross-border e-commerce platform as an example, you can use Python's requests library to send HTTP requests to obtain the HTML content of the target page. For example, for a product details page, you may get raw content containing a large number of JavaScript, CSS, and HTML tags. In order to extract useful information, you need to preprocess the HTML content. Using regular expressions can effectively remove noise such as JavaScript and CSS. For example, you can use the pattern " <script>.*?< / script> "and" <style>。*?< / style>" to match and delete these contents. The preprocessed text is clearer and easier to analyze later. Chinese word segmentation is a key step in understanding page content. The jieba library is an excellent Chinese word segmentation tool that can accurately segment continuous Chinese text into meaningful words. For example, "Huawei's latest smartphone" may be segmented into "Huawei / latest / smartphone". This word segmentation result lays the foundation for subsequent text analysis. Part-of-speech tagging further enriches the linguistic information of the text. The THULAC tool can tag parts of speech for each word segmentation result, such as nouns, verbs, adjectives, etc. This is crucial to understanding the role of words in sentences. For example, in "Red Limited Edition Mobile Phone", "red" and "limited edition" will be marked as adjectives, while "mobile phone" will be marked as a noun. The TextRank algorithm is a keyword extraction method based on a graph model. method. It regards the words in the text as nodes in the graph, and the co-occurrence relationship between words as edges. Through iterative calculation, the importance score of each word can be obtained. For example, in a page describing a smartphone, words such as "processor", "camera", and "battery life" may be extracted as keywords. Page type identification is an important part of e-commerce data analysis. Through the pre-built feature vocabulary, the similarity between the target page and pages of different types can be calculated. For example, a product details page may contain feature words such as "specifications", "parameters", and "add to shopping cart", while a product list page may contain words such as "sort", "filter", and "paging". By calculating the cosine similarity, the page type can be accurately determined. For product details pages, structured product information needs to be extracted. Regular expressions can be used to accurately match product attributes in HTML tags. For example, the pattern "<h1.*?> (.*?)" to extract the product name,<spanclass=“price”> (.*?)" to extract the price. This structured information provides the basis for subsequent product analysis and comparison. Parsing the product list page requires more complex HTML node traversal. XPath can be used to locate elements in HTML documents. For example, the XPath expression " / / div[@class='product-item']" can be used to select all product items, and then extract the link, picture, name, price and other information of each product separately. Event theme pages usually contain rich marketing information. Regular expressions can be used to extract key attributes of the event, such as "Event time: (\d{4}-\d{2}-\d{2})" can match the event date. This information is crucial for analyzing marketing strategies and effects. Finally, the extracted structured data is stored in the MongoDB database. MongoDB's document storage feature is very suitable for storing complex product information. For example, a product document may contain fields such as "name", "price", and "specifications".Using URL as the primary key can ensure the uniqueness and traceability of data. This systematic data collection and analysis process provides rich data support for e-commerce platforms. For example, by analyzing the keywords of the product details page, search engines can be optimized; by comparing the prices and specifications of different products, intelligent recommendations can be achieved; by tracking the visits and conversion rates of the activity pages, marketing effects can be evaluated. Based on these data, the results of multimodal translation can be corrected to improve the accuracy of translation, so that the translated web page content is more in line with the user's browsing habits, providing users with a smooth cross-language browsing experience.

[0018] Step S103, according to the determined target page type, the corresponding page address of the link in the target language version is obtained from a pre-established multi-language page mapping table to generate link mapping relationship data.

[0019] Get the URL address of the link to be processed, and use regular expression matching to determine the target page type of the link; for the determined target page type, obtain the pre-established multi-language page mapping table, which records the URL addresses of the corresponding pages in different language versions; from the multi-language page mapping table, obtain the corresponding page URL address that matches the target page type and target language version; if the corresponding page URL address is found in the multi-language page mapping table, associate the URL address of the link with the obtained corresponding page URL address to generate link mapping relationship data; otherwise, record the unmatched link URL address for subsequent manual processing. Use the jieba word segmentation tool to segment the anchor text of the link, calculate the importance score of each word based on the TF-IDF algorithm, and select the words with the highest scores as keywords; use the extracted keywords as attribute fields of the link mapping relationship data and store them together with the URL address. According to the similarity between the URL address and the anchor text of the link, the K-means clustering algorithm is used to group the link mapping relationship data; the edit distance between the URL addresses and the jaccard similarity between the anchor text keywords are calculated, and the data with high similarity are divided into the same cluster to obtain the link mapping relationship data set of different topics. The generated link mapping relationship data is stored in the pre-established MySQL relational database. The database table structure includes the link URL, target page type, target language version, corresponding page URL, and anchor text keyword fields; the application realizes the link jump of multi-language web pages by querying the database; at the same time, the stored multi-language link mapping relationship is used to machine translate the web page content to generate web pages in different language versions.

[0020] Exemplarily, first, the target page type is determined by matching the URL address with a regular expression. For example, for a product detail page, the pattern " / product / \d+" may be used to match; while for a product list page, " / category / \w+" may be used. This method can quickly and accurately identify different types of pages. After determining the page type, the system will query a pre-established multilingual page mapping table. This mapping table records the URL addresses of corresponding pages in different language versions. For example, the Chinese version of the "Mobile Phone" category page may be " / category / shouji", while the English version is " / category / mobile-phones". Through this mapping relationship, the system can establish accurate link correspondence between different language versions. If a matching URL is found in the mapping table, the system will generate link mapping relationship data. For links that cannot find a corresponding relationship in the mapping table, record them and perform manual matching processing later. This may occur on newly launched pages or special marketing campaign pages. By regularly reviewing these unmatched links, the mapping table can be continuously improved to improve the coverage of the system. This process not only includes the correspondence of URLs, but also involves the processing of anchor texts. Using the Jieba word segmentation tool to segment anchor text can effectively process Chinese text. For example, "latest smartphone" may be segmented into "latest / smart / mobile phone". Then, the word importance is calculated by the TF-IDF algorithm. In this example, "smart" and "mobile phone" appear more frequently and may get higher scores because they better represent the core content of the link. The extracted keywords are stored together with the URL address to form rich link mapping relationship data. This data is not only used for multilingual jumps, but also for content analysis and search optimization. For example, by analyzing keywords, you can understand the product features that users are most concerned about, so as to optimize product descriptions and marketing strategies. In order to better organize and manage this data, the system uses the K-means clustering algorithm to group the link mapping relationship data. This process takes into account the edit distance of the URL address and the Jaccard similarity of the anchor text keywords. For example, all links about "smartphones" may be clustered together, while links about "laptops" will form another cluster. This clustering method helps to discover potential topic structures, which is of great value for content organization and recommendation systems. Finally, these processed and clustered link mapping relationship data are stored in a MySQL database. The database table structure includes fields such as link URL, target page type, target language version, corresponding page URL and anchor text keywords. This structured storage is not only convenient for query and management, but also provides a basis for subsequent data analysis and application. Through this system, e-commerce platforms can achieve seamless multi-language web page jumps.For example, when a user switches from the Chinese version to the English version, the system can automatically map the current page to the corresponding English page to ensure the consistency of the user experience. In addition, this system also provides a basis for machine translation. Through the stored multilingual link mapping relationship, the system can more accurately translate web page content and generate high-quality multilingual versions. This not only improves translation efficiency, but also ensures the accuracy of professional terms and brand names, which is crucial for maintaining brand image and providing accurate information.

[0021] Step S104, formulating processing rules for different types of links for the generated link mapping relationship data, and using the processing rules to automatically convert the links in batches to obtain target language version links.

[0022] The link data of the source language and the target language are obtained, and a link mapping relationship model is established using a decision tree algorithm; the link mapping relationship model is used to predict the full amount of link data to obtain preliminary link mapping relationship data; corresponding processing rules are formulated for different link types in the link mapping relationship data, and the processing rules include the structure conversion rules of the link and the translation rules of the page content corresponding to the link; the source language links are batch processed using a distributed computing framework, and each link is structurally converted according to the processing rules, and the machine translation API is called to translate the page content into the target language to obtain the link data of the target language version; the link data of the target language version is quality tested, and the links are automatically checked using predefined quality assessment rules, and links with problems are identified by manual sampling; for the problematic links found in the quality test, the link conversion rules and machine translation parameters are optimized using a heuristic algorithm, and the link conversion processing is re-performed, and multiple rounds of iterations are performed until the preset quality standards are met; the optimized link conversion configuration is applied to the full amount of link data to generate the final target language version link, and the link data is pushed to the multilingual website system through the Web service API.

[0023] Exemplarily, first, obtain the link data of the source language and the target language, including information such as URL, page title, and keywords. For example, for an e-commerce website, the URL of the product page in the source language (Chinese) may be "https: / / example.com / zh / products / smartphone", while the corresponding URL in the target language (English) may be "https: / / example.com / en / products / smartphone". Next, analyze the structural features and content attributes of the link. Structural features include URL pattern and path depth. Taking the above example, it can be found that the URL pattern is " / language code / products / product name" and the path depth is 3. Content attributes include page title, description, and keywords. For example, the title of the Chinese page may be "Smartphone", while the title of the English page is "Smartphone". The structural features and content attributes of the link are used as input features, and the corresponding relationship of the link is used as a label. A model that can predict the link mapping relationship is obtained by training using a decision tree algorithm. This model will learn how to map the URL of the source language to the URL of the target language. The training data can be a part of the link correspondence that is manually annotated. After the model training is completed, it can be used to predict the mapping relationship of the entire link data. For different types of links, corresponding processing rules need to be formulated. For example, for product pages, the product name in the URL may need to be translated from Chinese to English. For category pages, the entire path structure may need to be adjusted. At the same time, the translation of page content, including titles, text, and keywords, needs to be considered. Using distributed computing frameworks such as MapReduce can efficiently process large amounts of link data. During the processing, the link structure will be converted according to the previously formulated rules, and machine translation APIs such as Google Translate API will be called to translate the page content. In this way, the link data of the target language version can be generated. In order to ensure the quality of the generated links, quality testing is required. This includes automatic checking of link validity, content integrity, etc., as well as manual spot checks of some links. For example, you can check whether the generated English URL can be accessed normally, whether the page content is complete, and whether the translation is accurate. For links with problems, heuristic algorithms can be used for optimization. Obtain the conversion rules and translation parameters of the machine translation system to be optimized as the initial input of the heuristic algorithm. Analyze the business content, extract key attributes, determine the correlation between attributes, and screen out attribute combinations with high mutual correlation. According to the attribute combination, one of the machine learning algorithms such as decision tree, support vector machine and neural network is used to build the attribute association model. The attribute association model is combined with the conversion rules and translation parameters, and the updated conversion rules and translation parameters are obtained through iterative optimization.On the basis of the updated conversion rules and translation parameters, the context logic and thinking chain are integrated to optimize the translation results and improve the coherence and accuracy of the translation. The optimized translation results are evaluated, and the BLEU value, semantic similarity and other indicators are calculated to determine whether the translation quality reaches the expected goal. If the translation quality does not reach the expected goal, the iterative optimization continues; if the translation quality has reached the expected goal, the optimized translation model is output to complete the optimization process of the heuristic algorithm. For example, if it is found that the translation of a certain type of product name is often wrong, the parameters of the translation engine can be adjusted or a special terminology dictionary can be used. Through multiple rounds of iterative optimization, the quality of link conversion can be continuously improved. Finally, the optimized link data is pushed to the multilingual website system, and the link relationship data is imported into the search engine database. This not only realizes the interconnection of web pages in different languages, but also ensures that users can find the corresponding language version of the page when searching. This whole process not only improves the user experience of multilingual websites, but also helps to improve the internationalization level of websites and the search engine optimization effect.

[0024] Step S105 , monitoring the effectiveness of the obtained target language version link. If an invalid link is detected, analyzing the user behavior log and the product information update record, and mapping the invalid link to the latest valid link.

[0025] Get all product detail page links in the target language version to form an initial link list; for each link in the initial link list, determine whether it can be accessed normally by sending an HTTP request; if the status code returned by the HTTP request is 200, mark the link as a valid link; otherwise, mark it as a broken link and record relevant information about the broken link; for each broken link, find out from the user behavior log of the business system whether there is a latest access record for the product ID and page ID corresponding to the broken link; if there is a latest access record, extract the link in the record as the potential latest valid link for the broken link; if the latest access record is not found in the user behavior log, find the latest update record of the product corresponding to the broken link in the product information update record, and extract the updated product detail page link as the potential latest valid link for the broken link; for For invalid links that still cannot find the latest valid links, use an item-based collaborative filtering algorithm based on the category and keyword features of the products to find the N products with the highest feature similarity from the link library, and obtain the detail page links of these N products as potential replacement links for the invalid links; filter out invalid links for all potential latest valid links and potential replacement links; establish a mapping relationship between the screened valid links and the corresponding invalid links to form a mapping table of invalid links to valid links; traverse all links on the page, and replace the invalid links with corresponding valid links in the mapping table; update the page after the link is replaced, and store the mapping table persistently in the database; remove the invalid links for which the corresponding valid links cannot be found from the page, and log the information of the invalid links that cannot be processed for regular manual review.

[0026] For example, first, obtain all product detail page links in the target language version. Take a cross-border e-commerce product website as an example. Its English version may have tens of thousands of product detail page links, such as "https: / / example.com / en / products / summer-dress-2023". Next, you need to verify the validity of these links. By sending an HTTP request and checking the returned status code, you can determine whether the link is accessible. For example, for the above link, if the status code 200 is returned, it means that the link is valid; if 404 is returned, it means that the link is invalid, which may be due to product removal or page migration. For invalid links, you need to find alternatives. First, you can check the user behavior log. Suppose a user recently browsed the summer dress with ID 12345, but the corresponding old link is invalid. By analyzing the log, you may find that the user actually visited a new link "https: / / example.com / en / products / summer-dress-2023-updated". This new link is likely to be the latest valid link for the product. If you cannot find relevant information in the user behavior log, you can turn to the product information update record. For example, you may find that the product with ID 12345 has been recently updated, and the new detail page link has become "https: / / example.com / en / products / summer-dress-2023-new-collection". This information can help find the latest valid link. If you still cannot find an alternative link, you can use an item-based collaborative filtering algorithm. Assuming that the invalid link is a red dress, you can find the most similar product in the link library based on its color, style, season and other characteristics. For example, you may find a similar pink dress with the link "https: / / example.com / en / products / pink-summer-dress-2023". This link can be used as a potential alternative option. After obtaining the potential alternative link, you need to verify its validity again. This step can help filter out links that may also be invalid, ensuring that the pages provided to users are ultimately accessible. Next, you need to establish a mapping relationship between invalid links and valid links. For example, you might get a mapping like this: {“https: / / example.com / en / products / summer-dress-2023”:“https: / / example.com / en / products / summer-dress-2023-new-collection”}. This mapping relationship will guide how to update the links in the web page.When updating page links, it is necessary to traverse all links in the page and replace them according to the mapping relationship. This process includes not only links to product detail pages, but may also include links in navigation bars, related recommendations, and other locations. The updated page will provide users with a more accurate browsing experience. For those broken links that cannot find replacement links, they need to be removed from the page. Doing so can prevent users from clicking on invalid links and improve the user experience. At the same time, the information of these unprocessable links should be recorded for subsequent manual review. This link management process can not only improve the usability of the website, but also help maintain the SEO effect of the website. By updating and managing links in a timely manner, the number of dead links encountered by search engine crawlers can be reduced, and the overall quality score of the website can be improved. In addition, this process can help to timely discover changes in product inventory or classification, providing valuable information for inventory management and product strategy.

[0027] Step S106, using an automated testing tool to simulate user click behavior, perform access testing on the mapped link, check whether it correctly jumps to the target page, and determine whether it is the correct target language version page based on the page content similarity.

[0028] Obtain all page links of the source language website, and generate the target language page link corresponding to each source language page link according to the predefined language mapping relationship. For each source language-target language link pair in the link mapping table, use the automated testing tool to simulate the user's click behavior, trigger the jump from the source language page to the target language page, and obtain the actual response target language page content. Extract the key content features of the response page, use the language detection tool to determine the page language type, and confirm whether it is the correct target language. Calculate the similarity between the text content of the response page and the text content of the expected target language page. Use the TF-IDF algorithm to convert the text of the two pages into a word frequency-inverse document frequency matrix, and calculate the cosine similarity of the two matrices to obtain a similarity score between 0 and 1. Set a predefined similarity threshold and judge the similarity score of each link pair. If the similarity is greater than the threshold, it is determined that the link jump is correct and the corresponding link mapping does not need to be adjusted; if the similarity is less than the threshold, it is determined that there is a problem with the link jump, and the link pair is recorded in the list to be optimized. For each source language-target language link pair in the list to be optimized, the latest content of the source language page is retrieved, and the source language page is translated into the target language through a machine translation interface to obtain a new target language page.

[0029] Exemplarily, first obtain all page links of the source language website, which can be achieved through a website crawler. Take a cross-border e-commerce website as an example. Assume that the Chinese version of the website has 100,000 product pages, and each page has a unique URL, such as "https: / / example.com / zh / products / summer-dress-2023". Next, generate the target language link according to the predefined language mapping relationship. For example, the mapping rule from Chinese to English may be to replace " / zh / " in the URL with " / en / ". In this way, the above Chinese link will be mapped to "https: / / example.com / en / products / summer-dress-2023". In this way, an initial link mapping table containing 100,000 pairs of Chinese and English links can be obtained. In order to verify the accuracy of these link mappings, it is necessary to simulate the actual browsing behavior of users. Using automated testing tools such as Selenium, you can write scripts to simulate users clicking the language switching button on the Chinese page, and then check whether the English page after the jump is as expected. This process can not only verify the validity of the link, but also detect whether the language switching function is working properly. After obtaining the target language page, it is necessary to extract the key content features of the page. This includes <title> The page title in the tag,< / title> <meta> Keywords in tags, <h1> arrive< / h1> <h6>The title content, and The paragraph text in the tag. These elements usually contain the core information of the page and can be used to determine the language and content of the page. Language detection is an important step to ensure that the page is translated correctly. Using tools such as Google Language Detector, the extracted text content can be analyzed to determine its language type. For example, if a page expected to be in English is detected as Chinese, this indicates that there may be a link mapping error or the page is not translated correctly. In order to more accurately evaluate the quality of the translation, it is necessary to calculate the text similarity between the source language page and the target language page. The TF-IDF (Term Frequency-Inverse Document Frequency) algorithm is an effective method. It converts the page text into numerical vectors and then calculates the cosine similarity of these vectors to obtain a score between 0 and 1. For example, if the similarity between a Chinese page and its corresponding English page is 0.9, this indicates a high translation quality; if the similarity is only 0.3, there may be serious translation problems. Setting a similarity threshold (such as 0.8) can help quickly identify links that need to be optimized. For link pairs with similarity below the threshold, further processing is required. This may involve re-acquiring the latest content of the source language page and generating a new target language page using a machine translation service such as Google Translate API. By comparing the similarity between the new page generated by machine translation and the original target language page, it can be determined whether the link mapping needs to be updated. If the similarity of the new page is significantly higher than that of the original page, for example, from 0.3 to 0.85, then it is possible to consider replacing the original wrong link with the new machine translated page. This optimization process needs to be repeated until all link pairs reach the preset similarity threshold. In the end, an optimized link mapping table will be obtained, which will become the basic configuration data for the website language version switching. At the same time, a detailed optimization result report will be generated, including the original similarity of each link pair, the optimized similarity, and the overall similarity improvement statistics. This method can not only improve the quality of multilingual websites, but also help website administrators quickly discover and solve translation problems. For example, if it is found that the pages of a certain product category generally have a low similarity problem, it may mean that the translation of professional terms in this category needs to be improved. By analyzing the optimization result report, website administrators can improve the translation quality and enhance the user experience in a targeted manner.

[0030] It should be noted that the above examples are only some specific embodiments of the present invention. Obviously, the present invention is not limited to the above embodiments, and there are many variations. All variations that can be directly derived or associated with the content disclosed by a person skilled in the art should be considered as the protection scope of the present invention. < / h6>

Claims

1. A multimodal translation method in a cross-border e-commerce scenario, characterized in that: The method comprises: Obtain the page content of the e-commerce platform, identify product links, category links and activity links from the page content, and extract the absolute path, relative path and dynamic parameter information of the link; For the identified links, natural language processing technology is used to analyze the link context to determine whether the target page the link points to is a product details page, a product list page, or an event feature page; According to the determined target page type, the corresponding page address of the link in the target language version is obtained from a pre-established multi-language page mapping table to generate link mapping relationship data; According to the generated link mapping relationship data, formulate processing rules for different types of links, use the processing rules to automatically convert the links in batches, and obtain the target language version links; Monitor the validity of the target language version links. If a broken link is detected, analyze the user behavior log and product information update records to map the broken link to the latest valid link. Use automated testing tools to simulate user click behavior, perform access tests on mapped links, check whether they jump to the target page correctly, and determine whether it is the correct target language version page based on the similarity of page content.

2. The method according to claim 1, characterized in that The method of obtaining the content of the e-commerce platform page, identifying product links, category links and activity links from the page content, and extracting the absolute path, relative path and dynamic parameter information of the link includes: Get the page content of the e-commerce platform and parse the page content into a DOM tree structure; According to the pre-defined characteristic rules of product links, category links and activity links, the corresponding link nodes are identified from the DOM tree; For each identified link node, determine whether it is an absolute path or a relative path; If the link node starts with "http: / / " or "https: / / ", it is considered an absolute path; Otherwise, it is determined to be a relative path; For link nodes that are determined to be relative paths, convert the relative paths into absolute paths based on the URL of the current page; From the URL of the link node, match and extract the dynamic parameter part through the pre-defined regular expression pattern to obtain the parameter name and parameter value; The absolute path, relative path and dynamic parameter information of the extracted link nodes are stored in a structured manner to build a link information database; The table structure of the link information database includes link ID, link URL, link type and dynamic parameter fields; Based on the manually annotated training data set, the features of the link nodes are extracted, and one of the naive Bayes or decision tree algorithms is trained to obtain a link classification model; Extract features from the newly collected link nodes, input the extracted features into the link classification model, and obtain the classification results of the link nodes; The classification results are stored in the link information database to improve the metadata information of the link nodes; Perform link analysis and application based on link information database.

3. The method according to claim 1, characterized in that The method of analyzing the link context using natural language processing technology for the identified link to determine whether the target page type pointed to by the link is a product details page, a product list page, or an activity topic page includes: Get the target web page link to be analyzed, and obtain the HTML content of the target web page through the requests library; Preprocess HTML content, use regular expressions to remove JavaScript and CSS noise, use the jieba library for Chinese word segmentation, and use the THULAC tool for part-of-speech tagging; For the preprocessed page content, the TextRank algorithm is used to extract keywords and key phrases; Get the feature word library of product detail page, product list page and activity topic page pre-built using TF-IDF; Calculate the cosine similarity between the target page content and the three types of page feature words, and determine the page type with the highest similarity as the type of the target page; If the target page is determined to be a product details page, regular expressions are used to match the HTML tags and their contents corresponding to the key attributes of product name, price, and specification parameters to extract structured product information fields; If the target page is determined to be a product list page, XPath is used to parse the HTML node tree to obtain the link, thumbnail URL, name and price information of each product in the product list, and the steps are recursively called for each product link to obtain product information; If the target page is determined to be an event-themed page, regular expressions are used to match the HTML content corresponding to the key attributes of event theme, time, and participation method to extract structured event information fields; The parsed structured product information or activity information is stored in the MongoDB database with the URL as the primary key, and the page type is recorded at the same time.

4. The method according to claim 1, characterized in that: The method of obtaining the corresponding page address of the link in the target language version from a pre-established multi-language page mapping table according to the determined target page type and generating link mapping relationship data includes: Get the URL address of the link to be processed, and use regular expression matching to determine the target page type of the link; For the determined target page type, a pre-established multi-language page mapping table is obtained, in which the URL addresses of corresponding pages in different language versions are recorded; From the multi-language page mapping table, obtain the corresponding page URL address that matches the target page type and target language version; If the corresponding page URL address is found in the multi-language page mapping table, the linked URL address is associated with the obtained corresponding page URL address to generate link mapping relationship data; Otherwise, record the unmatched link URL address for subsequent manual processing; Jieba word segmentation tool is used to segment the anchor text of the link, and the importance score of each word is calculated based on the TF-IDF algorithm, and the words with the highest scores are selected as keywords; The extracted keywords are used as attribute fields of link mapping relationship data and stored together with the URL address; According to the similarity between the link URL address and the anchor text, the K-means clustering algorithm is used to group the link mapping relationship data; Calculate the edit distance between URL addresses and the jaccard similarity between anchor text keywords, divide the data with high similarity into the same cluster, and obtain the link mapping relationship data set of different topics; The generated link mapping relationship data is stored in a pre-established MySQL relational database, and the database table structure includes link URL, target page type, target language version, corresponding page URL and anchor text keyword fields; The application queries the database to redirect links to multi-language web pages; At the same time, the stored multi-language link mapping relationship is used to perform machine translation on the web page content to generate web pages in different languages.

5. The method according to claim 1, characterized in that The method of formulating processing rules for different types of links for the generated link mapping relationship data, and using the processing rules to automatically convert the links in batches to obtain target language version links includes: Obtain link data of source language and target language, and use decision tree algorithm to establish link mapping relationship model; Use the link mapping relationship model to predict the full amount of link data to obtain preliminary link mapping relationship data; Formulate corresponding processing rules for different link types in the link mapping relationship data, including structure conversion rules for links and translation rules for the content of the page corresponding to the link; Use a distributed computing framework to batch process source language links, perform structural transformation on each link according to processing rules, and call the machine translation API to translate the page content into the target language to obtain the link data of the target language version; Perform quality checks on the target language version of the link data, automatically check the links using predefined quality assessment rules, and manually spot check to identify problematic links; For problematic links found in quality inspection, we use heuristic algorithms to optimize link conversion rules and machine translation parameters, and re-convert links for multiple rounds until the preset quality standards are met. Apply the optimized link conversion configuration to the full link data to generate the final target language version link, and push the link data to the multilingual website system through the Web service API.

6. The method according to claim 1, characterized in that The validity of the obtained target language version link is monitored. If an invalid link is detected, the user behavior log and product information update record are analyzed to map the invalid link to the latest valid link, including: Get all product detail page links of the target language version to form an initial link list; For each link in the initial link list, determine whether it can be accessed normally by sending an HTTP request; If the status code returned by the HTTP request is 200, the link is marked as a valid link; Otherwise, mark it as a broken link and record relevant information of the broken link; For each invalid link, check the user behavior log of the business system to see whether there is a latest access record of the product ID and page ID corresponding to the invalid link; If there is a latest access record, extract the link in the record as the potential latest valid link of the invalid link; If the latest access record is not found in the user behavior log, the latest update record of the product corresponding to the invalid link is searched in the product information update record, and the updated product details page link is extracted as the potential latest valid link of the invalid link; For invalid links that still cannot find the latest valid links, we use the item-based collaborative filtering algorithm to find the N products with the highest feature similarity from the link library according to the category and keyword features of the products they belong to, and obtain the detail page links of these N products as potential replacement links for the invalid links; For all potential latest valid links and potential alternative links, filter out invalid links; Establishing a mapping relationship between the filtered valid links and the corresponding invalid links to form a mapping table from invalid links to valid links; Traverse all the links on the page, and replace the invalid links with the corresponding valid links for those that have corresponding valid links in the mapping table; Update the page after link replacement and store the mapping table persistently in the database; For invalid links that cannot find corresponding valid links, they will be removed from the page. At the same time, the information of these invalid links that cannot be processed will be recorded in the log and reviewed manually on a regular basis.

7. The method according to claim 1, characterized in that The automated testing tool is used to simulate user click behavior, perform access testing on the mapped link, check whether the target page is correctly jumped to, and determine whether it is the correct target language version page based on the similarity of the page content, including: Obtain all page links of the source language website, and generate the target language page link corresponding to each source language page link according to the predefined language mapping relationship; For each source language-target language link pair in the link mapping table, an automated testing tool is used to simulate user click behavior, trigger a jump from the source language page to the target language page, and obtain the actual response target language page content; Extract key content features of the response page, use language detection tools to determine the page language type, and confirm whether it is the correct target language; Calculate the similarity between the text content of the response page and the text content of the expected target language page; Using the TF-IDF algorithm, the texts of the two pages are converted into term frequency-inverse document frequency matrices, and the cosine similarity of the two matrices is calculated to obtain a similarity score between 0 and 1; Set a predefined similarity threshold and judge the similarity score of each link pair; If the similarity is greater than the threshold, the link jump is determined to be correct and the corresponding link mapping does not need to be adjusted; If the similarity is less than the threshold, it is determined that there is a problem with the link jump, and the link pair is recorded in the list to be optimized; For each source language-target language link pair in the list to be optimized, the latest content of the source language page is retrieved, and the source language page is translated into the target language through a machine translation interface to obtain a new target language page.

Images