An intelligent device online language teaching and translation system based on image recognition

Abstract

The application belongs to the technical field of artificial intelligence, and specifically relates to an intelligent device online language teaching translation system based on image recognition, which discards the disadvantages of isolated recognition of characters or objects in images in the prior art, fuses visual information with accurate geographical positions, queries a scene-based expression rule set associated therewith, generates scene-based translation data flow and returns to a terminal device, first realizes deep perception and semantic understanding of a real physical environment by a translation system, and introduces individualized correction operation before driving and displaying at the terminal device, so that the final translation result is most applicable and understandable to a current user while taking into account an environmental context, the interaction behavior of the user is recorded synchronously, the decision logic of a scene cultural context is dynamically adjusted, the system realizes an intelligent closed loop of self-evolution, and long-term and continuously evolving individualized learning experience is provided for the user.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and specifically relates to an intelligent device online language teaching and translation system based on image recognition. BACKGROUND

[0002] With the deepening of globalization, the demand for cross-language communication is increasing. Smart wearable devices, such as phone watches, have become an ideal platform for realizing scenario-based language learning and translation due to their portability and real-time nature.

[0003] In the prior art, the mainstream solution of the online language teaching and translation of the smart wearable device captures images through the device camera, extracts text or object category labels through OCR or object recognition technology, and then sends them to a cloud translation engine for conversion. Finally, the results and glossaries are returned to the device interface for display. Although this solution realizes basic instant translation, its core logic has inherent limitations, specifically: the prior art only regards visual input as an isolated information source, and lacks the ability to perceive and integrate the cultural context carried by the geographical location and the deep environmental context. This leads to rigid translation results that are disconnected from the actual scenario culture, and learning content that is universal and static, which cannot be dynamically adjusted according to the specific environment and historical behavior of the user, ultimately restricting the accuracy and applicability of translation. SUMMARY

[0004] To overcome the shortcomings in the background art, the embodiments of the present application provide an intelligent device online language teaching and translation system based on image recognition, which can effectively solve the problems involved in the background art.

[0005] The purpose of the present application can be achieved by the following technical solution: an intelligent device online language teaching and translation system based on image recognition, comprising: a data acquisition module, a scene translation module, a display interaction module and a behavior learning module.

[0006] The data acquisition module is connected with the scene translation module, the scene translation module is connected with the display interaction module, the display interaction module and the behavior learning module are connected, and the behavior learning module is connected with the scene translation module.

[0007] The data acquisition module acquires visual scene images and synchronous geographical position coordinates collected by a terminal device, and generates a fusion data package.

[0008] The scene translation module performs multi-modal analysis on the fusion data package, identifies the text content and object semantics therein, and dynamically generates a scenario-based translation data stream in combination with the scenario cultural context bound to the geographical position coordinates.

[0009] displaying interaction module, returning the scene translation data stream to the terminal device to drive display, and recording user interaction behavior for displayed translation results in parallel.

[0010] behavior learning module, dynamically adjusting decision logic of the scene cultural context based on the user interaction behavior.

[0011] Compared with the prior art, embodiments of the present application have at least the following advantages or benefits: (1) The present application discards the drawbacks of the prior art of isolating recognition of characters or objects in images, fuses visual information with accurate geographic positions, queries a set of scene expression rules associated therewith, generates a scene translation data stream, and first realizes deep perception and semantic understanding of a real physical environment by a translation system, so that language learning fundamentally breaks through the paradigm of traditional abstract text understanding and moves towards a new paradigm of user embodied scene learning.

[0012] (2) The present application introduces personalized correction operation before driving display by a terminal device, dynamically replaces or simplifies rare words and complex sentence patterns in a translation into knowledge points mastered by a user by loading a personal vocabulary table and historical learning records of the user, ensures that the final translation result is most applicable and understandable to the current user individual while taking into account the environmental context, greatly improves the acceptance and learning efficiency of the user.

[0013] (3) The present application records user interaction behavior, quantizes it as positive / negative feedback signals for scene expression rules, and dynamically adjusts confidence weights of the rules according to the feedback signals, realizes an intelligent closed loop of self-evolution, and provides a long-term and evolving personalized learning experience for the user. BRIEF DESCRIPTION OF DRAWINGS

[0014] The present application is further described with the aid of the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present application, and other drawings can be obtained by those of ordinary skill in the art without creative labor on the premise of the following drawings.

[0015] Figure 1 It is a schematic diagram of module connection of the present application.

[0016] Figure 2 It is a schematic diagram of recognition logic of text content and object semantics of the present application.

[0017] Figure 3 It is a schematic diagram of execution logic of personalized correction operation of the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0019] Referring to Figure 1 As shown in the figure, the present application provides an intelligent device online language teaching translation system based on image recognition, comprising a data acquisition module, a scene translation module, a display interaction module and a behavior learning module.

[0020] The data acquisition module is connected with the scene translation module, the scene translation module is connected with the display interaction module, the display interaction module and the behavior learning module are connected, and the behavior learning module is connected with the scene translation module.

[0021] The data acquisition module acquires visual scene images and synchronous geographic position coordinates collected by a terminal device, and generates a fusion data packet.

[0022] The scene translation module performs multi-modal analysis on the fusion data packet, identifies Chinese text content and object semantics, and dynamically generates scene translation data stream in combination with a scene cultural context bound with the geographic position coordinates.

[0023] Referring to Figure 2 As shown in the figure, in a preferred embodiment of the present application, the text content and object semantic identification process is as follows: pre-processing is performed on the visual image data, and candidate text regions and candidate object regions in the image are preliminarily located by analyzing the color contrast at the pixel level and the connectivity of the regional morphology.

[0024] The above pre-processing performed on the visual image data is a series of standardization and enhancement operations performed before core recognition of the image, mainly including image noise reduction, scale standardization, color space conversion and enhancement, and the purpose is to make the text and object features required for subsequent identification more prominent and regular by eliminating noise, unifying scale and enhancing useful information. It should be noted that the operations involved in the above pre-processing are all mature existing technologies in the art, and will not be described here.

[0025] On this basis, in order to preliminarily separate the target to be identified from the pre-processed image, the system locates the candidate regions by analyzing the color contrast at the pixel level and the connectivity of the regional morphology. Specifically, since the edges of text or objects usually have obvious color or brightness mutations with the background, the system preliminarily detects a set of potential edge points by calculating the color contrast at the pixel level.

[0026] Further, it is considered that a meaningful recognition target must have a complete morphological structure: for text, the strokes of its characters are connected and the characters are arranged regularly.

[0027] For objects, they are composed of pixels with similar color texture in space.

[0028] Therefore, the system then employs morphological operations to connect the discrete text edge points into connected region blocks, and uses connected component search to aggregate the object pixels into connected candidate object regions.

[0029] At this point, the system aggregates discrete edge points into individual, connected candidate regions, and in order to filter out the real text and object regions from them, the last step is to screen them according to the geometric properties of these regions: the screening conditions for the candidate text regions are that the aspect ratio of the region meets the proportion characteristics of common characters or text lines, and its internal structure complexity meets the requirements of character configuration.

[0030] The screening conditions for the candidate object regions are that the outline of the region has integrity and its spatial distribution meets the morphological characteristics of a single object.

[0031] Perform character segmentation and sequence recognition on the candidate text regions, and output the text recognition result containing editable text strings and their bounding box position information in the image coordinate system.

[0032] Generate one or more predefined object class labels and corresponding classification confidence scores by extracting the depth visual features of the candidate object regions and performing classification inference.

[0033] The process of extracting the depth visual features of the candidate object regions and performing classification inference is as follows: input the located candidate object region into a pre-trained deep convolutional neural network, which performs a series of nonlinear transformations and feature dimensionality reduction on the input region through its multi-layer convolutional structure, and finally outputs a high-dimensional, abstract depth visual feature vector from the fully connected layer or a specific feature layer of the network. This vector is essentially a machine-readable, highly abstract numerical representation of the object region.

[0034] Input the depth visual feature vector into a classifier, which calculates the probability value of the region belonging to each predefined object class through matrix operations and normalization exponential operations, and the value range of the probability value is .

[0035] Iterate through all the calculated probability values, and take the class whose probability value is greater than zero and its corresponding probability value as the candidate result, where the probability value is the classification confidence score.

[0036] The text string, the bounding box position information, the object category label and the confidence score are associated and data-encapsulated, and are collectively used as the text content and the object semantics.

[0037] It should be noted that the above association and data encapsulation process specifically includes the following two core steps: one is the final selection of the object category label, that is, from multiple identification results that may exist, a most reliable object category label is determined, and the decision logic is a filtering and optimization process based on the classification confidence score: if only one object category label is identified, and the classification confidence score reaches the system preset reliability threshold, the label is directly taken as the final result.

[0038] If multiple object category labels are identified, all candidate labels with a confidence score lower than the reliability threshold are first filtered out, and then the candidate label with the highest confidence score is selected from the remaining candidate labels as the final object category label.

[0039] The preset reliability threshold is an empirical value determined based on the training and verification of the deep convolutional neural network on a large standard data set to balance the identification accuracy and recall rate, and is usually preset when the system is deployed.

[0040] The second is to establish the semantic relationship between the text and the object: according to the bounding box position information of the text string and the object region bounding box corresponding to the object category label, the overlapping area is calculated.

[0041] Based on the overlapping area, when the bounding box of a text string exceeds a preset proportion of the area falling within a certain object region bounding box, it is determined that the text is a label or attribute description of the object, and the text string and the object category label are bound and encapsulated, wherein the preset proportion may exemplarily be three-quarters.

[0042] In a preferred embodiment of the present application, the scene cultural context acquisition process bound with the geographic location coordinates is as follows: the geographic location coordinates are matched and queried with a preset geographic cultural knowledge base to obtain a set of scene expression rules of the associated area of the coordinates, and the set of scene expression rules at least includes local idioms, place-specific terms and cultural custom annotation information.

[0043] In a preferred embodiment of the present application, the construction process of the geographic cultural knowledge base includes: collecting multi-source geographic cultural data, and the data at least includes historical documents, folklore records, social media local topics and public place identification corpus of different regions.

[0044] The multi-source geographical and cultural data are cleaned and labeled, and are subjected to structured processing based on a preset cultural dimension label system, the cultural dimension label at least covering language habits, social etiquette and historical anecdotes.

[0045] The structured data are associated with points of interest in a geographical information system, to generate and store a preset geographical and cultural knowledge base, wherein each knowledge rule is bound to one or more geographical coordinate ranges.

[0046] In a preferred embodiment of the present application, the matching query implementation process is as follows: the geographical position coordinates are compared with the geographical coordinate ranges bound to each knowledge rule in the geographical and cultural knowledge base.

[0047] Based on a spatial indexing technology, all knowledge rules whose coordinate ranges cover the geographical position coordinates are retrieved to form an initial rule set.

[0048] According to the object semantics identified in the visual image data, the initial rule set is filtered to screen out a rule subset with high scene semantic correlation, as a final scene expression rule set.

[0049] It should be noted that the filtering of the initial rule set according to the identified object semantics is mainly to calculate the semantic correlation between the object semantics and the scene expression rules, and the specific quantification process is as follows: the identified object category label and the scene description applicable to each rule in the initial rule set are respectively mapped to the same high-dimensional semantic vector space to obtain corresponding object semantic vectors and rule semantic vectors.

[0050] In the semantic vector space, the cosine similarity between each object semantic vector and each rule semantic vector is calculated, and the numerical value of the cosine similarity is taken as the scene semantic correlation between the two, and the closer the numerical value is to 1, the stronger the semantic correlation is.

[0051] When there are multiple objects in the image, for each rule, the weighted average or maximum value operation is performed on the scene semantic correlations calculated for all identified objects to obtain the comprehensive correlation score of the rule and the current overall visual scene.

[0052] The rule subset with high semantic correlation is screened according to the following standard: all rules in the initial rule set are sorted in descending order according to the calculated comprehensive correlation scores, and a predetermined number of rules ranked at the top are selected to form the final scene expression rule set.

[0053] In a preferred embodiment of the present application, the dynamic generation of the scene expression translation data stream is implemented as follows: the identified text content is subjected to cross-language translation conversion to obtain a basic translation result.

[0054] Based on the scenario expression rule set, at least one of the following optimization operations is performed on the basic translation result: a. Replace the vocabulary in the basic translation with the local idiomatic vocabulary defined in the rule set.

[0055] b. According to the place type specified in the rule set, adjust the formal degree and sentence structure of the translation.

[0056] c. For specific cultural symbols in the translation, add background notes defined in the rule set.

[0057] The optimized translation and the added background notes are encapsulated together to generate a scenario translation data stream.

[0058] The embodiment of the present application discards the disadvantages of the prior art of isolated identification of characters or objects in images, fuses visual information with accurate geographical position, queries the scenario expression rule set associated therewith, generates a scenario translation data stream, and first realizes deep perception and semantic understanding of the real physical environment by the translation system, so that language learning fundamentally breaks through the paradigm of traditional abstract text understanding and moves towards a new paradigm of user embodied scene learning.

[0059] Referring to Figure 3 As shown in the preferred embodiment of the present application, before the scenario translation data stream returns to the terminal device to drive the display, the system also performs a personalized correction operation: loading pre-stored user portrait data, which at least includes the user's personal vocabulary and historical learning records.

[0060] The vocabulary and sentence patterns in the scenario translation data stream are matched and identified, and rare vocabulary and unmastered sentence patterns are screened out.

[0061] It should be noted that the above process of screening out rare vocabulary and unmastered sentence patterns is a dynamic evaluation process based on user historical data, and the specific analysis process is as follows: for rare vocabulary: the terminal device is built-in a general word frequency database and is maintained regularly, which counts the frequency of vocabulary in different languages in public corpus, and calculates the ratio of the frequency of a specific vocabulary to the average frequency of all vocabularies in the library, to quantify a standardized general commonness for each vocabulary, which is used as an objective benchmark to measure the general use degree of the vocabulary.

[0062] At the same time, the terminal device will perform time series analysis on the user's historical learning records to quantify the user's familiarity with specific vocabulary, and the calculation logic is to construct a time decay function of the user's last contact with the specific vocabulary, multiply the output result of the function with the frequency of the vocabulary in the user's historical learning records, and take the product as the user's familiarity with the specific vocabulary.

[0063] The general commonness and the user familiarity are linearly weighted and fused to generate a comprehensive rare index reflecting the cognitive difficulty of the vocabulary.

[0064] Finally, the vocabulary with a comprehensive rare index lower than a preset rare threshold is accurately determined as a rare vocabulary for the current user.

[0065] For the unmastered sentence pattern, the syntax of the scene-based translation data stream is parsed, and the syntactic structure features are extracted from multiple dimensions such as the number of sentences, the nesting depth, the used grammatical tense and voice.

[0066] The extracted syntactic structure features are compared with the historical learning records of the user, and the list of sentence pattern structures mastered by the user through exercises or tests is recorded in the historical learning records.

[0067] If the syntactic structure features of the current sentence do not appear in the list of sentence pattern structures mastered by the user, or although they appear, the proportion of the number of times of negative feedback signal attributes in the user's historical interaction behavior data is higher than that of positive feedback signal attributes, the sentence pattern is determined as an unmastered sentence pattern.

[0068] According to the recognition result, at least one of the following correction operations is performed: I. If the rare vocabulary exists in the preset synonym library, it is replaced by the mastered synonym in the user's personal vocabulary table and in the current scene-based context.

[0069] Ii. The unmastered sentence pattern is disassembled or reconstructed into a simplified sentence pattern that conforms to the mastered knowledge points in the user's historical learning records.

[0070] Iii. For the rare vocabulary and sentence pattern that cannot be replaced or simplified, an explanation note in accordance with the user's native language thinking habit is added.

[0071] The corrected translation and the added note are re-encapsulated into the final display data stream.

[0072] The personalized correction operation is introduced before the terminal device drives the display, the rare vocabulary and complex sentence pattern in the translation are dynamically replaced or simplified into the mastered knowledge points of the user by loading the personal vocabulary table and historical learning records of the user, the final translation result is ensured to be most applicable and understandable to the current user, and the acceptance and learning efficiency of the user are greatly improved.

[0073] The display interaction module returns the scene-based translation data stream to the terminal device to drive the display, and records the interaction behavior of the user for the displayed translation result in parallel.

[0074] The behavior learning module dynamically adjusts the decision logic of the scenario cultural context based on the user interaction behavior.

[0075] In a preferred embodiment of the present application, the dynamic adjustment of the decision logic of the scenario cultural context determines the feedback signal attribute of the user interaction behavior, adjusts the confidence weight of the associated rule triggered in the scenario expression rule set, and the attribute includes positive feedback signal or negative feedback signal.

[0076] It should be noted that the above adjustment of the confidence weight of the scenario expression rule follows the following explicit machine decision logic: when a rule is triggered and presented to the user, if it is associated with a positive feedback signal, the system performs a fixed step addition operation on the confidence weight of the rule.

[0077] If it is associated with a negative feedback signal, a fixed step subtraction operation is performed on the confidence weight of the rule.

[0078] And the initialization data for each rule confidence weight comes from large-scale historical user behavior statistics, by regularly integrating anonymous user interaction data from all such terminal devices, the total amount and proportion of positive and negative feedback signals associated with each rule in the history are calculated, and normalized to give each rule an initial confidence weight.

[0079] Based on the adjusted confidence weight, the query retrieval result of the scenario cultural knowledge base is dynamically sorted to ensure that high-weight rules are matched and returned in subsequent queries.

[0080] Continuously monitor the user interaction behavior of the negative feedback signal, when a specific behavior pattern appears continuously for many times, automatically trigger the knowledge base update logic, add user-verified alternatives to existing rules, or create new scenario expression rules.

[0081] It should be noted that the above specific behavior pattern specifically refers to an abnormal feedback sequence with a clear direction, and the following gives two specific examples: mode A: the user continuously performs editing operations on the translation result triggered by the same rule in the same scenario for many times, and the modification direction is consistent.

[0082] Mode B: the user ignores or requests to retranslate the output result triggered by the same rule for many times within a preset time window, wherein the preset time window can specifically refer to 1-3 minutes.

[0083] In a preferred embodiment of the present application, the determination of the feedback signal attribute of the user interaction behavior: if the user performs a long-time focusing operation on the displayed translation result, it is determined as a positive feedback signal.

[0084] It is necessary to supplement that the determination logic of the long-time focusing operation is: continuously monitoring the coordinate position of the user's visual focus on the screen through the front camera and the eye tracking algorithm, or monitoring the state of the continuous display of the translation result in the foreground of the screen.

[0085] When the user's visual focus continuously falls into the display area boundary box of the target translation result, or the translation result is in a continuously active display state, it is determined that the user is focusing on the content.

[0086] And the long-time focusing operation refers to the continuous duration of the above-mentioned focusing state reaching or exceeding a preset time threshold, for example, 3 seconds.

[0087] If the ignore operation, edit operation or request for re-translation operation is executed, it is determined that the negative feedback signal.

[0088] The embodiment of the application realizes the self-evolution of the intelligent closed loop by continuously recording the user's interactive behavior, quantifying it as positive / negative feedback signals to the scene expression rule, and dynamically adjusting the confidence weight of the rule according to the feedback signals, and provides the user with a long-term and evolving personalized learning experience.

[0089] In a preferred embodiment of the application, the fusion data packet generation process further includes data security processing: performing an encryption signature operation on the initially generated fusion data to generate a digital signature for verifying the data integrity and source authenticity.

[0090] The fusion data carrying the digital signature and the unique identifier of the terminal device are securely packaged to build a transmission data packet conforming to the security transmission protocol standard.

[0091] The transmission data packet is sent to the cloud through an encrypted communication channel.

[0092] The above is only an example and description of the structure of the application. Those skilled in the art can make various modifications or supplements to the described specific embodiments or use similar ways to replace them, as long as they do not deviate from the structure of the application or exceed the scope defined by the application, and should belong to the protection scope of the application.

Claims

1. An image recognition based intelligent device online language teaching translation system, characterized in that, The system comprises: a data acquisition module that acquires visual scene images and synchronous geographic coordinates collected by a terminal device, and generates a fusion data packet; a scene translation module that performs multi-modal analysis on the fusion data packet, identifies the text content and object semantics therein, and dynamically generates a scene-based translation data stream in combination with a scene culture context bound to the geographic coordinates; a display interaction module that returns the scene-based translation data stream to the terminal device to drive display and records user interaction behavior with respect to the displayed translation results in parallel; a behavior learning module that dynamically adjusts the decision logic of the scene culture context based on the user interaction behavior; The scene culture context bound to the geographic coordinates is obtained as follows: the geographic coordinates are matched and queried against a preset geographic culture knowledge base to obtain a set of scene-based expression rules for the region associated with the coordinates, the set of scene-based expression rules including at least local idioms, place-specific terms, and cultural custom annotation information; Before the scene-based translation data stream is returned to the terminal device to drive display, the system also performs a personalized correction operation: loads pre-stored user portrait data, the user portrait data including at least a user's personal vocabulary and historical learning records; matches and identifies the vocabulary and sentence patterns in the scene-based translation data stream to filter out rare vocabulary and unmastered sentence patterns; and performs at least one of the following correction operations according to the identification results: I. if the rare vocabulary exists in a preset synonym library, it is replaced with a synonym that is mastered by the user's personal vocabulary and conforms to the current scene-based context; Ii. the unmastered sentence patterns are disassembled or reconstructed into simplified sentence patterns that conform to the knowledge points mastered in the user's historical learning records; Iii. for rare vocabulary and sentence patterns that cannot be replaced or simplified, add explanatory notes that conform to the user's native language thinking habits; repackage the corrected translation and the added notes as a final display data stream; The decision logic of the dynamically adjusted scene culture context: determine the feedback signal attribute of the user interaction behavior, adjust the confidence weight of the associated rules triggered in the set of scene-based expression rules, the attribute including positive feedback signals or negative feedback signals; based on the adjusted confidence weight, dynamically sort the query and retrieval results of the scene culture knowledge base to ensure that high-weight rules are preferentially matched and returned in subsequent queries; continuously monitor the user interaction behavior of the negative feedback signal, and when a specific behavior pattern appears continuously for multiple times, automatically trigger the knowledge base update logic to add verified alternatives to the existing rules or create new scene-based expression rules.

2. The image recognition based intelligent device online language teaching translation system according to claim 1, characterized in that, The text content and object semantics identification process is as follows: perform preprocessing on the visual scene image to preliminarily locate candidate text regions and candidate object regions in the image by analyzing the color contrast and regional morphology connectivity at the pixel level; perform character segmentation and sequence recognition on the candidate text regions to output a text recognition result containing editable text strings and the position information of their bounding boxes in the image coordinate system; By extracting the depth visual features of the candidate object region and performing classification inference, one or more predefined object category labels and corresponding classification confidence scores are generated; The text string, bounding box position information, object category label, and confidence score are associated and data-encapsulated together as the text content and object semantics.

3. The image recognition based intelligent device online language teaching translation system according to claim 1, characterized in that, The construction process of the geographic and cultural knowledge base includes: Collecting multi-source geographic and cultural data, including at least historical documents, folklore records, local topics on social media, and identification corpus of public places in different regions; Cleaning and labeling the multi-source geographic and cultural data, and structuring based on the preset cultural dimension label system, which at least covers language habits, social etiquette, and historical anecdotes; Associating the structured data with points of interest in the geographic information system, and generating and storing as the pre-set geographic and cultural knowledge base, where each knowledge rule is bound to one or more geographic coordinate ranges.

4. The image recognition based intelligent device online language teaching translation system according to claim 2, characterized in that, The matching query implementation process is as follows: Compare the geographic location coordinates with the geographic coordinate ranges bound to each knowledge rule in the geographic and cultural knowledge base; Based on spatial indexing technology, retrieve all knowledge rules whose coordinate ranges cover the geographic location coordinates to form an initial rule set; According to the identified object semantics in the visual scene image, filter the initial rule set to select a rule subset with high relevance to the current scene semantics as the final scene-based expression rule set.

5. The image recognition based intelligent device online language teaching translation system according to claim 1, characterized in that, The dynamic generation of scene-based translation data stream is implemented as follows: Perform cross-language translation conversion on the recognized text content to obtain a basic translation result; Based on the scene-based expression rule set, perform at least one of the following optimization operations on the basic translation result: a. Replace the vocabulary in the basic translation with the local idioms defined in the rule set; b. Adjust the formality and sentence structure of the translation according to the place type specified in the rule set; c. Add background annotations defined in the rule set to specific cultural symbols in the translation; Encapsulate the optimized translation and the attached background annotations together to generate a scene-based translation data stream.

6. The image recognition based intelligent device online language teaching translation system according to claim 5, characterized in that, Determine the feedback signal properties of user interaction behavior: If the user performs a long-time focus operation on the displayed translation result, it is determined as a positive feedback signal; If the user performs an ignore operation, an edit operation, or a request for re-translation operation, it is determined as a negative feedback signal.

7. The image recognition based intelligent device online language teaching translation system according to claim 1, characterized in that, The fusion data packet generation process also includes data security processing: Perform encryption signature operation on the initially generated fusion data to generate a digital signature for verifying data integrity and source authenticity; Securely encapsulate the fusion data carrying the digital signature with the unique identifier of the terminal device to build a transmission data packet that meets the security transmission protocol standard; Send the transmission data packet to the cloud through an encrypted communication channel.

Images