Text display method and device, wearable equipment and storage medium

Abstract

The embodiment of the invention provides a text display method and a text display device based on wearable equipment, the wearable equipment and a computer readable storage medium. The method comprises the steps that a first image collected by the wearable device is acquired, and the first image at least comprises a text; translating a text on the first image to obtain a first text translation result; and when the validity of the first text translation result is greater than a first preset threshold value and the text of the first image is in the current display picture, obtaining a target position of the text on the first image in the current display picture, and displaying the first text translation result on the target position in an overlapping manner. The embodiment of the invention aims to improve the fitting degree of the translation result displayed on the basis of the wearable device and the text, so that a user can better associate the text with the translation result, and further can accurately understand and apply the translation result.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a text display method based on a wearable device, a text display device, a wearable device, and a computer-readable storage medium. Background Art

[0002] At present, when translating text, there are usually several methods in the related art: the first method is for the user to manually input the text into the terminal device for translation; the second method is for the user to take a photo of the real scene through the terminal device, translate the text in the photo, and then translate. The above methods all use the terminal device to achieve text translation, which requires the user to manually input into the terminal device or to hold the terminal device all the time, which may cause user fatigue and the text translation efficiency is low.

[0003] In view of the above reasons, some technicians have proposed related technologies for realizing text translation using wearable devices. However, although wearable devices can translate text in real time, improve text translation efficiency, and reduce user fatigue. However, when the text moves or the wearable device shakes, the display position of the translation result cannot be dynamically adjusted, that is, the translation result cannot be kept in line with the text, resulting in the user being unable to associate the text with the translation result, and thus causing the user to be unable to accurately understand and apply the translation result. Summary of the invention

[0004] The present application provides a text display method based on a wearable device, a text display device, a wearable device and a computer-readable storage medium, aiming to improve the degree of fit between the translation results displayed based on the wearable device and the text, so that the user can better associate the text with the translation results, and then accurately understand and apply the translation results.

[0005] To achieve the above object, the present application provides a text display method based on a wearable device, the method comprising:

[0006] Acquire a first image captured by the wearable device, wherein the first image includes at least text;

[0007] Translating the text on the first image to obtain a first text translation result;

[0008] When the validity of the first text translation result is greater than a first preset threshold and the text of the first image is on the current display screen, the text on the first image is acquired at a target position on the current display screen, and the first text translation result is superimposed and displayed on the target position.

[0009] In addition, to achieve the above-mentioned purpose, the present application also provides a text display device, which includes: an acquisition module, used to acquire a first image captured by the wearable device, wherein the first image includes at least text; a translation module, used to translate the text on the first image to obtain a first text translation result; a display module, used to acquire the text on the first image at a target position of the current display screen when the validity of the first text translation result is greater than a first preset threshold and the text of the first image is on the current display screen, and superimpose the first text translation result on the target position.

[0010] In addition, to achieve the above-mentioned purpose, the present application also provides a wearable device, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the steps of any one of the text display methods based on a wearable device provided in the embodiments of the present application when executing the computer program.

[0011] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the steps of any one of the text display methods based on a wearable device provided in the embodiments of the present application.

[0012] The text display method based on a wearable device, the wearable device and the computer-readable storage medium disclosed in the embodiment of the present application can obtain a first image collected by the wearable device, wherein the first image includes at least text. And the text on the first image is translated to obtain a first text translation result. Furthermore, when the validity of the first text translation result is greater than the first preset threshold value and the text of the first image is in the current display screen, the target position of the text on the first image in the current display screen is obtained, and the first text translation result is superimposed and displayed on the target position. Thus, the first text translation result can be superimposed and displayed on the target position. The present application aims to obtain the target position of the text in the display screen while translating the text, and then superimpose the text translation result on the target position in real time to improve the fit between the translation result and the text, so that the user can better associate the text with the translation result, and then accurately understand and apply the translation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0014] Figure 1 It is a flowchart of a text display method based on a wearable device provided in an embodiment of the present application;

[0015] Figure 2 is a scene schematic diagram of a text display method based on a wearable device provided in an embodiment of the present application;

[0016] Figure 3 is a schematic diagram of a scenario of collecting a first image through a wearable device provided in an embodiment of the present application;

[0017] Figure 4 It is a schematic diagram of a scenario in which a first text translation result is superimposed and displayed on a real-time position, provided in an embodiment of the present application;

[0018] Figure 5 It is a flowchart of refreshing text translation results provided by an embodiment of the present application;

[0019] Figure 6 is a schematic block diagram of a text display device provided in an embodiment of the present application;

[0020] Figure 7 It is a schematic block diagram of a wearable device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor do they have to be executed in the order described. For example, some operations / steps can also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions. In addition, although the functional modules are divided in the device schematic, in some cases, the module division can be different from that in the device schematic.

[0023] The term "and / or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0024] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0025] See also Figures 1 to 3 , Figure 1 It is a flowchart of a text display method based on a wearable device provided in an embodiment of the present application; Figure 2 is a scene schematic diagram of a text display method based on a wearable device provided in an embodiment of the present application; Figure 3 This is a schematic diagram of a scenario in which a first image is captured by a wearable device, as provided in an embodiment of the present application.

[0026] like Figure 1 As shown, the text display method based on the wearable device includes steps S11 to S13.

[0027] Step S11: Acquire a first image captured by the wearable device, wherein the first image includes at least text.

[0028] The first image is an image captured for the first time by an image acquisition unit of the wearable device, and includes at least some text. Therefore, the first image can be used to obtain and translate the text therein.

[0029] The wearable device may be smart glasses, such as AR glasses, VR glasses, etc., which is not limited in this application.

[0030] It should be noted that the present application does not limit the manner in which the wearable device collects the first image. For example, the first image may be collected by an image acquisition unit on the wearable device. The image acquisition unit includes components such as a camera, a sensor, and a lens installed on the wearable device. These components work together to collect images and information of the surrounding environment.

[0031] Furthermore, the present application does not limit the type of the above-mentioned camera, for example, the type of the camera is an RGB camera, a depth camera, a panoramic camera, etc. The present application and the type of the camera are described by taking an RGB camera as an example. RGB cameras are generally used to capture color images, and can generate color images by simultaneously capturing information of the three color channels of red, green, and blue. Therefore, RGB cameras implement tasks such as collecting images or videos, image recognition, and processing in wearable devices.

[0032] Specifically, the first image may be collected by components such as an RGB camera. Since the first image includes text, subsequent text translation steps may be performed based on the collected first image.

[0033] Optionally, the wearable device has no stored images before capturing the first image, or clears the originally stored images to prevent other images from conflicting with the first image, thereby preventing the accurate acquisition of the translated text required by the user.

[0034] Optionally, the initial display screen corresponding to the first image has no displayed text before the image is captured.

[0035] The initial display screen is the display screen corresponding to the first image.

[0036] It should be noted that the first image is an image of the real world collected by the wearable device, and the display screen is a screen on the wearable device for displaying virtual content. Therefore, the user can observe the images and information of the virtual world through the display screen.

[0037] It is understandable that when a user wears a wearable device, he or she can not only obtain images of the real world through the wearable device, but also obtain the display content of the display screen at the same time. Therefore, if text is displayed in the corresponding display screen when the first image is collected, it may cause the final translation result to conflict with the displayed text, and then the user will not be able to associate the translation result with the text to be translated, and will not be able to understand and apply the translation result. In addition, the displayed text in the display screen may also cause the algorithm to misjudge or confuse. Therefore, ensuring that the initial display screen corresponding to the first image has no displayed text before the image is collected will help reduce interference factors in the subsequent text translation process.

[0038] Optionally, before the first image is captured by the image acquisition unit of the wearable device, the process includes: determining first state information of the wearable device by a preset sensor; when the first state information meets a first preset condition, capturing the first image by the wearable device; or, in response to a user operation instruction, controlling the wearable device to capture the first image.

[0039] It should be noted that the present application does not limit the conditions for triggering the wearable device to capture the first image. For example, the image acquisition unit of the wearable device may be triggered to capture the first image by a preset sensor, or the image acquisition unit of the wearable device may be controlled to capture the first image directly based on a user operation instruction.

[0040] Among them, the first state information is the motion state information of the wearable device before collecting the image; the preset sensor can be an IMU (Inertial Measurement Unit) sensor, a visual sensor, and a positioning sensor, etc. This application takes the preset sensor as an IMU sensor as an example for explanation.

[0041] It should be noted that IMU sensors usually contain multiple built-in components such as gyroscopes, accelerometers, and magnetometers, which are used to measure and monitor the motion state and direction of the device without the need for external reference points. IMU sensors can provide data about the three-dimensional motion state and direction of the device, such as acceleration information, three-degree-of-freedom information, etc., from which the motion state information of the device can be determined.

[0042] Specifically, after the motion state information of the current wearable device is detected by the IMU sensor, it can be further determined whether it meets the first preset condition, and then when the motion state information of the wearable device meets the first preset condition, the image acquisition unit of the wearable device is controlled to capture the first image.

[0043] It should be noted that the present application does not limit the first preset condition. For example, the first preset condition is that the motion state of the wearable device is relatively stable, which can be specifically determined based on the acceleration information or three-degree-of-freedom information of the wearable device, and this application will explain it in detail later. It can be understood that when it is determined that the motion state of the wearable device has reached a relatively stable motion state, the image acquisition unit of the wearable device can be controlled to acquire a stable first image, and then the text on the first image can be clearly acquired to improve the quality of subsequent text translation.

[0044] It should be noted that the three-degree-of-freedom information can be used to indicate the motion state of the device, which generally involves the device in the three motion states of horizontal rotation, vertical rotation and pitch rotation. Therefore, by detecting the three-degree-of-freedom information of the wearable device, it can be used to determine the current motion state of the wearable device, and then when the motion state of the wearable device meets the first preset condition, the image acquisition unit is controlled to acquire a stable first image.

[0045] In addition, the wearable device may be controlled to capture the first image in direct response to the user's operation instruction. The user's operation instruction may be triggered by pressing a button of the wearable device or by a terminal device connected to the wearable device, and this application does not limit this.

[0046] Based on the above embodiment, the state of the wearable device includes a first state, a second state and a third state. When the first state information satisfies a first preset condition, a first image is captured by the wearable device, including: when the first state information satisfies the transition from the first state or the second state to the third state, and the stay time in the third state exceeds a first preset time length, the first image is captured by the wearable device.

[0047] The first state is a violent fluctuation state; the second state is a medium-speed fluctuation state; and the third state is a stable state. The first preset time length can be 0.5s, 1s, 2s, etc. This application takes the first preset time length of 0.5s as an example for explanation.

[0048] It should be noted that the above-mentioned violent fluctuation state, medium-speed fluctuation state and stable state can be determined according to the three-degree-of-freedom information or acceleration information of the wearable device. This application takes the three-degree-of-freedom information as an example for explanation. For example, the three-degree-of-freedom information of the wearable device can be classified according to specific intervals, and each specific interval corresponds to a state. Therefore, after the three-degree-of-freedom information of the wearable device is detected by the IMU sensor, the state of the wearable device can be determined based on the interval where the three-degree-of-freedom information of the wearable device is located.

[0049] Specifically, when the wearable device changes from a violent fluctuation state or a medium-speed fluctuation state to a stable state, and the stay time in the stable state exceeds 0.5s, it can be determined that the motion state of the wearable device has reached a relatively stable motion state. At this time, the image acquisition unit of the wearable device can be controlled to capture a stable first image, and then the text on the first image can be clearly obtained to improve the quality of subsequent text translation.

[0050] In an embodiment of the present application, the first image may be acquired by an image acquisition unit of the wearable device to obtain text on the first image for subsequent text translation.

[0051] Step S12: Translate the text on the first image to obtain a first text translation result.

[0052] The first text translation result is a translation result of the text on the first image.

[0053] It should be noted that this application does not limit the way to translate text. For example, the wearable device can communicate with the cloud or other devices with translation functions (such as mobile phones, tablets or computers, etc.), and then the wearable device sends the text to the cloud or other devices with translation functions for text translation. Alternatively, optical character recognition technology and translation engines can be integrated on the wearable device to achieve instant local translation on the wearable device.

[0054] In the embodiment of the present application, the text on the first image can be translated to obtain a corresponding first text translation result, so as to realize real-time text translation.

[0055] Step S13: when the validity of the first text translation result is greater than a first preset threshold and the text of the first image is in the current display screen, obtain the target position of the text on the first image in the current display screen, and overlay the first text translation result on the target position.

[0056] The validity of the first text translation result indicates the quality or accuracy of the first text translation result; and the target position is the real-time position of the text on the first image in the current display screen.

[0057] It can be understood that when the validity of the first text translation result is greater than the first preset threshold, it means that the quality or accuracy of the first text translation result is high, that is, the first text translation result is valid.

[0058] It should be noted that the present application does not limit the first preset threshold, for example, the first preset threshold is 80%, 90%, etc. In addition, the present application does not limit the method for obtaining the validity of the text translation result, for example, the accuracy of the text translation can be analyzed and evaluated by natural language processing tools, or the quality of the text translation can be automatically evaluated by BLEU (Bilingual Evaluation Understudy) or ROUGE to obtain the validity of the first text translation result.

[0059] Furthermore, when the validity of the first text translation result is greater than a first preset threshold and the text on the first image is in the current display screen, the real-time position of the text in the current display screen can be obtained, thereby superimposing the first text translation result on the real-time position of the text, so that the user can conveniently view the translation result at the position corresponding to the text.

[0060] Optionally, the first text translation result may also be displayed superimposed near the real-time position of the text, for example, superimposed above or below the text, etc., which is not limited in the present application.

[0061] It should be noted that the present application does not limit the method for obtaining the real-time position of text on the display screen. For example, it can be obtained through a visual odometer, a particle filter, or an optical flow tracking method. The present application uses the optical flow tracking method to obtain the real-time position of text on the display screen as an example.

[0062] Optical flow tracking is a computer vision technology used to track the real-time position of a moving object in consecutive image frames based on the movement pattern of pixels in the image. Specifically, optical flow tracking can obtain a series of consecutive image frames containing a specific target, analyze the pixels in each frame, and estimate their position in the next frame. In this way, the movement of the target can be tracked by combining the movement information of the pixels of a specific target.

[0063] In an embodiment of the present application, the position of the text on the display page can be tracked by an optical flow tracking method and continuously updated to reflect the real-time position of the text on the display page.

[0064] Optionally, after translating the text on the first image to obtain a first text translation result, the method further includes: when the validity of the first text translation result is less than or equal to a first preset threshold, re-capturing a second image by the image acquisition unit.

[0065] It is understandable that when the validity of the first text translation result is less than or equal to the first preset threshold, it means that the text quality on the first image is poor or the accuracy of the translation result is not high enough, so it is necessary to obtain a better translation result by re-capturing the image.

[0066] In an embodiment of the present application, the first text translation result can be superimposed on the real-time position of the text, thereby improving the fit between the translation result and the text, allowing the user to better associate the text with the translation result, and then accurately understand and apply the translation result.

[0067] Optionally, after translating the text on the first image and obtaining the first text translation result, the method further includes: when the validity of the first text translation result is greater than a first preset threshold and the text on the first image is not on the current display screen, obtaining the end position of the text on the first image leaving the current display screen; and displaying the first text translation result at the end position.

[0068] Specifically, when the validity of the first text translation result is greater than the first preset threshold, and the text on the first image is not in the current display screen, it means that the text on the first image may have left the current display screen due to the movement of the wearable device or the movement of the text. At this time, the end position where the text on the first image leaves the current display screen can be obtained, and then the first text translation result is displayed at the end position. In this way, the user can associate the first text translation result with the text, and then can accurately understand and apply the translation result.

[0069] Optionally, when the text on the first image is not on the current display screen, the first text translation result can also be displayed at other fixed positions on the current display screen, such as the middle position or the top position, so that the user can associate the first text translation result with the text, and then accurately understand and apply the translation result.

[0070] In the embodiment of the present application, when the validity of the first text translation result is greater than the first preset threshold value and the text on the first image is not in the current display screen, the first text translation result can be displayed at the end position. Thus, the user can associate the first text translation result with the text, and then can accurately understand and apply the translation result.

[0071] The text display method based on a wearable device, the wearable device and the computer-readable storage medium disclosed in the embodiment of the present application can collect a first image through a wearable device, wherein the first image includes at least text. And the text on the first image is translated to obtain a first text translation result. Furthermore, when the validity of the first text translation result is greater than a first preset threshold value and the text of the first image is on the current display screen, the target position of the text on the first image on the current display screen is obtained, and the first text translation result is superimposed and displayed on the target position. The present application aims to obtain the target position of the text in the display screen while translating the text, and then superimpose the text translation result on the target position in real time to improve the fit between the translation result and the text, so that the user can better associate the text with the translation result, and then can accurately understand and apply the translation result.

[0072] Please continue reading Figure 5 , Figure 5 1 is a flow chart of refreshing text translation results provided by an embodiment of the present application. Figure 5 As shown, refreshing the text translation result can be achieved through steps S21 to S24.

[0073] Step S21: determining second state information of the wearable device through a preset sensor.

[0074] Step S22: When the second state information satisfies a second preset condition, a second image is captured by the wearable device, wherein the second image includes at least text.

[0075] Among them, the second state information is the motion state information of the wearable device after the text translation result is displayed; the second image is the image captured for the second time by the image acquisition unit of the wearable device; the preset sensor can be an IMU (Inertial Measurement Unit) sensor, a visual sensor, and a positioning sensor, etc. This application takes the preset sensor as an IMU sensor as an example for explanation. The specific description refers to the above embodiment. To avoid repetition, it is not repeated here.

[0076] It is understandable that the accuracy of the first text translation result mainly depends on the clarity of the first image, that is, whether the text on the image can be obtained relatively clearly. Therefore, in order to improve the quality of the text translation result, after displaying the text translation result, a second image showing the text can be further obtained to translate the text on the second image to obtain a second text translation result, so that when the validity of the second text translation result is greater than that of the first text translation result, the text translation result is replaced, that is, the text translation result is refreshed.

[0077] Specifically, after the motion state information of the wearable device after displaying the text translation result is detected by the IMU sensor, it can be further determined whether it meets the second preset condition, and then when the motion state information of the wearable device meets the second preset condition, the image acquisition unit of the wearable device is controlled to capture the second image.

[0078] It should be noted that the present application does not limit the second preset condition. For example, the second preset condition is that the motion state of the wearable device is relatively stable, which can be determined based on the acceleration information or three-degree-of-freedom information of the wearable device. For a specific description, see the above embodiment. It can be understood that when it is determined that the motion state of the wearable device reaches a relatively stable motion state, the image acquisition unit can be controlled to acquire a stable second image, and then the text on the second image can be clearly acquired to improve the quality of text translation.

[0079] Optionally, when the second state information satisfies a second preset condition, a second image is captured through a wearable device, including: when the second state information satisfies the transition from the first state or the second state to the third state, and the stay time in the third state exceeds a first preset time length, the second image is captured through the wearable device; or, when the second state information satisfies the stay time in the second state and in the third state exceeds a second preset time length, the second image is captured through the wearable device.

[0080] The first state is a violent fluctuation state; the second state is a medium-speed fluctuation state; and the third state is a stable state. The first preset duration can be 0.5s, 1s, 2s, etc.; the second preset duration can be 3s, 5s, 6s, etc. This application takes the first preset duration of 0.5s and the second preset duration of 6s as an example for explanation.

[0081] It should be noted that the above-mentioned violent fluctuation state, medium-speed fluctuation state and stable state can be determined according to the three-degree-of-freedom information or acceleration information of the wearable device. This application takes the three-degree-of-freedom information as an example for explanation. For example, the three-degree-of-freedom information of the wearable device can be classified according to specific intervals, and each specific interval corresponds to a state. Therefore, after the three-degree-of-freedom information of the wearable device is detected by the IMU sensor, the state of the wearable device can be determined by determining the interval in which it is located.

[0082] Specifically, when the wearable device changes from a violent fluctuation state or a medium-speed fluctuation state to a stable state, and the dwell time in the stable state exceeds 0.5s, it is determined that the motion state of the wearable device has reached a relatively stable motion state, and at this time, the image acquisition unit of the wearable device can be controlled to acquire a stable second image. Alternatively, when the wearable device stays in a medium-speed fluctuation state and a stable state for more than 6s, it is determined that the motion state of the wearable device has reached a relatively stable motion state, and at this time, the image acquisition unit of the wearable device can be controlled to acquire a stable second image. In this way, the text on the second image can be clearly acquired to improve the quality of subsequent text translation.

[0083] Step S23: Translate the text on the second image to obtain a second text translation result.

[0084] Step S24: when the validity of the second text translation result is greater than the validity of the first text translation result, the first text translation result is replaced with the second text translation result by using the display unit of the wearable device.

[0085] The second text translation result is a translation result of the text on the second image.

[0086] Specifically, the text on the second image can be translated to obtain a corresponding second text translation result to achieve real-time text translation. Further, the validity of the second text translation result can be determined and compared with the validity of the first text translation result.

[0087] It should be noted that the method for implementing text translation and obtaining the validity of the text translation result can be referred to the above embodiment, and will not be described here in detail to avoid repetition.

[0088] It can be understood that when the validity of the second text translation result is greater than the validity of the first text translation result, it means that the quality of the second text translation result is higher than that of the first text translation result, and the first text translation result can be replaced by the second text translation result. On the contrary, it means that the quality of the second text translation result is not as high as that of the first text translation result, and the second text translation result can be abandoned and the first text translation result can be retained.

[0089] In the embodiment of the present application, a second image including text can be further acquired according to the state of the wearable device to translate the text on the second image to obtain a second text translation result, and then when the quality of the second text translation result is higher than that of the first text translation result, the first text translation result is replaced with the second text translation result. In this way, a more accurate and real-time text translation service is dynamically provided to ensure that users obtain more accurate translation results while meeting translation needs under specific conditions.

[0090] like Figure 2 As shown, the scenario of the text display method based on wearable devices of the present application mainly includes the following three processes: input process (Input process), processing process (Process process) and output process (Output process). Among them, the IMU (Inertial Measurement Unit) + 3dof (degree of freedom) scheme image acquisition process corresponds to the present application of determining the degree of freedom information of the wearable device based on the IMU sensor, and then determining the state of the wearable device according to the degree of freedom information of the wearable device, so that when the state of the wearable device reaches the preset condition, the image acquisition unit of the wearable device is controlled to acquire a stable first image.

[0091] Specifically, during the input process, the image acquisition unit can be controlled to capture the first image in response to the user's operation instruction, that is, by pressing a button of the wearable device (single-click to take a photo). Among them, the image acquisition unit of the wearable device has no stored images before capturing the first image, or clears the original stored images to prevent other images from conflicting with the first image, thereby failing to accurately obtain the translation text required by the user. In addition, the initial display screen corresponding to the first image has no displayed text before the image is captured to reduce interference factors in the subsequent text translation process. 。

[0092] It should be noted that the present application does not limit the above-mentioned implementation of the image acquisition unit of the wearable device having no stored images before acquiring the first image, and the method of implementing the initial display screen corresponding to the first image having no displayed text before acquiring the image. For example, the "forced start function" of the image acquisition unit can be turned on to clear the original text and status of the image acquisition unit (that is, the initial display screen corresponding to the first image has no displayed text before acquiring the image); or, the "initialization function" of the image acquisition unit is turned on, so as to implement the above implementation of the image acquisition unit having no stored images before acquiring the first image, and the initial display screen corresponding to the first image has no displayed text before acquiring the image.

[0093] Furthermore, before the image acquisition unit of the wearable device acquires the first image, the degree of freedom information of the wearable device can also be acquired through a preset sensor (such as an IMU sensor) to determine the state information of the wearable device. When the wearable device changes from a violent fluctuation state or a medium-speed fluctuation state to a stable state, and the stay time in the stable state exceeds 0.5s, it can be determined that the motion state of the wearable device has reached a relatively stable motion state, and at this time, the image acquisition unit of the wearable device can be controlled to acquire a stable first image.

[0094] During the processing, the text on the first image can be translated through the cloud. Thus, during the output process, the first text translation result can be obtained through the cloud output, and the first text result can be superimposed and displayed at the target position; or, the text displayed on the first image leaves the end position of the current display screen.

[0095] Optionally, the validity of the first text translation result can also be determined, that is, whether a valid result is generated. If the first text translation result is valid (a valid result is generated), the first text translation result will be maintained. If the first text translation result is invalid, the image is captured again by the image acquisition unit of the wearable device, and the above text translation steps are repeated.

[0096] Furthermore, when the first text translation result is valid, the first text translation result may be determined as the original translation copy, and the following steps are continued:

[0097] During the output process, the status information of the wearable device (corresponding to Figure 2 When the wearable device stays in the medium-speed fluctuation state and the stable state for more than 6 seconds, or changes from the violent fluctuation state or the medium-speed fluctuation state to the stable state, and the stay time in the stable state exceeds 0.5 seconds, it is determined that the motion state of the wearable device reaches a relatively stable motion state. At this time, the image acquisition unit of the wearable device can be controlled to capture a stable second image.

[0098] Therefore, during the processing, the text on the first image can be translated through the cloud, so that during the output process, the second text translation result can be obtained through the cloud output.

[0099] Optionally, the validity of the second text translation result can be further compared with the validity of the first text translation result (original translation copy), that is, whether a valid result is generated. If the validity of the second text translation result is greater than the validity of the first text translation result (original translation copy) (a valid result is generated), the first text translation result (original translation copy) is replaced with the second text translation result. If the validity of the second text translation result is less than or equal to the validity of the first text translation result (original translation copy) (no valid result is generated), the first text translation result (original translation copy) is maintained.

[0100] The method of the present application can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer terminal devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices.

[0101] See also Figure 6 , Figure 6 1 is a schematic block diagram of a text display device provided in an embodiment of the present application. The text display device of the image can be configured in a server to execute the aforementioned text display method based on a wearable device.

[0102] like Figure 6 As shown, the text display device 200 includes: an acquisition module 201 , a translation module 202 , and a display module 203 .

[0103] An acquisition module 201 is used to acquire a first image captured by the wearable device, wherein the first image at least includes text;

[0104] A translation module 202, configured to translate the text on the first image to obtain a first text translation result;

[0105] The display module 203 is used to obtain the text on the first image at a target position of the current display screen, and to superimpose and display the first text translation result on the target position when the validity of the first text translation result is greater than a first preset threshold and the text of the first image is on the current display screen.

[0106] The acquisition module 201 is also used to determine the first state information of the wearable device through a preset sensor; when the first state information meets a first preset condition, collect the first image through the wearable device; or, in response to a user operation instruction, control the wearable device to collect the first image.

[0107] The acquisition module 201 is further used to collect the first image through the wearable device when the first state information satisfies the conversion from the first state or the second state to the third state, and the stay time in the third state exceeds a first preset time length.

[0108] The display module 203 is further used to obtain the end position where the text on the first image leaves the current display screen when the validity of the first text translation result is greater than a first preset threshold and the text on the first image is not on the current display screen; and display the first text translation result at the end position.

[0109] The display module 203 is also used to determine the second state information of the wearable device through the preset sensor; when the second state information meets the second preset condition, collect a second image through the wearable device, wherein the second image at least includes text; translate the text on the second image to obtain a second text translation result; when the validity of the second text translation result is greater than the validity of the first text translation result, replace the first text translation result with the second text translation result.

[0110] The acquisition module 201 is also used to collect the second image through the wearable device when the second state information satisfies the transition from the first state or the second state to the third state, and the stay time in the third state exceeds the first preset time length; or, when the second state information satisfies the stay time in the second state and in the third state exceeds the second preset time length, collect the second image through the wearable device.

[0111] The acquisition module 204 is further configured to re-capture the second image through the wearable device when the validity of the first text translation result is less than or equal to the first preset threshold.

[0112] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0113] Exemplarily, the above method and apparatus may be implemented in the form of a computer program. The computer program may be implemented in Figure 7 The wearable device shown is running on the

[0114] See also Figure 7 , Figure 7 4 is a schematic diagram of a wearable device provided in an embodiment of the present application. The wearable device 400 may be a server.

[0115] like Figure 7 As shown, the wearable device 400 includes a processor 401 and a memory 402 connected via a system bus, wherein the memory 402 may include a volatile storage medium, a non-volatile storage medium and an internal memory.

[0116] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor 401 can execute any text display method based on a wearable device.

[0117] The processor 401 is used to provide computing and control capabilities to support the operation of the entire wearable device 400 .

[0118] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor 401, the processor 401 can execute any text display method based on the wearable device.

[0119] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art can understand that the structure of the wearable device 400 is only a block diagram of a part of the structure related to the present application scheme, and does not constitute a limitation on the wearable device 400 to which the present application scheme is applied. The specific wearable device 400 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0120] It should be understood that the processor 401 may be a central processing unit (CPU), and the processor 401 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0121] In some embodiments, the processor 401 is used to run a computer program stored in the memory 402 to implement the following steps: obtaining a first image captured by the wearable device, wherein the first image includes at least text; translating the text on the first image to obtain a first text translation result; when the validity of the first text translation result is greater than a first preset threshold and the text of the first image is on the current display screen, obtaining the text on the first image at a target position on the current display screen, and superimposing the first text translation result on the target position.

[0122] In some embodiments, the processor 401 is also used to determine first state information of the wearable device through a preset sensor; when the first state information meets a first preset condition, collect the first image through the wearable device; or, in response to a user operation instruction, control the wearable device to collect the first image.

[0123] In some embodiments, the processor 401 is further used to collect the first image through the wearable device when the first state information satisfies the transition from the first state or the second state to the third state, and the stay time in the third state exceeds a first preset time length.

[0124] In some embodiments, the processor 401 is further used to obtain the end position where the text on the first image leaves the current display screen when the validity of the first text translation result is greater than the first preset threshold and the text on the first image is not on the current display screen; and display the first text translation result at the end position.

[0125] In some embodiments, the processor 401 is further used to determine second state information of the wearable device through the preset sensor; when the second state information meets a second preset condition, capture a second image through the wearable device, wherein the second image includes at least text; translate the text on the second image to obtain a second text translation result; when the validity of the second text translation result is greater than the validity of the first text translation result, replace the first text translation result with the second text translation result.

[0126] In some embodiments, the processor 401 is further used to collect the second image through the wearable device when the second state information satisfies the transition from the first state or the second state to the third state, and the stay time in the third state exceeds the first preset time length; or, collect the second image through the wearable device when the second state information satisfies the stay time in the second state and in the third state exceeds the second preset time length.

[0127] In some implementations, the processor 401 is further configured to re-capture the second image through the wearable device when the validity of the first text translation result is less than or equal to the first preset threshold.

[0128] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions, and when the program instructions are executed, any one of the text display methods based on a wearable device provided in the embodiment of the present application is implemented.

[0129] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0130] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function, and the like.

[0131] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A text display method based on a wearable device, characterized in that: The method comprises: Acquire a first image captured by the wearable device, wherein the first image includes at least text; Translating the text on the first image to obtain a first text translation result; When the validity of the first text translation result is greater than a first preset threshold and the text of the first image is on the current display screen, the text on the first image is acquired at a target position on the current display screen, and the first text translation result is superimposed and displayed on the target position.

2. The method according to claim 1, characterized in that: Before acquiring the first image captured by the wearable device, the method includes: Determining first state information of the wearable device through a preset sensor; When the first state information satisfies a first preset condition, collecting the first image through the wearable device; or, In response to a user operation instruction, the wearable device is controlled to capture the first image.

3. The method according to claim 2, characterized in that The state of the wearable device includes a first state, a second state, and a third state, and when the first state information satisfies a first preset condition, collecting the first image through the wearable device includes: When the first state information satisfies the requirement of being transformed from the first state or the second state to the third state, and the stay time in the third state exceeds a first preset time length, the first image is collected through the wearable device.

4. The method according to claim 1, characterized in that: After translating the text on the first image to obtain a first text translation result, the method further includes: When the validity of the first text translation result is greater than a first preset threshold and the text on the first image is not on the current display screen, obtaining an end position of the text on the first image leaving the current display screen; The first text translation result is displayed at the end position.

5. The method according to claim 1, characterized in that After the first text translation result is superimposed and displayed on the target position, the method further includes: Determining second state information of the wearable device through the preset sensor; When the second state information satisfies a second preset condition, collecting a second image through the wearable device, wherein the second image includes at least text; Translating the text on the second image to obtain a second text translation result; When the validity of the second text translation result is greater than the validity of the first text translation result, the first text translation result is replaced with the second text translation result.

6. The method according to claim 5, characterized in that When the second state information satisfies a second preset condition, collecting a second image by the wearable device includes: When the second state information satisfies the transition from the first state or the second state to the third state, and the stay time in the third state exceeds the first preset time length, the second image is collected by the wearable device; or, When the second state information satisfies that the stay time in the second state and in the third state exceeds a second preset time length, the second image is collected through the wearable device.

7. The method according to claim 1, characterized in that After translating the text on the first image to obtain a first text translation result, the method further includes: When the validity of the first text translation result is less than or equal to the first preset threshold, the second image is collected again through the wearable device.

8. A text display device, characterized in that: The text display device comprises: An acquisition module, configured to acquire a first image captured by the wearable device, wherein the first image includes at least text; A translation module, used for translating the text on the first image to obtain a first text translation result; The display module is used for obtaining the text on the first image at a target position of the current display screen, and superimposing and displaying the first text translation result on the target position when the validity of the first text translation result is greater than a first preset threshold and the text of the first image is on the current display screen.

9. A wearable device, characterized in that: include: A memory and a processor; wherein the memory is connected to the processor for storing a program; the processor is used to implement the steps of the text display method based on a wearable device as described in any one of claims 1 to 7 by running the program stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the steps of the text display method based on a wearable device as described in any one of claims 1 to 7.

Images