Image translation method, image translation aparatus and electronic device

Abstract

Embodiments of this application provide an image translation method, image translation apparatus and electronic device. The method includes: obtaining a source image, where the source image includes a source text block with one or multiple source text lines; generating a target image based on geometric properties of the source text block, where the target image comprises one or multiple target text lines translated from the one or multiple source text lines and the geometric properties of the source text block comprises: a shape of the source text block and length difference(s) corresponding to the one or multiple source text lines. According to the technical solution, the image translation method is more efficient to adapt to different conditions.

Description

IMAGE TRANSLATION METHOD, IMAGE TRANSLATION APPARATUS ANDELECTRONIC DEVICETECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of image processing, and more specifically, to an image translation method, image translation apparatus and electronic device.BACKGROUND

[0002] Image translation technology allows converting text in images from one language to another through layout analysis, text recognition, and machine translation. This is useful for multilingual learning, international travel, and business communication.

[0003] However, conventional methods to perform image translation is not efficient.SUMMARY

[0004] This present disclosure provides an image translation method, image translation apparatus and electronic device, which can be more efficient to adapt to different conditions.

[0005] According to a first aspect, an embodiment of this application provides an image translation method, including: obtaining a source image, where the source image includes a source text block with one or multiple source text lines; generating a target image based on geometric properties of the source text block, where the target image includes one or multiple target text lines translated from the one or multiple source text lines and the geometric properties of the source text block includes: a shape of the source text block and length difference(s) corresponding to the one or multiple source text lines.

[0006] According to the above-mentioned technical solution, the target image is generated based on the geometric properties of the source text block, which may be more efficient to adaptto different conditions.

[0007] In one optional implementation, if the shape of the source text block is straight and / or the length difference(s) corresponding to the one or multiple source text lines is bigger than a first threshold, the generating a target image includes: generating one or multiple target line polygons based on one or multiple source line polygons, where the one or multiple source line polygons correspond to the one or multiple source text lines and the one or multiple target line polygons correspond to the one or multiple target text lines; generating the target image based on the one or multiple target line polygons.

[0008] According to the above-mentioned technical solution, if the shape of the source text block is straight and / or the length difference(s) corresponding to the one or multiple source text lines is bigger than a first threshold, the accurate boundary lines of the source line polygons are used in the generation of the target line polygons. The target line polygons are generated based on the source line polygons.

[0009] In one optional implementation, different target line polygons correspond to different rotation angles.

[0010] In one optional implementation, a rotation angle corresponds to a target line polygon is associated with a position of the target line polygon, an angle between the top boundary line and the bottom boundary line of the source text block and the number of the target line polygons.

[0011] The position of the target line polygon is a position relative to the top line polygon, which can be indicated by an index. According to the above-mentioned technical solution, different target line polygons in a block can be rotated with different rotation angles. In other words, different target text lines are rotated with different angles. The display effect can remain similar to the source image and there is less possibility for target text lines of different block to overlap with each other.

[0012] In one optional implementation, if the shape of the source text block is curved, the generating the target image based on the one or multiple target line polygons includes: dewarping the one or multiple target line polygons to one or multiple target line rectangles; fitting the one or multiple target text lines to the one or multiple target line rectangles; rewarping the one or multiple target line rectangles with the one or more target text lines to the one or multiple target line polygons.

[0013] According to the above-mentioned technical solution, the target line polygons can be dewarped then re-warped back and display effect remains before and after translation.

[0014] In one optional implementation, an offset corresponding to a target line polygon relative to the top target line polygon (the first target line polygon) is associated with a position of the target line polygon, a height of each target text line and a spacing between adjacent target text lines.

[0015] According to the above-mentioned technical solution, the position of the target line polygon can be indicated by such as an index of the target line polygon. The target line polygons can be generated by offsetting the top target line polygon.

[0016] In one optional implementation, the height of each target text line and the spacing between adjacent target text lines are associated with a height of each source text line and a spacing between adjacent source text lines.

[0017] According to the above-mentioned technical solution, the height of each source text line can be obtained by measuring average height of the source text lines in the source text block and the spacing between adjacent source text lines can be obtained by measuring average spacing between adjacent source text lines in the source text block. And the height of each target text line and the spacing between adjacent target text lines can be obtained based on the measured average height of source text lines and spacing between adjacent source text lines. For example, a ratio of the height of each target text line and spacing between the adjacent target text lines can remain the same with a ratio of the average height of the source text lines and the average spacing between adjacent source text lines.

[0018] In one optional implementation, the one or multiple target line polygons are generated based on a left boundary line and a right boundary line corresponding to the source text block, where the left boundary line and the right boundary line are generated based on the one or multiple source line polygons.

[0019] According to the above-mentioned technical solution, the left boundary line and the right boundary line can be used to fit the target line polygons into the original boundary area and the generated target line polygons will not overlap with other elements in the target image.

[0020] In one optional implementation, if the shape of the source text block is curved and the length difference(s) corresponding to the one or multiple source text lines is equal to orsmaller than a first threshold, the generating the target image includes: dewarping a text polygon to a text rectangle, where the text polygon corresponds to the source text block; generating one or multiple target line rectangles inside the text rectangle; fitting the one or multiple target text lines into the one or multiple target line rectangles; re-warping the text rectangle with the one or multiple text lines to the text polygon.

[0021] According to the above-mentioned technical solution, if the shape of the source text block is curved and the length difference(s) corresponding to the one or multiple source text lines is equal to or smaller than a first threshold, the source text block is relatively simple in shape. As the lengths are same for different source text lines, the target text lines can also have a same length. By dewarping and re-warping the text polygon, the target text lines can be rendered into a curved shape.

[0022] In one optional implementation, the number of the one or multiple target line rectangles is associated with a height of each source text line and a spacing between adjacent source text lines.

[0023] According to the above-mentioned technical solution, the number can be determined based on methods such as binary search.

[0024] According to a second aspect, an embodiment of this application provides an image translation apparatus, including: an obtaining unit, configured to obtain a source image, where the source image includes a source text block with one or multiple source text lines; a processing unit, configured to generate a target image based on geometric properties of the source text block, where the target image includes one or multiple target text lines translated from the one or multiple source text lines and the geometric properties of the source text block includes: a shape of the source text block and length difference(s) corresponding to the one or multiple source text lines.

[0025] In one optional implementation, if the shape of the source text block is straight and / or the length difference(s) corresponding to the one or multiple source text lines is bigger than a first threshold, the processing unit is configured to: generate one or multiple target line polygons based on one or multiple source line polygons, where the one or multiple source line polygons correspond to the one or multiple source text lines and the one or multiple target line polygons correspond to the one or multiple target text lines; generate the target image based onthe one or multiple target line polygons.

[0026] In one optional implementation, different target line polygons correspond to different rotation angles.

[0027] In one optional implementation, a rotation angle corresponds to a target line polygon is associated with a position of the target line polygon, an angle between the top boundary line and the bottom boundary line of the source text block and the number of the target line polygons.

[0028] In one optional implementation, if the shape of the source text block is curved, the processing unit is configured to: dewarp the one or multiple target line polygons to one or multiple target line rectangles; fit the one or multiple target text lines to the one or multiple target line rectangles; re- warp the one or multiple target line rectangles with the one or more target text lines to the one or multiple target line polygons.

[0029] In one optional implementation, an offset corresponding to a target line polygon relative to the top target line polygon is associated with an index of the target line polygon, a height of each target text line and a spacing between adjacent target text lines.

[0030] In one optional implementation, the height of each target text line and the spacing between adjacent target text lines are associated with a height of each source text line and a spacing between adjacent source text lines.

[0031] In one optional implementation, the one or multiple target line polygons are generated based on a left boundary line and a right boundary line corresponding to the source text block, where the left boundary line and the right boundary line are generated based on the one or multiple source line polygons.

[0032] In one optional implementation, if the shape of the source text block is curved and the length difference(s) corresponding to the one or multiple source text lines is equal to or smaller than a first threshold, the processing unit is configured to: dewarp a text polygon to a text rectangle, where the text polygon corresponds to the source text block; generate one or multiple target line rectangles inside the text rectangle; fit the one or multiple target text lines into the one or multiple target line rectangles; re- warp the text rectangle with the one or multiple text lines to the text polygon.

[0033] In one optional implementation, the number of the one or multiple target line rectangles is associated with a height of each source text line and a spacing between adjacentsource text lines.

[0034] According to a third aspect, an embodiment of this application provides an electronic device, including: a layout analysis module, configured to analyze a layout of a source image, where the layout of the source image includes a layout of a source text block and one or multiple source text lines in the source image; a text removal module, configured to remove the one or multiple source text lines for a prepared image; a translation module, configured to translate the one or multiple source text lines to one or multiple target text lines; a geometric layout preservation module, configured to generate the target image based on the prepared image, the one or multiple target text lines and the layout of the source text bock and the one or multiple source text lines.

[0035] According to a fourth aspect, an embodiment of this application provides a computer-readable storage medium, where the computer-readable storage medium stores instructions, and when the instructions run on a device, the device is enabled to perform the method according to the method in the first aspect or any optional implementation of the first aspect.

[0036] According to a fifth aspect, an embodiment of this application provides a computer program product, when the computer program product runs on a device, the device is enabled to perform the method according to the method in the first aspect or any optional implementation of the first aspect.

[0037] According to a sixth aspect, an embodiment of this application provides an chip system, including a memory and a processor, where the memory is configured to store a computer program, and the processor is configured to invoke the computer program from the memory and run the computer program, so that a device on which the chip system is disposed performs the method according to the method in the first aspect or any optional implementation of the first aspect.DESCRIPTION OF DRAWINGS

[0038] FIG.l is a schematic diagram of a hardware structure of an electronic device according to an embodiment of this application.

[0039] FIG. 2 is a block diagram of a software structure of the electronic device 100 according to an embodiment of this application.

[0040] FIG. 3 is a schematic diagram of an image translation method according to an embodiment of this application.

[0041] FIG. 4 is a schematic flowchart of a of a method to generate target line polygons according to an embodiment of this application.

[0042] FIG. 5 is a schematic diagram showing boundary points of a source line polygon.

[0043] FIG. 6 is a schematic diagram of generating a left boundary line and a right boundary line based on three source line polygons.

[0044] FIG. 7 is a schematic diagram of boundary lines of a source text bock.

[0045] FIG. 8 is a schematic diagram showing rotations of target line polygons according to an embodiment of this application.

[0046] FIG. 9 is a schematic diagram illustrating critical points and cross directions in different cases.

[0047] FIG. 10A and FIG. 10B is a schematic diagram illustrating fitting target line polygons to a left boundary line or a right boundary line.

[0048] FIG. 11 is a schematic diagram illustrating generation of target line polygons.

[0049] FIG. 12 is a schematic diagram illustrating the generation of the target image using a general rendering process.

[0050] FIG. 13 is a schematic diagram illustrating generating a text polygon with target text lines in a curved rendering process.

[0051] FIG. 14 is a schematic diagram showing generating a target image using a curved text rendering process.

[0052] FIG. 15 illustrates a process of generating a target image according to an embodiment of this application.

[0053] FIG. 16 is a schematic diagram of an electronic device according to an embodiment of this application.

[0054] FIG. 17 is a schematic bock diagram of an electronic device according to an embodiment of this application.DESCRIPTION OF EMBODIMENTS

[0055] The following describes the technical solutions in this application with reference to the accompanying drawings.

[0056] Terms used in the following embodiments of this application are merely intended to describe specific embodiments, but are not intended to limit this application. Terms “one”, “a”, “the”, “the foregoing”, “this”, and “the one” of singular forms used in this specification and the appended claims of this application are also intended to include plural forms like “one or more”, unless otherwise specified in the context clearly.

[0057] Reference to “an embodiment”, “some embodiments”, or the like described in this specification indicates that one or more embodiments of this application include a specific feature, structure, or characteristic described with reference to the embodiments. Therefore, in this specification, statements, such as “in an embodiment”, “in some embodiments”, “in some other embodiments”, and “in other embodiments”, that appear at different places do not necessarily mean referring to a same embodiment, instead, but mean “one or more but not all of the embodiments”, unless otherwise specified. The terms “include”, “comprise”, “have”, and their variants all mean “include but are not limited to”, unless otherwise specified.

[0058] In order to describe the image translation method provided by the embodiment of the present application more clearly, an electronic device for executing the image translation method is first introduced below.

[0059] In some embodiments, the electronic device may be a portable electronic device that further includes other functions such as a personal digital assistant function and / or a music player function, for example, a mobile phone, a tablet computer, or a wearable electronic device having a wireless communication function (for example, a smartwatch). An example embodiment of the portable electronic device includes but is not limited to a portable electronic device using HarmonyOS®, OpenHarmony®, iOS®, Android®, Microsoft®, or another operating system. The portable electronic device may alternatively be another portable electronic device, for example, a laptop computer (Laptop). It should be further understood that, in some other embodiments, the electronic device may alternatively be a desktop computer, but not a portable electronic device.

[0060] FIG. 1 illustrates a schematic diagram of a hardware structure of an electronic device 100 according to an embodiment of this application.

[0061] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) port 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identity module (subscriber identity module, SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, an optical proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

[0062] It may be understood that the structure shown in this embodiment of this application does not constitute a specific limitation on the electronic device 100. In some other embodiments of this application, the electronic device 100 may include more or fewer components than those shown in the figure, or combine some components, or split some components, or have different component arrangements. The components shown in the figure may be implemented by hardware, software, or a combination of software and hardware.

[0063] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a memory, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and / or a neural-network processing unit (neural-network processing unit, NPU). Different processing units may be independent components, or may be integrated into one or more processors.

[0064] The controller may be a nerve center and a command center of the electronic device 100. The controller may generate an operation control signal based on an instruction operation code and a time sequence signal, to complete control of instruction reading and instructionexecution.

[0065] A memory may be further disposed in the processor 110, and is configured to store instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that has been used or cyclically used by the processor 110. If the processor 110 needs to use the instructions or the data again, the processor may directly invoke the instructions or the data from the memory. This avoids repeated access, reduces waiting time of the processor 110, and improves system efficiency.

[0066] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (inter-integrated circuit, I2C) interface, an inter-integrated circuit sound (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous receiver / transmitter (universal asynchronous receiver / transmitter, UART) interface, a mobile industry processor interface (mobile industry processor interface, MIPI), a general-purpose input / output (general-purpose input / output, GPIO) interface, a subscriber identity module (subscriber identity module, SIM) interface, a universal serial bus (universal serial bus, USB) port, and / or the like.

[0067] The I2C interface is a two-way synchronization serial bus, and includes one serial data line (serial data line, SDA) and one serial clock line (serial clock line, SCL). In some embodiments, the processor 110 may include a plurality of groups of I2C buses. The processor 110 may be separately coupled to the touch sensor 180K, a charger, a flash, the camera 193, and the like through different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, to implement a touch function of the electronic device 100.

[0068] The I2S interface may be used for audio communication. In some embodiments, the processor 110 may include a plurality of groups of I2S buses. The processor 110 may be coupled to the audio module 170 through the I2S bus, to implement communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 may transmit an audio signal to the wireless communication module 160 through the I2S interface, to implement a function of answering a call through a Bluetooth headset.

[0069] The PCM interface may also be used for audio communication, and sample, quantize, and code an analog signal. In some embodiments, the audio module 170 may be coupled to the wireless communication module 160 through a PCM bus interface. In some embodiments, the audio module 170 may also transmit an audio signal to the wireless communication module 160 through the PCM interface, to implement a function of answering a call through a Bluetooth headset. Both the I2S interface and the PCM interface may be used for audio communication.

[0070] The UART interface is a universal serial data bus, and is used for asynchronous communication. The bus may be a two-way communication bus, and converts to-be-transmitted data between serial communication and parallel communication. In some embodiments, the UART interface is usually configured to connect the processor 110 to the wireless communication module 160. For example, the processor 110 communicates with a Bluetooth module in the wireless communication module 160 through the UART interface, to implement a Bluetooth function. In some embodiments, the audio module 170 may transmit an audio signal to the wireless communication module 160 through the UART interface, to implement a function of playing music through a Bluetooth headset.

[0071] The MIPI may be configured to connect the processor 110 to a peripheral component such as the display 194 or the camera 193. The MIPI includes a camera serial interface (camera serial interface, CSI), a display serial interface (display serial interface, DSI), and the like. In some embodiments, the processor 110 communicates with the camera 193 through the CSI, to implement a photographing function of the electronic device 100. The processor 110 communicates with the display 194 through the DSI, to implement a display function of the electronic device 100.

[0072] The GPIO interface may be configured by software. The GPIO interface may be configured as a control signal or a data signal. In some embodiments, the GPIO interface may be configured to connect the processor 110 to the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, or the like. The GPIO interface may alternatively be configured as an I2C interface, an I2S interface, a UART interface, an MIPI, or the like.

[0073] The USB port 130 is a port that conforms to a USB standard specification, and may be specifically a mini USB port, a micro USB port, a USB Type-C port, or the like. The USB iiport 130 may be configured to connect to a charger to charge the electronic device 100, or may be configured to transmit data between the electronic device 100 and a peripheral device, or may be configured to connect to a headset for playing audio through the headset. The port may be further configured to connect to another electronic device such as an AR device.

[0074] ft may be understood that an interface connection relationship between the modules illustrated in this embodiment of this application is merely an example for description, and constitutes no limitation on the structure of the electronic device 100. In some other embodiments of this application, the electronic device 100 may alternatively use an interface connection manner different from that in the foregoing embodiment, or use a combination of a plurality of interface connection manners.

[0075] The charging management module 140 is configured to receive a charging input from a charger. The charger may be a wireless charger or a wired charger. In some embodiments of wired charging, the charging management module 140 may receive a charging input of a wired charger through the USB port 130. In some embodiments of wireless charging, the charging management module 140 may receive a wireless charging input through a wireless charging coil of the electronic device 100. The charging management module 140 supplies power to the electronic device through the power management module 141 while charging the battery 142.

[0076] The power management module 141 is configured to connect to the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives an input of the battery 142 and / or the charging management module 140, to supply power to the processor 110, the internal memory 121, an external memory, the display 194, the camera 193, the wireless communication module 160, and the like. The power management module 141 may be further configured to monitor parameters such as a battery capacity, a battery cycle count, and a battery health status (electric leakage or impedance). In some other embodiments, the power management module 141 may alternatively be disposed in the processor 110. In some other embodiments, the power management module 141 and the charging management module 140 may alternatively be disposed in a same device.

[0077] A wireless communication function of the electronic device 100 may be implemented through the antenna 1, the antenna 2, the mobile communication module 150, thewireless communication module 160, the modem processor, the baseband processor, and the like.

[0078] The antenna 1 and the antenna 2 are configured to: transmit and receive an electromagnetic wave signal. Each antenna in the electronic device 100 may be configured to cover one or more communication frequency bands. Different antennas may be further multiplexed, to improve antenna utilization. For example, the antenna 1 may be multiplexed as a diversity antenna in a wireless local area network. In some other embodiments, the antenna may be used in combination with a tuning switch.

[0079] The mobile communication module 150 may provide a wireless communication solution that is applied to the electronic device 100 and that includes 2G / 3G / 4G / 5G or the like. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (low noise amplifier, LNA), and the like. The mobile communication module 150 may receive an electromagnetic wave through the antenna 1, perform processing such as filtering or amplification on the received electromagnetic wave, and transmit the electromagnetic wave to the modem processor for demodulation. The mobile communication module 150 may further amplify a signal modulated by the modem processor, and convert the signal into an electromagnetic wave for radiation through the antenna 1. In some embodiments, at least some functional modules in the mobile communication module 150 may be disposed in the processor 110. In some embodiments, at least some functional modules in the mobile communication module 150 may be disposed in a same device as at least some modules of the processor 110.

[0080] The modem processor may include a modulator and a demodulator. The modulator is configured to modulate a to-be-sent low-frequency baseband signal into a medium-high frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. Then, the demodulator transmits the low- frequency baseband signal obtained through demodulation to the baseband processor for processing. The low-frequency baseband signal is processed by the baseband processor and then transmitted to the application processor. The application processor outputs a sound signal through an audio device (which is not limited to the speaker 170A, the receiver 170B, or the like), or displays an image or a video through the display 194. In some embodiments, themodem processor may be an independent component. In some other embodiments, the modem processor may be independent of the processor 110, and is disposed in a same device as the mobile communication module 150 or another functional module.

[0081] The wireless communication module 160 may provide a wireless communication solution that is applied to the electronic device 100 and that includes a wireless local area network (wireless local area network, WLAN) (for example, a wireless fidelity (wireless fidelity, Wi-Fi) network), Bluetooth (Bluetooth, BT), a global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), a near field communication (near field communication, NFC) technology, an infrared (infrared, IR) technology, or the like. The wireless communication module 160 may be one or more components integrating at least one communication processing module. The wireless communication module 160 receives an electromagnetic wave through the antenna 2, performs frequency modulation and filtering processing on an electromagnetic wave signal, and sends a processed signal to the processor 110. The wireless communication module 160 may further receive a to-be-sent signal from the processor 110, perform frequency modulation and amplification on the signal, and convert the signal into an electromagnetic wave for radiation through the antenna 2.

[0082] In some embodiments, the antenna 1 and the mobile communication module 150 in the electronic device 100 are coupled, and the antenna 2 and the wireless communication module 160 in the electronic device 100 are coupled, so that the electronic device 100 can communicate with a network and another device by using a wireless communication technology. The wireless communication technology may include a global system for mobile communications (global system for mobile communications, GSM), a general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), wideband code division multiple access (wideband code division multiple access, WCDMA), time-division code division multiple access (time-division code division multiple access, TD-CDMA), long term evolution (long term evolution, LTE), BT, a GNSS, a WLAN, NFC, FM, an IR technology, and / or the like. The GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (global navigation satellite system, GLONASS), a BeiDou navigation satellite system (BeiDounavigation satellite system, BDS), a quasi-zenith satellite system (quasi-zenith satellite system, QZSS), and / or a satellite based augmentation system (satellite based augmentation system, SBAS).

[0083] The electronic device 100 may implement a display function through the GPU, the display 194, the application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display 194 and the application processor. The GPU is configured to: perform mathematical and geometric computation, and render an image. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0084] The display 194 is configured to display an image, a video, and the like. The display 194 includes a display panel. The display panel may be a liquid crystal display (liquid crystal display, LCD), an organic light-emitting diode (organic light-emitting diode, OLED), an activematrix organic light emitting diode (active-matrix organic light emitting diode, AMOLED), a flexible light-emitting diode (flexible light-emitting diode, FLED), a mini-LED, a micro-LED, a micro-OLED, a quantum dot light emitting diode (quantum dot light emitting diode, QLED), or the like. In some embodiments, the electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0085] The electronic device 100 may implement a photographing function through the ISP, the camera 193, the video codec, the GPU, the display 194, the application processor, and the like.

[0086] The ISP is configured to process data fed back by the camera 193. For example, during photographing, a shutter is pressed, and light is transmitted to a photosensitive element of the camera through a lens. An optical signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, to convert the electrical signal into a visible image. The ISP may further perform algorithm optimization on noise, brightness, and complexion of the image. The ISP may further optimize parameters such as exposure and a color temperature of a photographing scenario. In some embodiments, the ISP may be disposed in the camera 193.

[0087] The camera 193 is configured to capture a static image or a video. An optical image of an object is generated through the lens, and is projected onto the photosensitive element. Thephotosensitive element may be a charge coupled device (charge coupled device, CCD) or a complementary metal-oxide-semiconductor (complementary metal-oxide-semiconductor, CMOS) phototransistor. The photosensitive element converts an optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert the electrical signal into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard format such as RGB or YUV. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0088] The digital signal processor is configured to process a digital signal, and may process another digital signal in addition to the digital image signal. For example, when the electronic device 100 selects a frequency, the digital signal processor is configured to perform Fourier transform on frequency energy.

[0089] The video codec is configured to: compress or decompress a digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 may play or record videos in a plurality of coding formats, for example, moving picture experts group (moving picture experts group, MPEG)-1, MPEG-2, MPEG-3, and MPEG-4.

[0090] The NPU is a neural-network (neural-network, NN) computing processor, quickly processes input information by referring to a structure of a biological neural network, for example, by referring to a mode of transmission between human brain neurons, and may further continuously perform self-learning. Applications such as intelligent cognition of the electronic device 100 may be implemented through the NPU, for example, image recognition, facial recognition, speech recognition, and text understanding.

[0091] The external memory interface 120 may be used to connect to an external storage card, for example, a micro SD card, to extend a storage capability of the electronic device 100. The external storage card communicates with the processor 110 through the external memory interface 120, to implement a data storage function. For example, files such as music and videos are stored in the external storage card.

[0092] The internal memory 121 may be configured to store computer-executable program code. The executable program code includes instructions. The processor 110 runs the instructions stored in the internal memory 121, to perform various function applications of theelectronic device 100 and data processing. The internal memory 121 may include a program storage area and a data storage area. The program storage area may store an operating system, an application required by at least one function (for example, a voice playing function or an image playing function), and the like. The data storage area may store data (such as audio data and an address book) created during use of the electronic device 100, and the like. In addition, the internal memory 121 may include a high-speed random access memory, or may include a nonvolatile memory, for example, at least one magnetic disk storage device, a flash memory, or a universal flash storage (universal flash storage, UFS).

[0093] The electronic device 100 may implement an audio function, for example, music playing and recording, through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headset jack 170D, the application processor, and the like.

[0094] The audio module 170 is configured to convert digital audio information into an analog audio signal for output, and is also configured to convert analog audio input into a digital audio signal. The audio module 170 may be further configured to: code and decode an audio signal. In some embodiments, the audio module 170 may be disposed in the processor 110, or some functional modules in the audio module 170 are disposed in the processor 110.

[0095] The speaker 170A, also referred to as a "horn", is configured to convert an audio electrical signal into a sound signal. The electronic device 100 may be used to listen to music or answer a call in a hands-free mode over the speaker 170A.

[0096] The receiver 170B, also referred to as an "earpiece", is configured to convert an electrical audio signal into a sound signal. When a call is answered or speech information is received through the electronic device 100, the receiver 170B may be put close to a human ear to listen to a voice.

[0097] The microphone 170C, also referred to as a "mike" or a "mic", is configured to convert a sound signal into an electrical signal. When making a call or sending a voice message, a user may make a sound near the microphone 170C through the mouth of the user, to input a sound signal to the microphone 170C. At least one microphone 170C may be disposed in the electronic device 100. In some other embodiments, two microphones 170C may be disposed in the electronic device 100, to collect a sound signal and implement a noise reduction function. In some other embodiments, three, four, or more microphones 170C may alternatively bedisposed in the electronic device 100, to collect a sound signal, implement noise reduction, and identify a sound source, so as to implement a directional recording function and the like.

[0098] The headset jack 170D is configured to connect to a wired headset. The headset jack 170D may be a USB port 130, or may be a 3.5 mm open mobile terminal platform (open mobile terminal platform, OMTP) standard interface or cellular telecommunications industry association of the USA (cellular telecommunications industry association of the USA, CTIA) standard interface.

[0099] The pressure sensor 180 A is configured to sense a pressure signal, and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A may be disposed on the display 194. There are a plurality of types of pressure sensors 180A, such as a resistive pressure sensor, an inductive pressure sensor, and a capacitive pressure sensor. The capacitive pressure sensor may include at least two parallel plates made of conductive materials. When a force is applied to the pressure sensor 180A, capacitance between electrodes changes. The electronic device 100 determines pressure intensity based on the change in the capacitance. When a touch operation is performed on the display 194, the electronic device 100 detects intensity of the touch operation through the pressure sensor 180A. The electronic device 100 may also calculate a touch location based on a detection signal of the pressure sensor 180A. In some embodiments, touch operations that are performed at a same touch location but have different touch operation intensity may correspond to different operation instructions. For example, when a touch operation whose touch operation intensity is less than a first pressure threshold is performed on a Messages application icon, an instruction for viewing an SMS message is performed. When a touch operation whose touch operation intensity is greater than or equal to the first pressure threshold is performed on the Messages application icon, an instruction for creating a new SMS message is performed.

[0100] The gyro sensor 180B may be configured to determine a moving posture of the electronic device 100. In some embodiments, an angular velocity of the electronic device 100 around three axes (namely, axes x, y, and z) may be determined through the gyro sensor 180B. The gyro sensor 180B may be configured to implement image stabilization during photographing. For example, when the shutter is pressed, the gyro sensor 180B detects an angle at which the electronic device 100 jitters, calculates, based on the angle, a distance for which alens module needs to compensate, and allows the lens to cancel the jitter of the electronic device 100 through reverse motion, to implement image stabilization. The gyro sensor 180B may also be used in a navigation scenario and a somatic game scenario.

[0101] The barometric pressure sensor 180C is configured to measure barometric pressure. In some embodiments, the electronic device 100 calculates an altitude through the barometric pressure measured by the barometric pressure sensor 180C, to assist in positioning and navigation.

[0102] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 may detect opening and closing of a flip cover by using the magnetic sensor 180D. In some embodiments, when the electronic device 100 is a clamshell phone, the electronic device 100 may detect opening and closing of a flip cover based on the magnetic sensor 180D. Further, a feature such as automatic unlocking of the flip cover is set based on a detected opening or closing state of the flip cover.

[0103] The acceleration sensor 180E may detect accelerations of the electronic device 100 in various directions (usually on three axes). When the electronic device 100 is still, a magnitude and a direction of gravity may be detected. The acceleration sensor 180E may be further configured to identify a posture of the electronic device, and is used in an application such as switching between a landscape mode and a portrait mode or a pedometer.

[0104] The distance sensor 180F is configured to measure a distance. The electronic device 100 may measure the distance in an infrared manner or a laser manner. In some embodiments, in a photographing scenario, the electronic device 100 may measure a distance through the distance sensor 180F to implement quick focusing.

[0105] The optical proximity sensor 180G may include, for example, a light-emitting diode (LED) and an optical detector, for example, a photodiode. The light-emitting diode may be an infrared light-emitting diode. The electronic device 100 emits infrared light by using the lightemitting diode. The electronic device 100 detects infrared reflected light from a nearby object through the photodiode. When sufficient reflected light is detected, the electronic device 100 may determine that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 may determine that there is no object near the electronic device 100. The electronic device 100 may detect, by using the optical proximitysensor 180G, that the user holds the electronic device 100 close to an ear for a call, to automatically turn off a screen for power saving. The optical proximity sensor 180G may also be used in a smart cover mode or a pocket mode to automatically perform screen unlocking or locking.

[0106] The ambient light sensor 180L is configured to sense ambient light brightness. The electronic device 100 may adaptively adjust brightness of the display 194 based on the sensed ambient light brightness. The ambient light sensor 180L may also be configured to automatically adjust white balance during photographing. The ambient light sensor 180L may also cooperate with the optical proximity sensor 180G to detect whether the electronic device 100 is in a pocket, to avoid an accidental touch.

[0107] The fingerprint sensor 180H is configured to collect a fingerprint. The electronic device 100 may use a feature of the collected fingerprint to implement fingerprint-based unlocking, application lock access, fingerprint-based photographing, fingerprint-based call answering, and the like.

[0108] The temperature sensor 180J is configured to detect a temperature. In some embodiments, the electronic device 100 executes a temperature processing policy through the temperature detected by the temperature sensor 180J. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 lowers performance of a processor near the temperature sensor 180J, to reduce power consumption for thermal protection. In some other embodiments, when the temperature is less than another threshold, the electronic device 100 heats the battery 142 to prevent the electronic device 100 from being shut down abnormally due to a low temperature. In some other embodiments, when the temperature is less than still another threshold, the electronic device 100 boosts an output voltage of the battery 142 to avoid abnormal shutdown caused by a low temperature.

[0109] The touch sensor 180K is also referred to as a "touch panel" . The touch sensor 180K may be disposed on the display 194, and the touch sensor 180K and the display 194 constitute a touchscreen. The touch sensor 180K is configured to detect a touch operation performed on or near the touch sensor. The touch sensor may transfer the detected touch operation to the application processor to determine a type of the touch event. A visual output related to the touch operation may be provided through the display 194. In some other embodiments, the touchsensor 180K may also be disposed on a surface of the electronic device 100 at a location different from that of the display 194.

[0110] The bone conduction sensor 180M may obtain a vibration signal. In some embodiments, the bone conduction sensor 180M may obtain a vibration signal of a vibration bone of a human vocal-cord part. The bone conduction sensor 180M may also be in contact with a body pulse to receive a blood pressure beating signal. In some embodiments, the bone conduction sensor 180M may also be disposed in the headset, to obtain a bone conduction headset. The audio module 170 may obtain a speech signal through parsing based on the vibration signal that is of the vibration bone of the vocal-cord part and that is obtained by the bone conduction sensor 180M, to implement a speech function. The application processor may parse heart rate information based on the blood pressure beating signal obtained by the bone conduction sensor 180M, to implement a heart rate detection function.[oni] The button 190 includes a power button, a volume button, and the like. The button 190 may be a mechanical button, or may be a touch button. The electronic device 100 may receive a key input, and generate a key signal input related to a user setting and function control of the electronic device 100.

[0112] The motor 191 may generate a vibration prompt. The motor 191 may be configured to provide an incoming call vibration prompt and a touch vibration feedback. For example, touch operations performed on different applications (for example, photographing and audio playback) may correspond to different vibration feedback effects. The motor 191 may also correspond to different vibration feedback effects for touch operations performed on different areas of the display 194. Different application scenarios (for example, a time reminder, information receiving, an alarm clock, and a game) may also correspond to different vibration feedback effects. A touch vibration feedback effect may be further customized.

[0113] The indicator 192 may be an indicator light, and may be configured to indicate a charging status and a power change, or may be configured to indicate a message, a missed call, a notification, and the like.

[0114] The SIM card interface 195 is configured to connect to a SIM card. The SIM card may be inserted into the SIM card interface 195 or removed from the SIM card interface 195, to implement contact with or separation from the electronic device 100. The electronic device100 may support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 may support a nano-SIM card, a micro-SIM card, a SIM card, and the like. A plurality of cards may be simultaneously inserted into a same SIM card interface 195. The plurality of cards may be of a same type or different types. The SIM card interface 195 is compatible with different types of SIM cards. The SIM card interface 195 is also compatible with an external storage card. The electronic device 100 interacts with a network through the SIM card, to implement functions such as conversation and data communication. In some embodiments, the electronic device 100 uses an embedded SIM (embedded SIM, eSIM) card. The eSIM card may be embedded into the electronic device 100, and cannot be separated from the electronic device 100.

[0115] It should be understood that a calling card in embodiments of this application includes but is not limited to a SIM card, an eSIM card, a universal subscriber identity module (universal subscriber identity module, USIM), a universal integrated circuit card (universal integrated circuit card, UICC), and the like.

[0116] A software system of the electronic device 100 may use a layered architecture, an event-driven architecture, a microkernel architecture, a micro service architecture, or a cloud architecture. In an embodiment of this application, an Android system with a layered architecture is used as an example to describe a software structure of the electronic device 100.

[0117] FIG. 2 is a block diagram of a software structure of the electronic device 100 according to an embodiment of this application. In a layered architecture, software is divided into several layers, and each layer has a clear role and task. The layers communicate with each other through a software interface. In some embodiments, the Android system is divided into four layers: an application layer, an application framework layer, an Android runtime (Android runtime) and system library, and a kernel layer from top to bottom. The application layer may include a series of application packages.

[0118] As shown in FIG. 2, the application packages may include applications such as Camera, Gallery, Calendar, Phone, Map, Navigation, WLAN, Bluetooth, Music, Videos, and Messages.

[0119] The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for an application atthe application layer. The application framework layer includes some predefined functions.

[0120] As shown in FIG. 2, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0121] The window manager is configured to manage a window program. The window manager may obtain a size of the display, determine whether there is a status bar, perform screen locking, take a screenshot, and the like.

[0122] The content provider is configured to: store and obtain data, and enable the data to be accessed by an application. The data may include a video, an image, an audio, calls that are made and answered, a browsing history and bookmarks, an address book, and the like.

[0123] The view system includes visual controls such as a control for displaying a text and a control for displaying an image. The view system may be configured to construct an application. A display interface may include one or more views. For example, a display interface including an SMS message notification icon may include a text display view and an image display view.

[0124] The phone manager is configured to provide a communication function for the electronic device 100, for example, management of a call status (including answering, declining, or the like).

[0125] The resource manager provides various resources such as a localized character string, an icon, an image, a layout file, and a video file for an application.

[0126] The notification manager enables an application to display notification information in a status bar, and may be configured to convey a notification message. The notification manager may automatically disappear after a short pause without requiring user interaction. For example, the notification manager is configured to: notify download completion, give a message notification, and the like. The notification manager may alternatively be a notification that appears in a top status bar of the system in a form of a graph or a scroll bar text, for example, a notification of an application that is run on the background, or may be a notification that appears on the screen in a form of a dialog window. For example, text information is displayed in the status bar, an announcement is given, the electronic device vibrates, or the indicator light blinks.

[0127] The Android runtime includes a kernel library and a virtual machine. The Android runtime is responsible for scheduling and management of the Android system.

[0128] The kernel library includes two parts: a function that needs to be invoked in Java language and a kernel library of Android.

[0129] The application layer and the application framework layer run on the virtual machine. The virtual machine executes Java files of the application layer and the application framework layer as binary files. The virtual machine is configured to implement functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0130] The system library may include a plurality of functional modules, for example, a surface manager (surface manager), a media library (media library), a three-dimensional graphics processing library (for example, OpenGL ES), and a 2D graphics engine (for example, SGL).

[0131] The surface manager is configured to: manage a display subsystem and provide fusion of 2D and 3D layers for a plurality of applications.

[0132] The media library supports playback and recording in a plurality of commonly used audio and video formats, and static image files. The media library may support a plurality of audio and video coding formats, for example, MPEG-4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0133] The three-dimensional graphics processing library is configured to implement three- dimensional graphics drawing, image rendering, composition, layer processing, and the like.

[0134] The 2D graphics engine is a drawing engine for 2D drawing.

[0135] The kernel layer is a layer between hardware and software. The kernel layer includes at least a display driver, a camera driver, an audio driver, and a sensor driver.

[0136] It should be understood that the technical solutions in embodiments of this application may be applied to systems such as Android, iOS, and Harmony.

[0137] FIG. 3 is a schematic diagram of an image translation method according to an embodiment of this application.

[0138] S310, obtain a source image, where the source image includes a source text block with one or multiple source text lines.

[0139] S320, generate a target image based on geometric properties of the source text block, where the target image includes one or multiple target text lines translated from the one or multiple source text lines and the geometric properties of the source text block includes: a shape of the source text block and length difference(s) corresponding to the one or multiple source text lines.

[0140] In S310, the source image may be an image taken by a camera or a photography unit of the electronic device of FIG. 1. In some embodiments, the source image may be a real-time image captured by the photography unit of the electronic device. In some other embodiments, the source image may be a frame from a video.

[0141] There may be one or multiple source text blocks in the source image and for easy explanation, we focus on one source text block in the following. The source text block can be obtained based on a layout analysis process.

[0142] The layout analysis process is performed to analyze the structure and arrangement of elements in an image. Based on the layout analysis, locations and regions of text, non-text elements such as sub-images, icons or shapes can be identified using neural networks or other object detecting methods.

[0143] The text regions may be further divided into blocks or lines and relationships between different blocks such as relative positions and orientations can be analyzed. The layout analysis process may also employ OCR techniques to recognize the orientation and the alignment of text, for example, the text may be aligned horizontal or vertical.

[0144] The source text block includes one or multiple source text lines (the one or multiple source text lines may also be referred to as a source text) in a source language, which needs to be translated into one or multiple target text lines in a target language (for simplicity, assuming there are multiple source text lines and target text lines). The source language and the target language may be set by the user or determined by the electronic device based on other information. For example, the source language is determined based on optical character recognition (OCR) result on the source image and the target language is determined to be the electronic device’s system language or determined based on the user’s customs or historical operation records.

[0145] In some embodiments, the source text block may correspond to more than onelanguage and the source text lines may need to be translated into a language different from all of the more than one language. In some other embodiments, the source text block may correspond to more than one language and the target language is one of the more than one language, only part of the one or multiple source text lines need to be translated to the target language. For ease of understanding, assuming there is one source language.

[0146] In S320, the target image is generated based on the geometric properties of the source text block.

[0147] The shape of the source text block can be straight or curved and the length difference(s) corresponding to the one or multiple source text lines can be obvious (i.e. bigger than a first threshold) or not (i.e. equal to or smaller than a first threshold).

[0148] In some embodiments, to obtain the geometric properties of the source text block, one or multiple source line polygons which indicate boundaries of the one or multiple source text lines are also obtained from the source text block based on the layout analysis process. The geometric properties of the source text block are represented by the geometric properties of the one or multiple source line polygons.

[0149] For example, top lines of the source line polygons could be used to determine the shape of the source text block and the length difference(s) corresponding to the one or multiple source text lines.

[0150] The curved or straight top lines of the source line polygons can indicate the shape of the source text block.

[0151] For one example, if a ratio of the minimum length of the top lines and the maximum length of the top lines are obvious, e.g. smaller than a ratio threshold (e.g. 0.7, 0.8, 0.9 or other values), the length difference(s) corresponding to the one or multiple source text lines can be regarded as obvious or bigger than the first threshold. Correspondingly, if the ratio of the minimum length of the top lines and the maximum length of the top lines are equal to or bigger than the ratio threshold, the length difference(s) corresponding to the one or multiple source text lines can be regarded as not obvious or smaller than the first threshold.

[0152] It is possible that bottom lines of the source line polygons could be used to indicate the geometric properties of the source text block. Details could be deduced from the top line example and are omitted for brevity.

[0153] In other embodiments, a ratio of the maximum length of the top lines and the minimum length of the top lines can be used to indicate the length difference(s) corresponding to the one or multiple source text lines. Optionally, the shape of the source text block such as the shape of the left boundary line or right boundary line of the source text block or other methods can be used to estimate the length difference(s) corresponding to the one or multiple source text lines.

[0154] Based on the geometric properties of the source text block, different steps could be performed to generate the target image.

[0155] In a first implementation, if the shape of the source text block is straight and / or the length difference(s) corresponding to the one or multiple source text lines is bigger than a first threshold, the generating the target image includes: generating one or multiple target line polygons based on one or multiple source line polygons, where the one or multiple source line polygons correspond to the one or multiple source text lines and the one or multiple target line polygons correspond to the one or multiple target text lines; generating the target image based on the one or multiple target line polygons.

[0156] FIG. 4 illustrates a schematic diagram of a method to generate the target line polygons.

[0157] S410, generate the left boundary line, the right boundary line, the top boundary line and the bottom boundary line of the source text block based on the one or multiple source line polygons.

[0158] The boundary lines of the source text block are also the boundary lines of the target text block. They can be generated based on the boundary lines of the source line polygons, which can be represented by multiple boundary points corresponding to each source line polygon. The boundary points include comer points. For example, if a source line polygon is a rectangle, there are four comer points. In some embodiments, for a finer granularity, more boundary points could be determined which may arrange at intervals on the boundary line of a source line polygon. For example, the distance between adjacent boundary points may be several pixels.

[0159] Fig. 5 illustrates boundary points of a source line polygon. Points A to D are corner points and points E to H are not comer points and the distance between the boundary points Aand E, E and F, F and B, C and G, G and H, H and D may be the same.

[0160] In one implementation, the left boundary line and the right boundary line can be generated using comer points of the source line polygons. In fact, comer points on the left boundary line and right boundary line of the source line polygons are used to generate the left boundary line and the right boundary line of the source text block respectively. The corner points on the left boundary line and the right boundary line of the source line polygons are denoted as left comer points and right comer points, respectively.

[0161] As a schematic explanation, SI to S3 could be performed to generate the left boundary line of the source text block.

[0162] SI, the left comer points of the first source line polygon can be included in the left boundary line of the source text block. If the first source line is the only source text line (source text block contains a single source text line) we have to include at least the first and the last left comer points.

[0163] We could denote the source line polygons from the top to the bottom as the first source line polygon, the second source line polygon, ..., the K-th source line polygon, K is a positive integer, the K-th source line polygon can also be denoted as the last source line polygon. Left comer points on the left boundary line of the first source line polygon to the last source line polygon are considered successively for the creation of the left boundary line of the source text block. The source line polygon being considered can be referred to as a current line polygon and the corner point being considered can be referred to as a current point.

[0164] Referring to FIG. 6, for the first source line polygon 610, left comer points A and C are included in the left boundary line of the source text block.

[0165] S2, add the current point to the left boundary line if a first condition is fulfilled.

[0166] The first condition is: an angle between a current downward direction and a current direction is smaller than 7i / 2, where the current downward direction is the current line direction rotated clockwise by / 2, and the current direction is the direction from the last boundary point (the last point added to the left boundary line of the source text block) to the current point. The current line direction is the direction from the first point to the last point of the current source line polygon’s top line. If the first condition is not fulfilled, the current point will be skipped.

[0167] For example, the source line polygon being considered is the source line polygon620 and the current source line polygon is the source line polygon 620. Candidate points for the left boundary line are points E and G. For the current point E, the last boundary point is pointC, the current line direction is EF, the current downward direction is EG and the current> > ) direction is CE. As the angle between EG and GE is smaller than TT / 2, the current point E can be included in the left boundary line. For the current point G, point E becomes the last boundary point, the current line direction is EF, the current downward direction is EG and the current direction is EG. As the angle between EG and EG is smaller than / 2, the current point G is included in the left boundary line of the source text block.

[0168] S3, repeat S2 for the left comer points of the following source line polygon until all the source line polygons are considered.

[0169] For example, the source line polygon being considered is the source line polygon 630 and the current source line polygon is the source line polygon 630. Candidate points for the left boundary line are points I and K. For the current point I, the last boundary point is the point)B)G, the current line direction is I], the current downward direction is IK and the current direction is G I. As the angle between GI and IK is smaller than 7i / 2, the current point I can be included in the left boundary line.

[0170] For other points and other source line polygons, similar steps are performed to determine whether to include corresponding left comer points in the left boundary line of the source text block.

[0171] It is to be noted that for the right boundary line, details can be deduced from the left boundary line description and are not repeated here for brevity.

[0172] FIG. 6 also shows the left boundary line 640 and the right boundary line 650 of the source text block based on the source line polygons 610 to 630.

[0173] In one implementation, the top boundary line can be the top line of the first source line polygon and the bottom boundary line can be the bottom line of the last source line polygon if a second condition is fulfilled. Otherwise, use the first and last points of the left boundary line and the right boundary line to create the top boundary line and the bottom boundary line. For example, the line segment connecting the first points of the left boundary line and the rightboundary line can be used as the top boundary line, the line segment connecting the last points of the left boundary line and the right boundary line can be used as the bottom boundary line.

[0174] The second condition is: the first and the last points of the top line of the first source line polygon and the first and the last points of the bottom line of the last source line polygon belong to the left boundary line and the right boundary line of the source text block.

[0175] Referring to FIG. 6, the first and the last points of the top line of the first source line polygon 610 and the last source line polygon 630 belong to the left boundary line 640 and the right boundary line 650. The top line AC of the first source line polygon 610 is used as the top boundary line 660 and the bottom line KL of the last source line polygon 630 is used as the bottom boundary line 670.

[0176] S420, estimate the number of target line polygons.

[0177] In this application, the number of target line polygons can be calculated using a height of the source text block, a spacing between adjacent target text lines, a height of each target text line.

[0178] The height of the source text block can be determined using the top boundary line and the bottom boundary line, which can be identified as aforementioned in S410.

[0179] The height of each target text line and the spacing between adjacent target text lines are associated with a height of each source text line and a spacing between adjacent source text lines. In one example, for consistency of display effect of the source image and the target image, a ratio of the spacing between adjacent target text lines and the height of each target text line can be the same as a ratio of a spacing between adjacent source text lines and the height of each source text line.

[0180] The spacing between adjacent target text lines and the height of the target text line can be determined using methods such as binary search. The binary search aims to find the optimal height of the target text lines so that the target text lines can fit the region occupied by the source text lines.

[0181] The number of the target line polygons N can be calculated using the following formula:

[0182] N= (H1+S) / (H2+S), where Hl is the height of the source text block, S is the spacing between adjacent target text lines, H2 is the height of each target text line.

[0183] S430, generate target line polygons that fit the boundary lines of the source text block.

[0184] The top boundary line of the source text block can be used as the top line of the first target line polygon. Bottom line of the first target line polygon, top lines and bottom lines of the other target line polygons can be generated by offsetting the points on the top line of the first target line polygon.

[0185] To determine the offset for other lines, a current point direction and a downward direction can be defined. The current point direction is defined as the direction from the current point to the next point and by rotating the current direction clockwise by n / 2, the downward direction is obtained.

[0186] In one implementation, an offset corresponding to a target line polygon relative to the top target line polygon is associated with a position of the target line polygon, a height of each target text line and a spacing between adjacent target text lines. In other words, distance of corresponding points on the top boundary line and on the top line of the target line polygon is associated with a position of the target line polygon, a height of each target text line and a spacing between adjacent target text lines. For example, the offset between the current point on the top boundary line (i.e., the top line of the first target line polygon) and a corresponding point on top line of a target line polygon with an index i#, denoted as current_offset_i, conform to the following formula:

[0187] current_offset_i= i*(H2+S)

[0188] where H2 is the height of each target text line and S is the spacing between adjacent target text lines, from the top target line polygon the bottom target line polygon, the corresponding indexes are 0, 1, ..., N-l, and are denoted as target line polygon 0#, target line polygon 1#, ..., target line polygon (N-l)#, N is the number of the target line polygons.

[0189] And an offset between the corresponding points on the top line and the bottom line of one target line polygon is H2.

[0190] Based on the current point direction and the downward direction, a current point angle (0) is defined as the angle between the current point direction and the direction of the vector (1,0) and a downward angle is (0-JC / 2). Assuming the coordinate of the current point on the top line of the first target line polygon is (x, y), coordinates of the points corresponding tothe current point on the top line and bottom line of the target line polygon i# follows the following formula:

[0191] top_x=x+current_offset_i * cos(9-7i / 2)

[0192] top_y= y+current_offset_i*sin(0-7t / 2)

[0193] bot_x= top_x +H2*cos(0-7i / 2)

[0194] bot_y= top_y +H2*sin(0-7i / 2)

[0195] where top_x and top_y are the x-coordinate and y-coordinate of corresponding point on the top line of the target line polygon i# and bot_x and bot_y are the x-coordinate and y- coordinate of corresponding point on the bottom line of the target line polygon i#.

[0196] FIG. 7 shows a schematic diagram of boundary line of the source text block.

[0197] The boundary line of the source text block includes: the top boundary line AKLMB, which is also used as the top line of the first target line polygon, the bottom boundary line INOPJ, the left boundary line ACFI and the right boundary line BEDGHJ. To generate the bottom line of the first target line polygon, and top lines and bottom lines of the other target line polygons, points on the top line of the first target line polygon (A, K, L, M, B) are offset.

[0198] Assuming the current point is K, the current point direction is KL, the downward direction is perpendicular to the current point direction. Corresponding points to the point K can be found by offsetting K by different values. For example, the offset for corresponding point on the bottom line of the first target line polygon K' is H2, the offsets for corresponding points on the top line and the bottom line of the target line polygon i# are i*(H2+S) and i*(H2+S)+H2, respectively. In other words, the distance between K and K', K and K_i, K and K'_i are H2, i*(H2+S) and i*(H2+S)+H2, respectively.

[0199] By going through all the points on the top line of the first target line polygon, corresponding points of other target line polygons are determined and all the target line polygons can be generated using the points.

[0200] In some embodiments, if there are two or more target line polygons, different target line polygons correspond to different rotation angles and a rotation angle to each target line polygon can be calculated. For example, the source line polygons may correspond to different rotation angles and to remain similar display effect, target line polygons can also be rotated.

[0201] In one implementation, a rotation angle to a target line polygon is associated with a position of the target line polygon, an angle between the top boundary line and the bottom boundary line of the source text block.

[0202] The position of the target line polygon can be its position relative to the top target line polygon, which can be indicated by an index. As aforementioned, from the top to the bottom, the target line polygons are denoted as target line polygon 0#, target line polygon 1#, ..., target line polygon (N-l)#. The rotation angle of the target line polygon i# relative to the first target line polygon (target line polygon 0#) A_i can be calculated using the following formula:

[0203] A_i= i*A0 / (N-l), where A0 is the angle between the top boundary line and the bottom boundary line of the source text block.

[0204] FIG. 8 is a schematic diagram showing the rotation of target line polygons according to an embodiment of this application.

[0205] Top lines of three target line polygons, ACDEB, A'C'D'E'B' and A"C"D"E"B" and the bottom boundary line (of the source text block corresponding to the three target line polygons) FGH are shown in FIG. 8 for simplicity. The direction of a line can be the direction from its first point to its last point. The direction of the top line of the first target line polygon ACDEB can be denoted as AB, and the direction of the bottom boundary line FGH can be denoted as FH. A0 is the angle between the top line of the first target line polygon AB and the bottom boundary line FH. The rotation angle of the second target line polygon and the third target line polygon relative to the first target line polygon can be AO / 2 and A0, respectively. In other words, the angle between the top lines of the first target line polygon and the second target line polygon is AO / 2, that is, the angle between AB and A'B' is AO / 2. The angle between the top lines of the first target line polygon and the third target line polygon is A0, that is, the angle between AB and A"B" is A0. The first target line polygon has the same direction as the top boundary line and the last target line polygon has the same direction as the bottom boundary line. For the rotation of each target line polygon, the rotation center can be set as for example, the first point of the corresponding top line.

[0206] It is to be noted that the points generated by offset (or further rotated) can becandidate points for the target line polygons. For a better display effect, all the points of the target line polygons are inside the boundary area limited by the top boundary line, the bottom boundary line, the left boundary line and the right boundary line, the points generated by offsetting needs to be fitted to the boundary lines. In other words, the target line polygons are generated based on the boundary lines.

[0207] For example, critical points among the candidate points needs to be processed to fit the boundary lines of the source text block. In the following discussion about the fitting of critical points to the boundary lines, the candidate points generated by offset are referred to as “point” for simplicity.

[0208] Critical points are candidate points of the top lines and the bottom lines of the target line polygons. And the critical points are near the boundary lines of the source text block and are fitted to the left boundary line and the right boundary line of the source text bock.

[0209] A critical point of this application can be defined as a point that is outside the boundary area (of the source text block) limited by the four boundary lines and there is a point (if exists) next to the critical point that is inside the boundary area. If at the left side or the right side there is no point outside the boundary area, the end point at the side can be considered the critical point. (In some embodiments, a point that belongs to the boundary line or is extremely near the boundary line can be considered as a critical point, which will be detailed in the following) There are at most two critical points for the top line or the bottom line of a target line polygon. If the i-th point is outside the boundary area on the left side and the (i+ 1 )-th point is inside the boundary area, the i-th point can be considered as a critical point, which can be considered as a left critical point. For the right critical point, if the i-th point is inside the boundary area and the (i+ 1 )-th point is outside the boundary area on the right side, the (i+l)-th point can be considered as a critical point. And for a top line or a bottom line of a target line polygon, from the left to the right, the candidate points are referred to as the first point, the second point, ..., the i-th point, ...the M-th point, where M is the number of candidate points for the top line or the bottom line.

[0210] Cross direction is defined as the direction related to the critical points and the cross of the left boundary line or the right boundary line. If the critical point is an end point, the cross direction is defined as parallel to the direction of the line segment connecting the critical pointand the adjacent point. It is to be noted that in this application, the cross direction can be defined as a direction from the critical point or to the critical point. For example, if the line segment between points A and B intersects with the left boundary line or the right boundary line, it is not matter whether we define the AB or BA as the cross direction.

[0211] FIG. 9 is a schematic diagram illustrating critical points and cross directions in different cases.

[0212] Referring to FIG. 9, based on the top boundary line AiBiCiDiEiFi(i.e., the top line of the first target line polygon), several lines (top lines or bottom lines) for other target line polygons are generated, which are A2B2C2D2E2F2, A3B3C3D3E3F3 and A4B4C4D4E4F4. The generated lines need to be fitted to the left boundary line 910 and the right boundary line 920.

[0213] For the line A2B2C2D2E2F2, A2 is a point outside the boundary area and the point next to A2 is inside the boundary area (B2), therefore A2 is considered as a critical point. At the right side there is no point outside the boundary area and the end point F2 is considered as a critical point. Direction A2B2is related to the critical point A2 and the A2B2is related to the cross of the left boundary line 910 and is considered as the cross direction for A2. E2F2is related to the critical point F2 and since F2 is the end point, E2F2is considered as the cross direction for F2.

[0214] Similarly, For the line A3B3C3D3E3F3, A3 and F3 are critical points and corresponding cross directions are .4353and E3F3, respectively. For the line A4B4C4D4E4F4, there is only one critical point E4 for the right boundary line 920 and corresponding cross directions is D E4. It is to be noted for the point C which is on the boundary line or is extremely near the boundary line, it may also be considered as a critical point in some embodiments. Then for the line A4B4C4D4E4F4, C4 and E4 are critical points and corresponding cross directions are C4£)4and D4E4, respectively.

[0215] To fit the target line polygons to the left boundary line 910 or the right boundary line 920, the critical points are replaced with the intersection points of the cross direction and the corresponding right boundary line 920 or the left boundary line 910.

[0216] FIG. 10 is a schematic diagram illustrating fitting a target line polygon to the left boundary line or the right boundary line. The top boundary line, the bottom boundary line, the left boundary line and the right boundary line are AKLMB, INOP J, ACFI and BEDGHJ, respectively.

[0217] Referring to FIG. 10A, point A' is the end point of a top line or a bottom line of a target line polygon generated by offset and is considered as a critical point. The cross direction is parallel to the direction of AK . The intersection point of the cross direction and the corresponding left boundary line is point A". And the critical point A' is replaced with the intersection point A". Referring to FIG. 10B, point B' is outside the boundary area and the point next to B' (M') is inside the boundary area, The B' is the critical point and can be replaced with the point B".

[0218] In some embodiments, for the fitting of the target line polygons to the left boundary line or the right boundary line, the critical points may not be defined as aforementioned, but be defined as the first point inside the boundary area from the left end point or the right end point. For example, if there is no point outside the boundary area at one side, the end point at that side can be considered as the critical point; otherwise, If the i-th point is outside the boundary area on the left side and the (i+l)-th point is inside the boundary area, the (i+l)-th point can be considered as a critical point, which can be considered as a left critical point. For the right critical point, if the i-th point is inside the boundary area and the (i+1 )-th point is outside the boundary area on the right side, the i-th point can be considered as a critical point. Also referring to FIG. 9, for the line A2B2C2D2E2F2, point B2 is inside the boundary area and the point next to B2 (A2) is outside the boundary area and B2 can be considered as the critical point. F2 as the right end point can be considered as the critical point. The cross direction can also be defined as a direction related to the critical point and the cross of the corresponding left boundary line or the right boundary line. The cross direction is parallel to the direction of A2B2. And the point B2is replaced with the intersection point of cross direction and the left boundary line (G).

[0219] The replacement when defining the critical point as a point outside the boundary area or inside the boundary area actually has a same fitting effect. And some replacement will make the shrinking of corresponding target line polygon and some replacement will make theexpanding of the corresponding target line polygon. The target line polygon fitted to the boundary lines in FIG. 1 OB result in a shrinking of the target line polygon. However, for the case in FIG. 10A, the replacement results in an expanding of corresponding target line polygon.

[0220] It is to be noted that it is no matter whether the top line or bottom line are rotated like FIG. 8. If there is a rotation of corresponding line, the cross direction will also be changed accordingly (rotated correspondingly).

[0221] The critical points are fitted to the boundary line and other points that are out of the boundary area are discarded for the target line polygons. To check if a point is inside the boundary area, if a current point direction from the point intersects the left boundary line and the right boundary line and if a downward direction from the point intersects the top boundary line and the bottom boundary line can be checked. If the answer is yes, the point can be considered as inside the boundary area. In more complex scenarios with curved text lines we might not have all 4 intersections even if the point is inside the bounding area. To check if the point is inside or outside the bounding area we can use the information on where the point is located relative to the intersection point with the boundary line, e.g. the point is outside the boundary area if it is located to the left / top / right / bottom of the left / top / right / bottom boundary. Also, the point is outside the boundary area if there are no intersections with the boundary lines.

[0222] Also, there are other methods to determine whether a point is inside the boundary area, for example, we can use the coordinate of the point and corresponding points on the boundary lines. Details will be omitted here for brevity.

[0223] Based on previous discussion, the method for generating the target line polygons are introduced. Next, we will discuss how to generate the target image using the generated target line polygons.

[0224] If the shape of the source text block is curved (the target line polygons are curved), to generate the target image, the target line polygons are dewarped to target line rectangles first; then the target text lines are fitted into the target line rectangles; finally, the target line rectangles with the target text lines are re-warped into target line polygons (with target text lines). It is to be noted that although boundary of the line polygons, line rectangles are shown in diagrams of this application for better explaining how to arrange the target text lines, the boundaries may be invisible in the source image or the target image.

[0225] In this application, the process to generate target line polygons and combine target line polygons with target text lines to generate the target image in the first implementation can be referred to as a general rendering process. FIG. 11 is a schematic diagram illustrating generation of target line polygons with target text lines in the general process when the shape of the source text block is curved. After the dewarping, fitting and re- warping process, the target text lines are rendered into the target line polygons.

[0226] If the shape of the source text block is straight, the target line polygons are straight and can be rectangles. The target text lines are directly fitted into the target line polygons.

[0227] In this application, before generating the target image, a text removal process can be performed. The text removal process is performed to remove source text lines from the source image before the target text lines is added. The process can be performed based on a result of the layout analysis process. To remove the source text lines, several approaches can be performed. For example, the text region can be filled with neighboring pixels to generate plausible background content. For another example, the text region can be hided using a mask. For another example, the text region is replaced with a pattern or background. The source image after a text removal process may be denoted as a prepared image.

[0228] Also, a translation process can be performed to generate the target text lines. The translation process is also performed based on a result of the layout analysis process. The source text lines are translated into target text lines. The translation can be performed using neural networks or based on particular algorithms or a combination of both.

[0229] FIG. 12 is a schematic diagram illustrating the generation of the target image using the general rendering process. As shown in FIG 12, the source line polygons, the prepared image 1201 (source image with removal of the source text lines) and the target text lines are used in the general rendering process. The prepared image includes a sub-image.

[0230] In S 1210, boundary lines 1211 are created based on the source line polygons of the source text lines 1212. In SI 220, target line polygons 1221 are generated inside the boundary lines. Rotation of the target line polygons may be performed. In SI 230, target text lines 1231 are rendered into the target line polygons to generate the target image.

[0231] In a second implementation, if the shape of the source text block is curved and the length difference(s) corresponding to the one or multiple source text lines is equal to or smallerthan a first threshold, the generating the target image includes: dewarping a text polygon to a text rectangle, where the text polygon corresponds to the source text block; generating one or multiple target line rectangles inside the text rectangle; fitting the one or multiple target text lines into the one or multiple target line rectangles; re- warping the text rectangles with the one or multiple text lines to the text polygon.

[0232] As aforementioned, the layout analysis process is performed to detect text regions. The text polygon can be generated based on the layout analysis process and the text polygon outlines the boundaries of the source text block. The line rectangles generated in the text rectangle are same in their shapes. The number of the line rectangles can be determined based on methods such as binary search as aforementioned, which is associated with a height of the text polygon, a height of each target text line and a spacing between adjacent target text lines. In one embodiment, to remain similar display effect, a ratio of the height of each target text line and the spacing between adjacent target text lines can remain the same with a ratio of the height of each source text line and the spacing between adjacent source text lines. That is, the number of target line polygons is associated with the height of each source text line and the spacing between adjacent source text lines.

[0233] In this application, processes on the text polygon (such as dewarping the text polygon into the text rectangle, generate line rectangles, fit the target text lines inside the line rectangles and re- warping the text rectangle into the text polygon to generate the target image) can be referred to as a curved rendering process.

[0234] FIG. 13 is a schematic diagram illustrating generating text polygon with the target text lines in the curved rendering process. After the dewarping, fitting and re-warping process, the target text lines are rendered into the text polygon.

[0235] Similar to the first implementation, a text removal process and a translation process can be performed before generating the target image.

[0236] FIG. 14 is a schematic diagram showing generating the target image using curved text rendering process. As shown in FIG 14, the source text block, the prepared image and the target text lines are used in the curved rendering process.

[0237] In S 1410, the text polygon is dewarped to a text rectangle and line rectangles are generated inside the text rectangles. In SI 420, target text lines are fitted into the line rectangles.In SI 430, the text rectangle with target text lines are rendered(re- warped) back to the text polygon.

[0238] FIG. 15 illustrates the process of generating the target image according to an embodiment of this application.

[0239] In S 1510, choose rendering process based on the source text block.

[0240] As aforementioned, if the shape of the source text block is straight and / or the length difference(s) corresponding to the one or multiple source text lines is bigger than a first threshold, the general rendering process is performed; otherwise, the curved rendering process is performed.

[0241] To perform the curved rendering, the prepared image (source image with removal of source text lines) and the target text lines translated from the source text lines are used in the curved rendering process 1530 or the general rendering process 1520 to generate the target image.

[0242] It is to be noted that in previous figures, the text lines are horizontal linage, left, right, top, bottom has been aforementioned. In detail, the left and right are parallel to the text line direction, the top and bottom are perpendicular to the text line direction.

[0243] In cases where the text lines are vertical linage, left and right are also along the text line direction, the top and the bottom are also perpendicular to the text line direction. Left boundary line, right boundary line, top boundary line and bottom boundary line, top line and bottom line are corresponding to the directions, details can be deduced from the aforementioned description and will not be repeated for brevity.

[0244] In summary, the image translation method provided in this application includes the following points.

[0245] 1. Intelligent text flow

[0246] Polygon-based text fitting: Instead of using simple rectangular bounding boxes, the system generates new polygons that fit the boundary area of the source text. This allows for more precise control over text placement and sizing, especially for non-rectangular text areas, maintaining its position and relationship to other image elements.

[0247] Rotation and direction preservation: The algorithm calculates and applies rotation angles to each line of text, ensuring that the text direction and orientation match the original,even in complex layouts.

[0248] 2. Geometric text transformation techniques

[0249] Irregular-shaped text handling: For non-linear text (e.g., text following a curve), the system dewarps the text into a rectangular image, processes it, and then re-warps it back to the original curved shape. This preserves the original text shape while accommodating the translated content.

[0250] 3. Preservation of text alignment and positioning relative to other image elements

[0251] Original layout analysis: The system utilizes OCR and text clustering results to identify the source text blocks and the geometry of the source text lines.

[0252] Bounding line generation: The system creates left, right, top, and bottom boundary lines for each text block, allowing for more complex text shapes than simple rectangles.

[0253] Boundary intersection checks: The system checks if new text polygons intersect with the boundary lines of the original text area, ensuring that the translated text stays within the intended region and doesn't overflow into other parts of the image.

[0254] The present disclosure offers an enhanced image translation method that preserves geometric layouts after translation. Key improvements include:

[0255] Limitation of the translated text by the original area of the text block.

[0256] Adaptive direction of text lines preserving the original text flow.

[0257] Irregular-shaped text rendering.

[0258] Improved visual fidelity of translated images compared to original versions.

[0259] Preservation of design intent and aesthetic qualities in translated content.

[0260] Enhanced user experience by maintaining familiar layouts across languages.

[0261] Reduced risk of misinterpretation due to layout changes.

[0262] It is to be noted that although from FIG. 3 to FIG. 15, the target text lines are introduced as translated from the source text lines to a different language. In some embodiments, the target text lines can be processed differently from the source text lines. For example, the target text lines can be an explanation, summary or extraction based on the source text lines.

[0263] Also, when the source image is a frame from a video, by processing multiple frames from the video, video translation, text replacement on video or real-time editing on video can be performed.

[0264] In some embodiments, the method provided can be used in text rendering on images or video using placeholders with required geometric properties. For example, the source image with straight source text lines needs to be rendered into a image with curved text lines.

[0265] In some other embodiments, the method provided in this application can be used on different surfaces, such as text rendering in fields of 3D modeling, augmented reality (AR) or virtual reality (VR).

[0266] The methods according to embodiments of this application are described above in detail with reference to FIGS. 3-15. The apparatuses provided in embodiments of this application are described below in detail with reference to FIGS. 16-17. The description of apparatus embodiments corresponds to the description of the method embodiments. Therefore, for content that is not described in detail, refer to the foregoing method embodiments. For brevity, details are not described herein again.

[0267] FIG. 16 is a schematic diagram of an electronic device according to an embodiment of this application.

[0268] As shown in FIG. 16, the electronic device includes: a layout analysis module, a text removal module, a translation module and geometric layout preservation module.

[0269] The layout analysis module can perform the layout analysis process as aforementioned. The input of the layout analysis module is the source image. Based on the layout analysis module, regions of the source text such as the source text block, the source line polygons can be determined. Also, the results of the layout analysis module are used by the text removal module, the translation module and the geometric layout preservation module.

[0270] The text removal module can perform the text removal process as aforementioned. By using the results of the layout analysis module, the source image is processed into the prepared image with the removal of the source text.

[0271] The translation module can perform the translation process as aforementioned. The source text lines are translated into target text lines.

[0272] The geometric layout analysis module can choose the rendering method and perform the general rendering process or the curved rendering process as aforementioned. Based on the source text block, the source line polygons, the prepared image and the target text lines, the target image is generated. FIG. 17 is a schematic bock diagram of an electronic device accordingto an embodiment of this application.

[0273] As shown in FIG. 17, the electronic device 1700 may include a receiver 1701, a processor 1702, a memory 1703, and a transmitter 1704. The memory 1703 may be configured to store code, instructions, and the like executed by the processor 1702.

[0274] It should be understood that the processor 1702 may be an integrated circuit chip and has a signal processing capability. In an implementation process, steps of the foregoing method embodiments may be completed by using a hardware integrated logic circuit in the processor, or by using instructions in a form of software. The processor may be a general purpose processor, a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), a system on chip (SoC) or another programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The processor may implement or perform the methods, the steps, and the logical block diagrams that are disclosed in the embodiments of the present application. The general purpose processor may be a microprocessor, or the processor may be any conventional processor or the like. The steps of the methods disclosed with reference to the embodiments of the present application may be directly performed and completed by the processor, or may be performed and completed by using a combination of hardware in the processor and a software module. The software module may be located in a mature storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, or a register. The storage medium is located in the memory, and the processor reads information in the memory and completes the steps of the foregoing methods performed by the electronic device in combination with hardware in the processor.

[0275] It may be understood that the memory 1703 in the embodiments of the present application may be a volatile memory or a nonvolatile memory, or may include both a volatile memory and a nonvolatile memory. The nonvolatile memory may be a read-only memory (Read-Only Memory, ROM), a programmable read-only memory (Programmable ROM, PROM), an erasable programmable read-only memory (Erasable PROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), or a flash memory. The volatile memory may be a random access memory (Random Access Memory, RAM) and is used as an external cache. By way of example rather than limitation, many formsof RAMs may be used, and are, for example, a static random access memory (Static RAM, SRAM), a dynamic random access memory (Dynamic RAM, DRAM), a synchronous dynamic random access memory (Synchronous DRAM, SDRAM), a double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM), an enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), a synchronous link dynamic random access memory (Synchronous link DRAM, SLDRAM), and a direct rambus random access memory (Direct Rambus RAM, DR RAM).

[0276] It should be noted that the memory in the electronic device and the methods described in this specification includes but is not limited to these memories and a memory of any other appropriate type.

[0277] The present application provides a computer readable storage medium including instructions. When the instructions run on an electronic device, the electronic device is enabled to perform the aforementioned method.

[0278] The present application provides a chip system. The chip system includes a memory and a processor, and the memory is configured to store a computer program, and the processor is configured to invoke the computer program from the memory and run the computer program, so that an electronic device on which the chip system is disposed performs the aforementioned method.

[0279] The present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device is enabled to perform the aforementioned method.

[0280] In the embodiments of the present application, "at least one" means one or more, and "a plurality of means two or more. The term "and / or" describes an association relationship between associated objects and represents that three relationships may exist. For example, A and / or B may represent the following three cases: only A exists, both A and B exist, and only B exists, where A and B may be singular or plural. The character " / " generally indicates an "or" relationship between the associated objects. "At least one of the following" and a similar expression thereof refer to any combination of these items, including any combination of one item or a plurality of items. For example, at least one of a, b, and c may indicate: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c may be singular or plural.

[0281] A person of ordinary skill in the art may be aware that, in combination with the examples described in the embodiments disclosed in this specification, units and algorithm steps can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed by hardware or software depends on particular applications and design constraints of the technical solutions. A person skilled in the art may use different methods to implement the described functions for each particular application, but it should not be considered that the implementation goes beyond the scope of this application.

[0282] It may be clearly understood by a person skilled in the art that, for the purpose of convenient and brief description, for a detailed working process of the foregoing system, apparatus, and unit, refer to a corresponding process in the foregoing method embodiment. Details are not described herein again.

[0283] In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus, and method may be implemented in other manners. For example, the described apparatus embodiment is merely an example. For example, the unit / module division is merely logical function division and may be other division in actual implementation. For example, a plurality of units or components may be combined or integrated into another system, or some features may be ignored or not performed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections may be implemented through some interfaces. The indirect couplings or communication connections between the apparatuses or units may be implemented in electronic, mechanical, or other forms.

[0284] The units described as separate parts may be or may not be physically separate, and parts displayed as units may be or may not be physical units, may be located in one position, or may be distributed on a plurality of network units. Some or all of the units may be selected based on actual requirements to achieve the objectives of the solutions of the embodiments.

[0285] In addition, functional units in the embodiments of this application may be integrated into one processing unit, or each of the units may exist alone physically, or two or more units are integrated into one unit.

[0286] When the functions are implemented in a form of a software functional unit and sold or used as an independent product, the functions may be stored in a computer readable storagemedium. Based on such an understanding, the technical solutions in this application essentially, or the part contributing to the prior art, or some of the technical solutions may be implemented in a form of a software product. The computer software product is stored in a storage medium, and includes several instructions for instructing a computer device (which may be a personal computer, a server, a network device, or the like) to perform all or some of the steps of the methods described in the embodiments of this application. The foregoing storage medium includes: any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc.

[0287] The foregoing descriptions are merely specific implementations of this application, but are not intended to limit the protection scope of this application. Any variation or replacement readily figured out by a person skilled in the art within the technical scope disclosed in this application shall fall within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

CLAIMSWhat is claimed is:

1. An image translation method, comprising: obtaining a source image, wherein the source image comprises a source text block with one or multiple source text lines; generating a target image based on geometric properties of the source text block, wherein the target image comprises one or multiple target text lines translated from the one or multiple source text lines and the geometric properties of the source text block comprises: a shape of the source text block and length difference(s) corresponding to the one or multiple source text lines.

2. The image translation method according to claim 1, wherein if the shape of the source text block is straight and / or the length difference(s) corresponding to the one or multiple source text lines is bigger than a first threshold, the generating a target image comprises: generating one or multiple target line polygons based on one or multiple source line polygons, wherein the one or multiple source line polygons correspond to the one or multiple source text lines and the one or multiple target line polygons correspond to the one or multiple target text lines; generating the target image based on the one or multiple target line polygons.

3. The image translation method according to claim 2, wherein different target line polygons correspond to different rotation angles.

4. The image translation method according to claim 3, wherein a rotation angle corresponds to a target line polygon is associated with a position of the target line polygon, an angle between the top boundary line and the bottom boundary line of the source text block and the number of the target line polygons.

5. The image translation method according to claim 3 or 4, wherein if the shape of the source text block is curved, the generating the target image based on the one or multiple target line polygons comprises: dewarping the one or multiple target line polygons to one or multiple target line rectangles; fitting the one or multiple target text lines to the one or multiple target line rectangles;re-warping the one or multiple target line rectangles with the one or more target text lines to the one or multiple target line polygons.

6. The image translation method according to any one of claims 2 to 5, wherein an offset corresponding to a target line polygon relative to the top target line polygon is associated with a position of the target line polygon, a height of each target text line and a spacing between adjacent target text lines.

7. The image translation method according to claim 6, wherein the height of each target text line and the spacing between adjacent target text lines are associated with a height of each source text line and a spacing between adjacent source text lines.

8. The image translation method according to any one of claims 2 to 7, wherein the one or multiple target line polygons are generated based on a left boundary line and a right boundary line corresponding to the source text block, wherein the left boundary line and the right boundary line are generated based on the one or multiple source line polygons.

9. The image translation method according to claim 1, wherein if the shape of the source text block is curved and the length difference(s) corresponding to the one or multiple source text lines is equal to or smaller than a first threshold, the generating the target image comprises: dewarping a text polygon to a text rectangle, wherein the text polygon corresponds to the source text block; generating one or multiple target line rectangles inside the text rectangle; fitting the one or multiple target text lines into the one or multiple target line rectangles; re-warping the text rectangle with the one or multiple text lines to the text polygon.

10. The image translation method according to claim 9, wherein the number of the one or multiple target line rectangles is associated with a height of each source text line and a spacing between adjacent source text lines.

11. An image translation apparatus, comprising: an obtaining unit, configured to obtain a source image, wherein the source image comprises a source text block with one or multiple source text lines; a processing unit, configured to generate a target image based on geometric properties of the source text block, wherein the target image comprises one or multiple target text lines translated from the one or multiple source text lines and the geometric properties of the sourcetext block comprises: a shape of the source text block and length difference(s) corresponding to the one or multiple source text lines.

12. The image translation apparatus according to claim 11, wherein if the shape of the source text block is straight and / or the length difference(s) corresponding to the one or multiple source text lines is bigger than a first threshold, the processing unit is configured to: generate one or multiple target line polygons based on one or multiple source line polygons, wherein the one or multiple source line polygons correspond to the one or multiple source text lines and the one or multiple target line polygons correspond to the one or multiple target text lines; generate the target image based on the one or multiple target line polygons.

13. The image translation apparatus according to claim 12, wherein different target line polygons correspond to different rotation angles.

14. The image translation apparatus according to claim 13, wherein a rotation angle corresponds to a target line polygon is associated with a position of the target line polygon, an angle between the top boundary line and the bottom boundary line of the source text block and the number of the target line polygons.

15. The image translation apparatus according to claim 13 or 14, wherein if the shape of the source text block is curved, the processing unit is configured to: dewarp the one or multiple target line polygons to one or multiple target line rectangles; fit the one or multiple target text lines to the one or multiple target line rectangles; re-warp the one or multiple target line rectangles with the one or more target text lines to the one or multiple target line polygons.

16. The image translation apparatus according to any one of claims 12 to 15, wherein an offset corresponding to a target line polygon relative to the top target line polygon is associated with a position of the target line polygon, a height of each target text line and a spacing between adjacent target text lines.

17. The image translation apparatus according to claim 6, wherein the height of each target text line and the spacing between adjacent target text lines are associated with a height of each source text line and a spacing between adjacent source text lines.

18. The image translation apparatus according to any one of claims 12 to 17, wherein theone or multiple target line polygons are generated based on a left boundary line and a right boundary line corresponding to the source text block, wherein the left boundary line and the right boundary line are generated based on the one or multiple source line polygons.

19. The image translation apparatus according to claim 11, wherein if the shape of the source text block is curved and the length difference(s) corresponding to the one or multiple source text lines is equal to or smaller than a first threshold, the processing unit is configured to: dewarp a text polygon to a text rectangle, wherein the text polygon corresponds to the source text block; generate one or multiple target line rectangles inside the text rectangle; fit the one or multiple target text lines into the one or multiple target line rectangles; re-warp the text rectangle with the one or multiple text lines to the text polygon.

20. The image translation apparatus according to claim 19, wherein the number of the one or multiple target line rectangles is associated with a height of each source text line and a spacing between adjacent source text lines.

21. An electronic device, comprising a layout analysis module, configured to analyze a layout of a source image, wherein the layout of the source image comprises a layout of a source text block and one or multiple source text lines in the source image; a text removal module, configured to remove the one or multiple source text lines for a prepared image; a translation module, configured to translate the one or multiple source text lines to one or multiple target text lines a geometric layout preservation module, configured to generate the target image based on the prepared image, the one or multiple target text lines and the layout of the source text bock and the one or multiple source text lines.

22. A computer-readable storage medium, wherein the computer-readable storage medium stores instructions, and when the instructions run on a device, the device is enabled to perform the method according to any one of claims 1 to 10.

23. A computer program product, wherein when the computer program product runs on adevice, the device is enabled to perform the method according to any one of claims 1 to 10.

24. A chip system, comprising a memory and a processor, wherein the memory is configured to store a computer program, and the processor is configured to invoke the computer program from the memory and run the computer program, so that a device on which the chip system is disposed performs the method according to any one of claims 1 to 10.

Images