Adaptive layout method and device based on comic translation

Abstract

The application provides a self-adaptive layout method and device based on cartoon translation, which comprises the following steps: S1, obtaining the top-left vertex coordinate of a translation layout area, and taking the coordinate as the starting position of translation drawing; S2, pre-drawing the translation words in the translation word list one by one based on the starting position and a first drawing font size; S3, if the height of the drawn translation is greater than the height value of the translation layout area, reducing the first drawing font size by one to obtain a second drawing font size; S4, pre-drawing the translation words in the translation word list again based on the second drawing font size and the translation layout area; and S5, repeating S3-S4 until the height of the drawn translation is less than or equal to the height value of the translation layout area. The application realizes the process unification of cartoon translation, standardizes the layout standard, reduces the influence of subjective factors in the translation process, reduces the manual participation, and saves manpower.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, in particular to a self-adaptive layout method and device based on comic translation, a computer device and a storage medium. BACKGROUND

[0002] People who have flipped through comics will not be unfamiliar with the term "Hanization Group". For a long time, the work of comic translation has been following a process with a long cycle and a large amount of repetitive manual labor.

[0003] The tools used by the comic translation group are mostly Photoshop (PS). As a professional photo editing software, PS has a high learning cost and does not fully meet the needs of comic translation. The comic translation group needs to manually extract and translate the text information in the picture, erase the original text in the picture, and then embed the translated text one by one, adjust the font and layout to be suitable for the original picture, and then export it. With the continuous development of the domestic comic market, the field of comic translation has gradually developed related tools to assist manual translation, which mainly involves original text detection, text quick erasing, translation of different languages, multiple font style selection, and other aspects to help manual translation reduce the complex work of operating PS step by step.

[0004] Although the existing translation tools have reduced some manual work, they still require manual correction of the results of the translation tools. Some of these are deterministic work, such as incomplete original text erasure and inaccurate machine translation, which require consistent judgment standards from different designers. The other part is complex and uncertain work, such as the determination of the size of the translated text and the layout of the translated text, which involves some subjective factors in the judgment of designers. Therefore, the existing comic translation tools have the following problems in layout:

[0005] (1) Complex word segmentation adjustment: punctuation marks in the translated text need to be manually adjusted when they appear at the beginning of a line.

[0006] (2) Rough determination of layout area: adding a text editing area directly to fill in the translated text in the original text area, or simply dividing the grid according to the number of translated words; manual adjustment of the text area is required, and the grid division method has a large impact on the layout of words with large length spans such as English.

[0007] (3) Non-adaptive font size: fixed font size or maximum font size determined by grid division in the translated text area, which requires constant manual adjustment to adapt to the page.

[0008] (4) Translation area expansion relies on manual subjective judgment: adding a text editing area and then manually stretching it to fit, or directly using the original text area for layout.

[0009] Therefore, there is an urgent need for a self-adaptive layout method and device based on comic translation to solve the above technical problems. SUMMARY

[0010] Embodiments of the present application provide a self-adaptive layout method and device based on comic translation to solve the problems of rough layout area determination, complicated word segmentation adjustment, inappropriate font size, and subjective judgment of translation area expansion in existing comic translation tools.

[0011] To achieve the above object, in a first aspect, the embodiments of the present application provide a self-adaptive layout method based on comic translation, comprising:

[0012] S1, after determining the translation layout area, obtaining the top-left vertex coordinate of the translation layout area as the starting position of translation drawing;

[0013] S2, based on the starting position and a first drawing font size, pre-drawing the translation words in the translation word segmentation list one by one, the first drawing font size being the maximum font size corresponding to the translation layout area;

[0014] S3, if the height of the drawn translation is greater than the height value of the translation layout area, reducing the first drawing font size by one to obtain a second drawing font size;

[0015] S4, based on the second drawing font size and the translation layout area, re-drawing the translation words in the translation word segmentation list;

[0016] S5, repeating S3-S4 until the height of the drawn translation is less than or equal to the height value of the translation layout area, and taking the drawn translation as the target layout result.

[0017] Optionally, in a possible implementation manner of the first aspect, the pre-drawing the translation words in the translation word segmentation list one by one based on the starting position and the first drawing font size comprises:

[0018] judging whether the width value of the current drawn translation is greater than the maximum width value of the layout area;

[0019] if yes, adding the vertical coordinate of the starting position to the single-line drawing height value to obtain a first starting position; and drawing the current translation word based on the first starting position and the first drawing font size;

[0020] if no, adding the horizontal coordinate of the starting position to the width value of the current drawn translation without the current translation word to obtain a second starting position; and drawing the current translation word based on the second starting position and the first drawing font size.

[0021] Optionally, in a possible implementation manner of the first aspect, the step S2 further includes:

[0022] segmenting the translation;

[0023] judging whether each segmented translation is a punctuation mark or not;

[0024] if the current segmented translation is a punctuation mark, merging the punctuation mark and the previous segmented translation into one group to obtain a segmented translation list.

[0025] Optionally, in a possible implementation manner of the first aspect, the step S1 further includes:

[0026] pre-setting a plurality of sets of expansion step values in different directions, the expansion step values being used for gradually expanding the translation area in horizontal and / or vertical directions;

[0027] expanding the initial translation area based on the area gradual expansion method and each set of expansion step values to obtain a candidate expansion area, and calculating an area of the candidate expansion area;

[0028] selecting an expansion step value with the largest expansion area, and taking the candidate expansion area corresponding to the expansion step value as a translation layout area.

[0029] Optionally, in a possible implementation manner of the first aspect, the step of expanding the initial translation area based on the area gradual expansion method and each set of expansion step values to obtain a candidate expansion area, and calculating an area of the candidate expansion area includes:

[0030] obtaining a rectangular coordinate of the initial translation area, the rectangular coordinate being a coordinate of a top-left corner, a top-right corner, a bottom-left corner and a bottom-right corner of the initial translation area, and calculating a pixel mean value of the initial translation area according to the rectangular coordinate;

[0031] selecting an arbitrary side boundary of the initial translation area, the side boundary including a left side boundary, an upper side boundary, a right side boundary and a lower side boundary;

[0032] expanding the side boundary to a corresponding side direction to obtain a current expansion area based on an expansion step value, and calculating a pixel mean value of the current expansion area according to a rectangular coordinate of the current expansion area;

[0033] determining a side boundary coordinate value of the candidate expansion area according to the pixel mean values of the current expansion area and the initial translation area, the side boundary coordinate value including an upper side boundary longitudinal coordinate, a lower side boundary longitudinal coordinate, a left side boundary horizontal coordinate and a right side boundary horizontal coordinate.

[0034] In a second aspect of the embodiment of the present application, a self-adaptive layout device based on comic translation is provided, which includes:

[0035] An initial position obtaining module is configured to obtain a top-left vertex coordinate of the translation layout area as an initial position for drawing the translation after determining the translation layout area;

[0036] A first pre-drawing module is configured to pre-draw the translation words in the translation word list one by one based on the initial position and a first drawing font size, the first drawing font size being a maximum font size corresponding to the translation layout area;

[0037] A translation height judging module is configured to decrease the first drawing font size by one to obtain a second drawing font size if the height of the drawn translation is greater than the height value of the translation layout area;

[0038] A second pre-drawing module is configured to pre-draw the translation words in the translation word list again based on the second drawing font size and the translation layout area;

[0039] A target layout result generating module is configured to repeatedly execute the translation height judging module and the second pre-drawing module until the height of the drawn translation is less than or equal to the height value of the translation layout area, and to take the drawn translation as the target layout result.

[0040] Optionally, in a possible implementation of the second aspect, the first pre-drawing module comprises:

[0041] judging whether the width value of the currently drawn translation is greater than the maximum width value of the layout area;

[0042] If yes, adding the vertical coordinate of the initial position to the single-line drawing height value to obtain a first initial position, and drawing the current translation word based on the first initial position and the first drawing font size;

[0043] If no, adding the horizontal coordinate of the initial position to the width value of the currently drawn translation without the current translation word to obtain a second initial position, and drawing the current translation word based on the second initial position and the first drawing font size.

[0044] Optionally, in a possible implementation of the second aspect, the first pre-drawing module further comprises:

[0045] performing word segmentation on the translation;

[0046] judging whether each translation word in the segmented translation is a punctuation mark;

[0047] If the current translation word is a punctuation mark, merging the punctuation mark and the previous translation word into one group to obtain the translation word list.

[0048] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program capable of running on the processor, and the processor implements the steps of each method embodiment of the present application when executing the computer program.

[0049] In a fourth aspect, the present application provides a readable storage medium, wherein the readable storage medium stores a computer program, and the computer program is used to implement the steps of the method of the first aspect and various possible designs of the first aspect when executed by a processor.

[0050] The present application provides a self-adaptive layout method and device based on comic translation, a computer device and a storage medium. In S1, after determining a layout area of a translation, a top-left vertex coordinate of the layout area of the translation is obtained, and the coordinate is taken as a starting position of translation drawing. In S2, the translation words in a segmentation list of segmented translation are pre-drawn one by one based on the starting position and a first drawing font size. The first drawing font size is a maximum font size corresponding to the layout area of the translation. In S3, if a height of the drawn translation is greater than a height value of the layout area of the translation, the first drawing font size is reduced by one to obtain a second drawing font size. In S4, the translation words in the segmentation list of the translation are pre-drawn again based on the second drawing font size and the layout area of the translation. In S5, S3-S4 are repeatedly executed until the height of the drawn translation is less than or equal to the height value of the layout area of the translation, and the drawn translation is taken as a target layout result. The present application solves three processes that need strong human participation in the comic translation process, i.e., determination of a best translation area, determination of a best font size and unification of layout logic, realizes unification of the comic translation process, standardizes the layout standard, reduces the influence of subjective factors in the translation process, reduces human participation and saves manpower. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 An effect diagram of self-adaptive layout;

[0052] Figure 2 A flowchart of a first embodiment of a self-adaptive layout method based on comic translation;

[0053] Figure 3 An effect diagram of font size self-adaptive adjustment;

[0054] Figure 4 A diagram of multi-scale expansion step area expansion;

[0055] Figure 5 A diagram of a candidate expansion area;

[0056] Figure 6 A structure diagram of a first embodiment of a self-adaptive layout device based on comic translation;

[0057] Figure 7 Schematic diagram of a machine typesetting framework. DETAILED DESCRIPTION

[0058] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work belong to the protection scope of the present application.

[0059] The terms "first", "second", "third", "fourth" and the like in the description and claims of the present application and above drawings (if any) are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in other than the order illustrated or described herein.

[0060] It should be understood that, in various embodiments of the present application, the magnitude of the serial number of each process does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0061] It should be understood that, in the present application, "comprising" and "having" and any variants thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units does not necessarily limit to the clearly listed steps or units, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0062] It should be understood that, in the present application, "a plurality of" means two or more. "And / or" is only a description of the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents that the associated objects before and after are in an "or" relationship. "Including A, B and C", "including A, B, C" means that A, B and C are all included, "including A, B or C" means that one of A, B and C is included, and "including A, B and / or C" means that any one or any two or three of A, B and C is included.

[0063] It should be understood that, in the present application, "B corresponding to A", "B corresponding to A", "A corresponding to B" or "B corresponding to A" means that B is associated with A, and B can be determined according to A. Determining B according to A does not mean that B is determined only according to A, but B can also be determined according to A and / or other information. The matching of A and B is that the similarity of A and B is greater than or equal to a preset threshold.

[0064] Depending on the context, "if" as used herein can be interpreted as "when" or "when" or "in response to determining" or "in response to detecting".

[0065] The technical solutions of the present application will be described in detail below with specific examples. The following specific examples can be combined with each other, and the same or similar concepts or processes may not be described in some embodiments.

[0066] In view of the problem that manual participation is required for continuous correction and layout specification in the layout process of a comic tool, and the subjectivity of layout specification is strong, the present application aims to design a general and standard comic text machine layout framework, standardize the layout specification, reduce the influence of subjective factors in the translation process, reduce manual participation, and greatly save manpower while outputting logically unified layout comics.

[0067] The technical solutions of the present application focus on realizing adaptive layout of the translation, as shown in Figure 1 The left side is the layout of the initial text area in the bubble box, wherein the solid line is the initial coordinate of the text area, and the dashed line is the maximum coordinate of the desired text area; the right side is the adaptive layout effect of the expanded translation text area. That is, it mainly involves two steps: 1, expanding the translation text area to the size corresponding to the bubble box; 2, drawing the translation text to the translation text area after expanding the translation text area to the corresponding size, and adaptively adjusting the translation text to maximize the translation text size while avoiding overflow of the translation text area.

[0068] Since the present application mainly focuses on the adaptive layout of the translation, the elimination of the original text and the acquisition of the initial translation text area are ignored, and it is assumed that the part of data has been obtained; the implementation of machine drawing text uses open source drawing toolkit Pillow and the like.

[0069] Embodiment 1:

[0070] The present application provides a kind of based on the adaptive layout method of comic translation, as shown in Figure 2 The flow chart thereof, it includes:

[0071] Step S1, after determining the translation layout area, the left top vertex coordinate of the translation layout area is obtained, and the coordinate is used as the starting position of the translation drawing.

[0072] In this step, after obtaining the optimal translation layout area by expanding according to the initial translation area coordinates, the translation needs to be drawn into this translation layout area. Therefore, the upper left vertex coordinates (X1, X2) of this translation layout area will be used as the starting point for drawing the translation, and then the translation will be drawn.

[0073] Step S2: Based on the starting position and the first drawing font size, pre-draw each translation word in the translated word list after word segmentation one by one. The first drawing font size is the largest font size corresponding to the translation layout area.

[0074] In step S2,

[0075] Before pre-drawing each translation word one by one, it is necessary to segment the obtained translation first. For example: the original translation is "What is he singing?!", after word segmentation: ['他', '唱', '什', '么', '呢', '?', '!']; then judge whether each translation word in the translated text after word segmentation is a punctuation mark. If the translation word is a punctuation mark, then combine the punctuation mark and the previous translation word into a group to optimize and obtain the translated word list. Taking the above translated text after word segmentation as an example, the optimized translated word list obtained is: ['他', '唱', '什', '么', '呢?!']. The above word segmentation method is not restricted by any language during the word segmentation process, and appropriate methods can be adopted according to different language grammars. For example, Chinese uses character-by-character word segmentation, English uses space character word segmentation, Japanese uses katakana word segmentation, etc. Taking the above Chinese as an example, the first line is the original translation, the second line is the translated word list after word segmentation, and the third line is the translated word list after optimizing the combination of punctuation marks. The optimization result combines (歌,) and (呢!?) as a whole, ensuring that there will be no punctuation marks at the beginning of a line during typesetting, without manual correction.

[0076] The process of pre-drawing the translation words in the translation word list one by one, one translation word at a time, and each time drawing needs to be performed according to the starting coordinate position. The drawing implementation includes the following two kinds: 1. When drawing the first translation word, the top-left vertex coordinate (draw_x=x1, draw_y=y1) of the translation layout area is taken as the starting position to start drawing; when drawing the third translation word, the starting position is the horizontal coordinate of the top-left vertex coordinate plus the width value of the current drawing translation (draw_x=x1+2t_w, draw_y=y1), here the number of characters of the translation word segmentation drawn each time is normalized, that is, the number of characters of the translation word segmentation drawn each time is the same, t_w represents the width of a translation word segmentation. When drawing the nth translation word, the starting position of the drawing is also updated to (draw_x=x1+(n-1)*t_w, draw_y=y1). Here n should be less than or equal to the width value of the layout area. 2. In actual application, because the number of characters of each translation word segmentation is different, the horizontal coordinate of the starting position of each drawing is represented by draw_x=w_draw+t_w, where draw_x represents the total width of the translation line when drawing to the current translation word segmentation, t_w represents the width of the current translation word segmentation, and w_draw represents the total width of the current line cumulative translation drawing. For example: when drawing the first translation word, the top-left vertex coordinate (draw_x=x1, draw_y=y1) of the translation layout area is still taken as the starting position to start drawing; when drawing the third translation word segmentation, the starting position is (draw_x=w_draw+t_w, draw_y=y1), t_w is the width of the third translation word segmentation, and w_draw is the total width of the first two translation word segmentations.

[0077] According to the drawing method, the process of drawing needs to judge whether the width of the current drawing text is greater than the maximum width of the layout area (w). If yes, the vertical coordinate of the starting position is added to the single-line drawing height value to obtain the first starting position. The current text word is drawn based on the first starting position and the first drawing font size. If no, the horizontal coordinate of the starting position is added to the width of the current drawing text without the current text word to obtain the second starting position. The current text word is drawn based on the second starting position and the first drawing font size. For example, 1. When the 11th text word is drawn, the text width is 11 unit lengths, and the width of the layout area is 10 unit lengths. 11>10, so the 11th text word needs to be drawn to the left end of the next line. Therefore, the horizontal coordinate of the drawing starting position is x1, and the vertical coordinate is y1+single-line drawing height value h_draw, that is, “if yes, the vertical coordinate (y1) of the starting position is added to the single-line drawing height value (h_draw) to obtain the first starting position. The current text word is drawn based on the first starting position and the first drawing font size”. 2. When the 9th text word is drawn, the text width is 9 unit lengths, and the width of the layout area is 10 unit lengths. 9<10, so the 9th text word is continued to be drawn in the line. Therefore, the vertical coordinate (y1) of the drawing starting position of the 9th text word is not changed, and the horizontal coordinate is adjusted to x1+(8)*t_w, that is, “if no, the horizontal coordinate of the starting position is added to the width of the current drawing text without the current text word to obtain the second starting position. The current text word is drawn based on the second starting position and the first drawing font size”. In addition, the value of the single-line drawing height value (h_draw) is h_draw=max(h_draw, t_h), t_h represents the maximum height of the text word in the text that has been drawn in the line, which is used to determine the vertical coordinate y of the drawing starting position when the line is changed. The purpose is to avoid the problem of too tight lines, that is, the overlap of the text in the previous line and the text in the next line, and the unattractive layout.

[0078] Step S3, if the height of the drawn text is greater than the height value of the text layout area, the first drawing font size is reduced by one to obtain a second drawing font size.

[0079] Step S4, the text words in the text word list are pre-drawn again based on the second drawing font size and the text layout area.

[0080] Step S5, steps S3-S4 are repeatedly executed until the height of the drawn text is less than or equal to the height value of the text layout area, and the drawn text is taken as the target layout result.

[0081] In the steps S3-S5, after all the translated words in the translated word list are drawn in the layout area according to the drawing method in step S2, it is determined whether the height of the drawn translated words is greater than the height value of the translated layout area. If yes, it means that the current font size is too large and does not adapt to the translated layout area, and the font size needs to be adjusted to be smaller, for example, No. 1, small No. 1, No. 2, small No. 2, No. 3, small No. 3, and so on. After the initial No. 1 font size is selected, it is found that the height of the drawn translated words is greater than the height value of the translated layout area, and the font size is adjusted to be small No. 2. Then, the height of the drawn translated words is compared with the height value of the translated layout area. If yes, the font size is continuously adjusted until the height of the drawn translated words is less than or equal to the height value of the translated layout area. Specifically, as shown in FIG. 6, Figure 3 The dashed rectangle in the figure is the translated layout area. The left side is the result of self-adaptive layout using the initial maximum font size, and the right side is the layout result obtained after using the font size self-adaptive adjustment method.

[0082] In an embodiment, before step S1, the implementation of the translated layout area is also determined, as follows:

[0083] In step S11, a plurality of sets of expansion step values in different directions are preset, which are used for progressive expansion of the translated area in the horizontal and / or vertical directions.

[0084] In step S11, a plurality of sets of expansion step values in different directions are preset, which are used for progressive expansion of the translated area in the horizontal and / or vertical directions. Figure 4 As shown in FIG. 5, the solid rectangle area in the first bubble chart is the initial translated drawing area, and the subsequent three bubbles represent the final expansion areas (dashed rectangle) obtained by using different scale expansion step values. Si represents the area of the corresponding area.

[0085] In step S12, based on the area progressive expansion method and each set of expansion step values, the initial translated area is expanded to obtain a candidate expansion area, and the area of the candidate expansion area is calculated.

[0086] In step S12, after obtaining a plurality of sets of expansion step values (scale_anchors), the expansion step values (ai, bi) of each set are selected in sequence, and the initial translated area is expanded by using the progressive expansion method based on the mean value change of the area pixels to obtain the coordinates and area of the current expansion area based on the current step value.

[0087] Step S13, selecting the expansion step value with the largest area of the expansion region, and taking the candidate expansion region corresponding to the step value as the layout region of the translation.

[0088] In step S13, the expansion region coordinates and area based on each set of step values are obtained in sequence, and the set of step values with the largest area of the region is selected, and the step corresponding to the expansion region coordinates is the final layout region of the translation.

[0089] In one embodiment, the step S12 comprises:

[0090] Step S121, obtaining the rectangular coordinates of the initial translation region, which are the coordinates of the upper left and right four vertex positions of the initial translation region, i.e. box = [(x1, y1), (x2, y1), (x2, y2), (x1, y2)], and calculating the pixel mean crop_mean of the initial translation region according to the rectangular coordinates, the specific process being as follows: first, according to the rectangular coordinates of the initial translation region, using an existing computer image processing library, performing cutout processing on the original picture at the corresponding position; then, based on the pixel representation of the image after the cutout processing, using an existing computer data processing library, calculating the mean of all pixels in the image.

[0091] Step S122, selecting any one side boundary of the initial translation region, the side boundary including: left side boundary, upper side boundary, right side boundary, and lower side boundary; based on the expansion step value, expanding the side boundary to the corresponding side direction to obtain the current expansion region, and calculating the pixel mean (crop_mean_left / top / right / down) of the current expansion region according to the rectangular coordinates of the current expansion region, the specific process being as follows: first, according to the rectangular coordinates of the current expansion region, using an existing computer image processing library, performing cutout processing on the original picture at the corresponding position; then, based on the pixel representation of the image after the cutout processing, using an existing computer data processing library, calculating the mean of all pixels in the image.

[0092] In this step, the current expansion region is obtained by expanding the selected side boundary of the initial translation region to its corresponding side direction. The expansion to the corresponding side direction means expanding in the direction on the same side of the side boundary. For example, when the upper side boundary of the initial translation region rectangular region is selected, the upper side boundary is expanded upward. Similarly, the upper side boundary is upward, the lower side boundary is downward, the left side boundary is leftward, and the right side boundary is rightward. The expansion of the side boundary to the corresponding side direction based on the expansion step value to obtain the current expansion region means that when the upper side boundary of the initial translation region rectangular region and the expansion step value (a, b) are selected, the corresponding coordinate points of the boundary are [(x1, y1), (x2, y1)], the upper side is expanded to obtain new coordinates [(x1, y1-b), (x2, y1-b)], and then the new rectangular coordinates [(x1, y1-b), (x2, y1-b), (x2, y2), (x1, y2)] are obtained.

[0093] In step S123, the side boundary coordinate values of the candidate expansion region are determined based on the pixel mean values of the current expansion region and the initial translation region, and the side boundary coordinate values include the upper side boundary longitudinal coordinate, the lower side boundary longitudinal coordinate, the left side boundary horizontal coordinate, and the right side boundary horizontal coordinate.

[0094] In step S123, the region expansion stop threshold (expand_threshold >= 0) is set in advance. If the pixel mean value difference between the current expansion region and the initial translation region is greater than the region expansion stop threshold, the corresponding side boundary coordinate values of the initial translation region are taken as the side boundary coordinate values of the candidate expansion region. If the pixel mean value difference between the current expansion region and the initial translation region is less than or equal to the preset threshold, the corresponding side boundary coordinate values of the current expansion region are taken as the side boundary coordinate values of the candidate expansion region.

[0095] Taking step S122 above, where the upper boundary of the rectangular region is selected and the expansion step value (a, b) is used for expansion, as an example: the current expanded region is obtained by expanding the upper boundary of the initial translation region upwards, along with the pixel mean of the current expanded region, crop_mean_top; then, the difference between the pixel mean of the current expanded region and the pixel mean of the initial translation region is processed, i.e., (crop_mean_top - crop_mean); if the result of this difference processing is greater than the region expansion stopping threshold, i.e., abs(crop_mean_top - crop_mean) > expand_threshold, then the ordinate of the upper boundary is determined to be y1 and no further expansion is made, i.e., "the corresponding side boundary coordinate value of the initial translation region is used as the side boundary coordinate value of the candidate expanded region"; otherwise, the ordinate of the upper boundary of the region is updated to y1 = y1 - a, i.e., "the corresponding side boundary coordinate value of the current expanded region is used as the side boundary coordinate value of the candidate expanded region". If the region expansion is "upward expansion from the top," then the ordinate value of the top boundary of the candidate expansion region needs to be determined; similarly, if it's "downward expansion from the bottom," then the ordinate value of the bottom boundary of the candidate expansion region needs to be determined; if it's "leftward expansion from the left," then the abscissa value of the left boundary of the candidate expansion region needs to be determined; and if it's "rightward expansion from the right," then the abscissa value of the right boundary of the candidate expansion region needs to be determined. Here, "abs" is a computational term representing the absolute value.

[0096] Step S124: Based on the expansion step size value, the initial translation region is expanded in the four directions of up, down, left, and right to obtain the coordinate values ​​of the four side boundaries of the candidate expansion region; the area of ​​the candidate expansion region is calculated based on the coordinate values ​​of the four side boundaries.

[0097] In step S124, based on steps S122-S123 above, the initial translation region is expanded in four directions: up, down, left, and right, thereby determining four boundary coordinate values. These four boundary coordinate values ​​form a rectangular region (i.e., the candidate expansion region), and the area of ​​the rectangular region is calculated. Figure 5 As shown, when the four boundary coordinate values ​​are determined, a rectangular region will naturally be formed.

[0098] The application provides a self-adaptive layout method based on comic translation, which comprises the following steps: S1, obtaining the top-left vertex coordinate of a translation layout area as a starting position for drawing the translation after determining the translation layout area; S2, pre-drawing the translation words in a translation word segmentation list one by one based on the starting position and a first drawing font size, wherein the first drawing font size is the maximum font size corresponding to the translation layout area; S3, if the height of the drawn translation is greater than the height value of the translation layout area, then reducing the first drawing font size by one to obtain a second drawing font size; S4, re-pre-drawing the translation words in the translation word segmentation list based on the second drawing font size and the translation layout area; S5, repeating S3-S4 until the height of the drawn translation is less than or equal to the height value of the translation layout area, and taking the drawn translation as a target layout result.

[0099] Embodiment 2

[0100] The embodiment of the application further provides a self-adaptive layout device based on comic translation, which comprises the following steps: Figure 6 as shown in the figure, comprising:

[0101] A starting position acquisition module is configured to obtain the top-left vertex coordinate of a translation layout area as a starting position for drawing the translation after determining the translation layout area;

[0102] A first pre-drawing module is configured to pre-draw the translation words in a translation word segmentation list one by one based on the starting position and a first drawing font size, wherein the first drawing font size is the maximum font size corresponding to the translation layout area;

[0103] A translation height judgment module is configured to reduce the first drawing font size by one to obtain a second drawing font size if the height of the drawn translation is greater than the height value of the translation layout area;

[0104] A second pre-drawing module is configured to re-pre-draw the translation words in the translation word segmentation list based on the second drawing font size and the translation layout area;

[0105] A target layout result generation module is configured to repeat the translation height judgment module and the second pre-drawing module until the height of the drawn translation is less than or equal to the height value of the translation layout area, and take the drawn translation as a target layout result.

[0106] In one embodiment, the first pre-drawing module comprises:

[0107] determining whether the width value of the current drawing translation is greater than the maximum width value of the layout area;

[0108] If greater, adding the vertical coordinate of the starting position to the single-line drawing height value to obtain a first starting position; drawing the current translation word based on the first starting position and the first drawing font size;

[0109] If not greater, adding the horizontal coordinate of the starting position to the width value of the current drawing translation without the current translation word to obtain a second starting position; drawing the current translation word based on the second starting position and the first drawing font size.

[0110] In one embodiment, the first pre-drawing module further comprises:

[0111] segmenting the translation;

[0112] determining whether each translation word in the segmented translation is a punctuation mark;

[0113] If the current translation word is a punctuation mark, merging the punctuation mark and the previous translation word into a group to obtain a translation segmentation list.

[0114] The self-adaptive layout device based on comic translation provided by the application comprises the following steps: S1, after determining a translation layout area, obtaining the top-left vertex coordinate of the translation layout area, and taking the coordinate as a starting position for drawing the translation; S2, based on the starting position and a first drawing font size, pre-drawing the translation words in the translation segmentation list one by one after segmentation, wherein the first drawing font size is the maximum font size corresponding to the translation layout area; S3, if the height of the drawn translation is greater than the height value of the translation layout area, reducing the first drawing font size by one to obtain a second drawing font size; S4, based on the second drawing font size and the translation layout area, pre-drawing the translation words in the translation segmentation list again; S5, repeating S3-S4 until the height of the drawn translation is less than or equal to the height value of the translation layout area, and taking the drawn translation as a target layout result. The application solves the three processes that need strong human participation in the comic translation process, namely, the determination of the best translation area, the determination of the best font size, and the unification of the layout logic, realizes the process unification of comic translation, standardizes the layout standard, reduces the influence of subjective factors in the translation process, reduces the human participation, and saves manpower.

[0115] Embodiment 3:

[0116] The application designs a machine layout framework, as shown in Figure 7 .

[0117] The translation region expansion module based on the average change of the original text background region pixel, the translation multi-direction expansion area maximum module, the loop maximum word number determination module, and the word segmentation optimization module. The specification layout process is reduced, the manual participation is reduced, the manpower is greatly saved, and the logically unified layout comic is output.

[0118] 1. Translation word segmentation module fused with punctuation optimization

[0119] The purpose of the module is to assist in adaptive layout of the translation, prevent the punctuation symbol from appearing at the beginning of the line after layout, and the method process is as follows:

[0120] 1) Word segmentation of the translation

[0121] 2) Loop judgment of whether each character after word segmentation is a punctuation symbol

[0122] 3) If the current character is a punctuation symbol, merge the punctuation symbol and the previous character into a group

[0123] 4) Obtain the optimized word segmentation translation

[0124] The module is not limited by any language, and the word segmentation method can adopt a suitable method according to the grammar of different languages (such as Chinese character-by-character segmentation, English space symbol segmentation, Japanese Kana segmentation, etc.). Taking Chinese as an example, the first line is the original translation, the second line is the word segmentation translation list, and the third line is the word segmentation translation list fused with punctuation optimization. The optimized result takes (song,) and (haha!?) as a whole, and ensures that the punctuation symbol will not appear at the beginning of the line during layout, without the need for manual correction.

[0125] Original text: 'Li Lei is singing, what is he singing?! '

[0126] Normal word segmentation: ['Li', 'Lei', 'is','singing','song', ',', 'he','sings', 'what', 'is', 'he', '?', '!']

[0127] Optimized word segmentation: ['Li', 'Lei', 'is','singing','song', ',', 'he','sings', 'what', 'is', 'he', '?', '!']

[0128] 2. Multi-scale region progressive expansion module based on pixel mean

[0129] The function of the module is to determine the maximum translation layout region, and the main steps are as follows:

[0130] 2.1 Region progressive expansion method based on background region pixel mean change

[0131] The main steps are as follows:

[0132] 1) Given the initial translation region rectangle coordinates, the four clockwise points are represented as box = [(x1,y1),(x2,y1),(x2,y2),(x1,y2)], where (x1,y1) are the x and y coordinates of the top-left vertex. The pixel mean of the current region is obtained as crop_mean. The expansion stop threshold is set to expand_threshold >= 0.

[0133] 2) Set the horizontal expansion step size to 'a' and the vertical expansion step size to 'b', where both 'a' and 'b' are positive integers or 0, and not both 0.

[0134] 3) Select the left boundary of the rectangular region, with the corresponding coordinates of the boundary points being [(x1,y1),(x1,y2)]. Expand outwards to the left to obtain new coordinates [(x1-a,y1),(x1-a,y2)], resulting in a new rectangular region with coordinates [(x1-a,y1),(x2,y1),(x2,y2),(x1-a,y2)]. Calculate the mean pixel value of the currently expanded region as `crop_mean_left`. If `abs(crop_mean_left-crop_mean)>expand_threshold`, then determine the left boundary x-coordinate as `x1` and stop expanding; otherwise, update the left boundary x-coordinate of the region to `x1 = x1-a`.

[0135] 4) Select the upper boundary of the rectangular region, with the corresponding coordinates of the boundary points being [(x1,y1),(x2,y1)]. Expand upwards to obtain new coordinates of [(x1,y1-b),(x2,y1-b)], resulting in a new rectangular coordinate system of [(x1,y1-b),(x2,y1-b),(x2,y2),(x1,y2)]. Calculate the pixel mean of the currently expanded region as crop_mean_top. If abs(crop_mean_top-crop_mean)>expand_threshold, then determine the upper boundary ordinate as y1 and stop expanding; otherwise, update the upper boundary ordinate of the region to y1-b.

[0136] 5) Select the right boundary of the rectangular region, with the corresponding coordinates of the boundary points being [(x2,y1),(x2,y2)]. Expand outwards to the right to obtain new coordinates [(x2+a,y1),(x2+a,y2)], resulting in a new rectangular coordinate system [(x1,y1),(x2+a,y1),(x2+a,y2),(x1,y2)]. Calculate the pixel mean of the currently expanded region as `crop_mean_right`. If `abs(crop_mean_right-crop_mean)>expand_threshold`, then determine the right boundary x-coordinate as `x2` and stop expanding; otherwise, update the right boundary x-coordinate of the region to `x2 = x2+a`.

[0137] 6) Select the lower side boundary of the rectangular region, the boundary corresponding coordinate points are [(x2, y2), (x1, y2)], and the new coordinates are obtained by extending outward on the lower side, that is, [(x2, y2+b), (x1, y2+b)], and the new rectangular coordinates are [(x1, y1), (x2, y1), (x2, y2+b), (x1, y2+b)], and the pixel mean of the current expanded region is calculated as crop_mean_down. If abs(crop_mean_down-crop_mean)>expand_threshold, it is determined that the vertical coordinate of the lower side boundary y2 is no longer expanded; otherwise, the horizontal coordinate of the lower side boundary y2 is updated as y2 = y2 + b

[0138] 7) Loop 3)-6) process until the four directions of up, down, left and right stop expanding, and the best translation region is obtained at this time.

[0139] Note that when expanding, it is necessary to judge whether the expanded coordinates overflow the image boundary. If it overflows, the current direction stops expanding.

[0140] 2.2 Method for determining the maximum translation region area by different direction and multi-scale step expansion

[0141] Based on the method proposed in 2.1, the optimal drawing translation region can be obtained by expanding the initial translation region coordinates. On this basis, a multi-scale step optimal drawing region search method is proposed

[0142] The main steps of the proposed method are as follows:

[0143] 1) Set the multi-scale step scale_anchors = [(a1, b1), (a2, b2),..., (an, bn)] for horizontal and vertical expansion, where a represents the step of horizontal progressive expansion, and b represents the step of vertical progressive expansion.

[0144] 2) Loop scale_anchors, select the i-th step (ai, bi), and use the 1.1 progressive expansion method based on the mean value of the region pixels to obtain the expansion translation region coordinates and area based on the current step

[0145] 3) One of the steps will make the region expansion area maximum, and the step corresponding to the expansion region coordinates is the final translation layout region.

[0146] 3. Font size and layout adaptive module

[0147] This module can adaptively layout the translation according to the translation region coordinates, and can also adapt the font size of the translation according to whether it overflows the given translation region after layout.

[0148] 3.1, the typesetting method based on the word segmentation translation text fills in the translation text area line by line

[0149] The main steps of the method are as follows:

[0150] 1) The left upper vertex horizontal and vertical coordinates of the drawing area of the known translation text are (x1, y1), and the maximum width w of the drawing area; the optimized word segmentation translation text list text=[t1, t2,..., tn]; the drawing font size is font_size; the word segmentation translation text drawing index is index=0; the cumulative drawing width is w_draw=0, and the single line drawing height is h_draw=0; the initial drawing coordinates are (draw_x=x1, draw_y=y1)

[0151] 2) Take out the word segmentation translation text t=text[index], and according to the font size font_size, the length t_w and t_h of the current character t after drawing can be obtained, and w_draw=w_draw+t_w is updated

[0152] 3) If w_draw<w, that is, the word segmentation t does not exceed the limited width, the word segmentation can be drawn, and the drawing starting point is updated as (draw_x=draw_x+t_w, draw_y=draw_y), h_draw=max(h_draw, t_h), and index=index+1;

[0153] Otherwise, the drawing coordinates are updated as (draw_x=x1, draw_y=y1+h_draw), h_draw=0, and w_draw=0

[0154] 4) Return to step 2) and continue to execute until all word segmentation translation texts are drawn.

[0155] 3.2, the maximum font size determination method of the word size cycle self-decrement typesetting text without overflowing the translation text area

[0156] Based on the method proposed in 3.1, the automatic filling and drawing of the machine on the translation text can be realized, and on this basis, the invention further proposes an adaptive font size translation typesetting to prevent the translation text from overflowing the best drawing area after typesetting, and truly realizes the machine adaptive typesetting process.

[0157] The main steps of the method are as follows:

[0158] 1) The known translation text drawing area upper left vertex horizontal and vertical coordinates are (x1, y1), and the maximum width w and height h of the drawing area; the optimized word segmentation translation text list text = [t1, t2,..., tn]; the initial maximum drawing font size is font_size; the word segmentation translation text drawing index is index = 0; the cumulative drawing width is w_draw = 0, the single line drawing height is h_draw = 0, and the cumulative drawing height is h_draw_acc; the initial drawing coordinates are (draw_x = x1, draw_y = y1)

[0159] 2) Take out the word segmentation translation text t = text[index], and according to the font size font_size, the length t_w and t_h of the current character t after drawing in the image can be obtained, if t_w > w, font_size = font_size - 1, go to step 1); otherwise, update w_draw = w_draw + t_w and go to step 3)

[0160] 3) If w_draw < w, that is, the word segmentation t does not exceed the limited width. The drawing starting point is updated as (draw_x = draw_x + t_w, draw_y = draw_y), h_draw = max(h_draw, t_h), index = index + 1; otherwise, the drawing coordinates are updated as (draw_x = x1, draw_y = y1 + h_draw), w_draw = 0, h_draw_acc = h_draw_acc + h_draw, h_draw = 0

[0161] 4) Continue to step 2) until all word segmentation translation texts are drawn, and then go to step 5)

[0162] 5) If h_draw_acc > h, font_size = font_size - 1, go to step 1)

[0163] 6) According to the final font_size, use the 3.1 layout method based on word segmentation translation text to fill in the translation text area line by line to draw the final translation text.

[0164] In summary, the application provides a self-adaptive translated character size and layout framework based on comic translation, comprising a translated text segmentation module with punctuation fusion optimization, a multi-scale region progressive expansion module based on pixel mean, and a character size and layout self-adaptive module. The three processes that require strong human intervention in the comic translation process, i.e., determination of the best translated text region, determination of the best character size, and unification of the layout logic, are solved. The region progressive expansion method based on the change of the pixel mean of the background region, the method of determining the maximum translated text region area by multi-scale expansion step length in different directions, the punctuation-based segmentation optimization method, the layout method based on the row-by-row filling of the segmented translated text in the translated text region, and the maximum character size determination method that the text does not overflow the translated text region after the layout of the character size cyclically decreasing are proposed. The process of comic translation is unified, and the human resources are greatly released.

[0165] The readable storage medium can be a computer storage medium or a communication medium. The communication medium includes any medium that facilitates the transfer of computer program from one place to another. The computer storage medium can be any available medium that can be accessed by a general or special purpose computer. For example, the readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). In addition, the ASIC can be located in a user equipment. Of course, the processor and the readable storage medium can also exist as discrete components in the communication device. The readable storage medium can be a read only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0166] The application further provides a program product, which comprises execution instructions stored in a readable storage medium. At least one processor of a device can read the execution instructions from the readable storage medium, and the at least one processor executes the execution instructions to enable the device to implement the method provided by various embodiments described above.

[0167] In the above-mentioned embodiments of the terminal or the server, it should be understood that the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), etc. The general-purpose processor can be a microprocessor, or the processor can also be any conventional processor, etc. The steps of the method disclosed in the application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0168] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An adaptive typesetting method based on comic translation, characterized in that, include: S11, multiple sets of expansion step values ​​in different directions are preset, and the expansion step values ​​are used to gradually expand the translation area in the horizontal and / or vertical directions; S12, Based on the region progressive expansion method and the expansion step size value of each group, the initial translation region is expanded to obtain candidate expansion regions, and the area of ​​the candidate expansion region is calculated; S13, Select the expansion step size value with the largest expansion area, and use the candidate expansion area corresponding to this step size value as the translation typesetting area; S1, After determining the translation layout area, obtain the coordinates of the upper left vertex of the translation layout area, and use these coordinates as the starting position for drawing the translation; S2, based on the starting position and the first drawing font size, pre-draw each translated word in the word segmentation list of the translated text after word segmentation, where the first drawing font size is the largest font size corresponding to the translated text layout area; S3, If the height of the drawn translation is greater than the height of the translation layout area, then the first drawn font size is reduced by one to obtain the second drawn font size; S4, Based on the second drawing font size and translation typesetting area, the translated words in the translation word segmentation list are re-pre-drawn; S5, repeat S3-S4 until the height of the drawn translation is less than or equal to the height of the translation typesetting area, and use the drawn translation as the target typesetting result.

2. The adaptive typesetting method based on comic translation according to claim 1, characterized in that, The process of pre-drawing each translated word in the segmented word list based on the starting position and the first drawing font size includes: Determine if the width of the currently drawn translation is greater than the maximum width of the layout area; If it is greater than the given value, the ordinate of the starting position is added to the single-line drawing height value to obtain the first starting position; the current translated word is drawn based on the first starting position and the first drawing font size. If it is not greater than, the x-coordinate of the starting position is added to the width value of the currently drawn translation excluding the current translation word to obtain the second starting position; the current translation word is drawn based on the second starting position and the first drawing font size.

3. The adaptive typesetting method based on comic translation according to claim 1, characterized in that, Step S2 also includes: Segment the translation into words; Determine whether each word in the segmented translation is a punctuation mark; If the current translated word is a punctuation mark, then the punctuation mark is merged with the previous translated word into a group to obtain a translated word segmentation list.

4. The adaptive typesetting method based on comic translation according to claim 1, characterized in that, The method based on the progressive expansion of regions and the expansion step size value for each group expands the initial translation region to obtain candidate expansion regions, and calculates the area of ​​the candidate expansion regions, including: Obtain the rectangular coordinates of the initial translation region, where the rectangular coordinates are the coordinates of the four vertices of the initial translation region (top, bottom, left, and right), and calculate the average pixel value of the initial translation region based on the rectangular coordinates. Select any one of the side boundaries of the initial translation region, wherein the side boundaries include: left side boundary, top side boundary, right side boundary, and bottom side boundary; Based on the expansion step size value, the side boundary is expanded outward in the corresponding side direction to obtain the current expansion area, and the pixel mean value of the current expansion area is calculated according to the rectangular coordinates of the current expansion area; Based on the average pixel values ​​of the current extended region and the initial translation region, the side boundary coordinate values ​​of the candidate extended region are determined. The side boundary coordinate values ​​include: the ordinate of the upper side boundary, the ordinate of the lower side boundary, the abscissa of the left side boundary, and the abscissa of the right side boundary.

5. An adaptive typesetting device based on comic translation, characterized in that, include: An expansion step setting module is used to pre-set multiple sets of expansion step values ​​in different directions, wherein the expansion step values ​​are used to progressively expand the translation area in the horizontal and / or vertical directions; The candidate expansion region generation module is used to expand the initial translation region to obtain candidate expansion regions based on the region progressive expansion method and each set of expansion step size values, and to calculate the area of ​​the candidate expansion region. The translation typesetting area determination module is used to select the expansion step size value with the largest expansion area and use the candidate expansion area corresponding to the step size value as the translation typesetting area; The starting position acquisition module is used to acquire the coordinates of the upper left vertex of the translation layout area after the translation layout area is determined, and use the coordinates as the starting position for drawing the translation. The first pre-drawing module is used to pre-draw each translated word in the word segmentation list of the translated text after word segmentation based on the starting position and the first drawing font size. The first drawing font size is the largest font size corresponding to the translation typesetting area. The translation height judgment module is used to subtract one from the first drawn font size to obtain the second drawn font size if the height of the drawn translation is greater than the height of the translation typesetting area. The second pre-drawing module, based on the second drawing font size and the translation layout area, re-draws the translated words in the translation word segmentation list; The target typesetting result generation module is used to repeatedly execute the translation height judgment module and the second pre-drawing module until the height of the drawn translation is less than or equal to the height value of the translation typesetting area, and then use the drawn translation as the target typesetting result.

6. The adaptive typesetting device based on comic translation according to claim 5, characterized in that, The first pre-drawing module includes: Determine if the width of the currently drawn translation is greater than the maximum width of the layout area; If it is greater than the given value, the ordinate of the starting position is added to the single-line drawing height value to obtain the first starting position; the current translated word is drawn based on the first starting position and the first drawing font size. If it is not greater than, the x-coordinate of the starting position is added to the width value of the currently drawn translation excluding the current translation word to obtain the second starting position; the current translation word is drawn based on the second starting position and the first drawing font size.

7. The adaptive typesetting device based on comic translation according to claim 5, characterized in that, The first pre-drawing module further includes: Segment the translation into words; Determine whether each word in the segmented translation is a punctuation mark; If the current translated word is a punctuation mark, then the punctuation mark is merged with the previous translated word into a group to obtain a translated word segmentation list.

8. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method described in any one of claims 1 to 4.

Images