An image conversion method, device, computer equipment and storage medium

Abstract

The disclosure provides an image conversion method and device, computer equipment and a storage medium, and the method comprises the following steps: obtaining an original image to be converted; detecting a first image region corresponding to an image grid from the original image; the original image comprises a plurality of image grids; performing region range expansion processing on the first image region to obtain a second image region corresponding to the image grid; obtaining an image grid image based on the first image region and the second image region; and generating a target image corresponding to the original image based on the image grid image.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more specifically, to an image conversion method, apparatus, computer device, and storage medium. Background Technology

[0002] With the development of mobile communication technology, various lightweight and portable mobile smart terminals (such as smartphones) have brought great convenience to people's lives, and the number of users of mobile smart terminals is gradually increasing. At the same time, this raises the question of how to display images originally shown on desktops on mobile devices, or how to scan and display scanned paper documents on mobile devices. The traditional solution is mostly to directly migrate the original image to the mobile device according to its layout and style.

[0003] However, the above display method will cause problems with page layout that is not compatible with mobile screen size, and may even cause some image content to be unable to be displayed, which will seriously affect the user's reading experience. Summary of the Invention

[0004] This disclosure provides at least one image conversion method, apparatus, computer device, and storage medium.

[0005] In a first aspect, embodiments of this disclosure provide an image conversion method, including:

[0006] Obtain the original image to be converted;

[0007] The first image region corresponding to the image grid is detected from the original image; the original image includes multiple image grids;

[0008] The first image region is expanded to obtain the second image region corresponding to the image grid.

[0009] Based on the first image region and the second image region, an image grid image is obtained;

[0010] Based on the image grid, a target image corresponding to the original image is generated.

[0011] In one possible implementation, detecting the first image region corresponding to the image grid from the original image includes:

[0012] Identify the dialog box in the original image, segment the dialog box out of the original image, and obtain an updated original image;

[0013] From the updated original image, the first image region corresponding to the image grid is detected.

[0014] In one possible implementation, obtaining an image grid image based on the first image region and the second image region includes:

[0015] Based on the region mask information of the first image region, image grid detection is performed in the second image region to determine the corner positions of the image grid image;

[0016] Based on the determined corner positions and the segmented dialog box, the image grid is obtained.

[0017] In one possible implementation, the image grid is obtained based on the determined corner point positions and the segmented dialog box, including:

[0018] Based on the determined positions of each corner point, an image grid image that does not contain the dialog box is obtained;

[0019] Based on the image grid corresponding to each dialog box, each dialog box is pasted back to the corresponding image grid image that does not contain the dialog box, thus obtaining an image grid image carrying the dialog box.

[0020] In one possible implementation, identifying the dialog box in the original image and segmenting the dialog box from the original image to obtain an updated original image includes:

[0021] Determine the mask information of the dialog box in the original image;

[0022] Based on the mask information of the dialog box, the dialog box located in each layer is deleted from the original image to obtain the image to be repaired; wherein, in the image to be repaired, the pixel value of the pixel to be repaired at the position corresponding to the dialog box is a preset value;

[0023] Based on the pixel values ​​of all pixels in the image to be repaired, excluding the pixel to be repaired, the pixel values ​​of the pixel to be repaired are adjusted to obtain the updated original image.

[0024] In one possible implementation, generating the target image corresponding to the original image based on the image grid image includes:

[0025] Based on the reading order of the image grids in the original image, determine the arrangement order of each image grid in the original image in a preset direction;

[0026] According to the arrangement order, the image grid images corresponding to each of the image grids are arranged sequentially in the preset direction to generate the target image.

[0027] In one possible implementation, the target image is generated by sequentially arranging the image grids corresponding to each of the image grids in the preset direction according to the arrangement order, including:

[0028] The image grids corresponding to each of the aforementioned image grids are adjusted to have the same width or the same length; wherein, the aspect ratio of the image grids remains unchanged after adjustment compared to before adjustment;

[0029] The target image is generated by arranging the image grids of the same width or length sequentially in the preset direction according to the arrangement order.

[0030] Secondly, embodiments of this disclosure also provide an image conversion apparatus, comprising:

[0031] The acquisition module is used to acquire the original image to be converted;

[0032] The first detection module is used to detect a first image region corresponding to an image grid from the original image; the original image includes multiple image grids.

[0033] An adjustment module is used to expand the region of the first image region to obtain the second image region corresponding to the image grid.

[0034] The second detection module is used to obtain an image grid image based on the first image region and the second image region;

[0035] The generation module is used to generate a target image corresponding to the original image based on the image grid image.

[0036] Thirdly, an optional implementation of this disclosure also provides a computer device, a processor, and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is configured to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, the steps of the first aspect above, or any possible implementation of the first aspect, are performed.

[0037] Fourthly, an optional implementation of this disclosure also provides a computer-readable storage medium storing a computer program that, when run, performs the steps of the first aspect or any possible implementation of the first aspect.

[0038] For a description of the effects of the aforementioned image conversion apparatus, computer equipment, and computer-readable storage medium, please refer to the description of the image conversion method above; it will not be repeated here.

[0039] The image conversion method, apparatus, computer device, and storage medium provided in this disclosure, through preliminary detection of the original image, can first detect the first image region corresponding to each image grid in the original image, thus achieving preliminary division of the image grids included in the original image. By expanding the region range of the first image region to obtain the second image region, and then combining the first and second image regions for secondary detection, accurate division of the region corresponding to each image grid can be achieved, thereby determining the accurate image grid image. Finally, the detected image grid images are used to generate the target image, which can ensure the consistency between the image grid images in the converted target image and the image grid images in the original image, and can also generate a target image suitable for mobile devices, thereby achieving reasonable layout and display of the original image on the mobile device. In addition, after obtaining the image grid images, the image grid images can be rearranged (that is, the layout of the original image can be adjusted) so that the rearranged layout matches the display device, and then the rearranged image can be displayed on the display device, thereby improving the layout rationality of the display.

[0040] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0042] Figure 1 A flowchart of an image conversion method provided by an embodiment of this disclosure is shown;

[0043] Figure 2 A schematic diagram of an original image provided by an embodiment of this disclosure is shown;

[0044] Figure 3 This diagram illustrates a first image region provided by an embodiment of the present disclosure;

[0045] Figure 4 A schematic diagram of an image grid provided in an embodiment of this disclosure is shown;

[0046] Figure 5A schematic diagram of a target image provided by an embodiment of this disclosure is shown;

[0047] Figure 6 This illustration shows a specific implementation diagram of an image conversion method provided in an embodiment of the present disclosure;

[0048] Figure 7 A schematic diagram of an image conversion apparatus provided in an embodiment of this disclosure is shown;

[0049] Figure 8 A schematic diagram of the structure of a computer device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown herein can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0051] Furthermore, the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein.

[0052] In this article, "multiple or several" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0053] Research has revealed that as comic reading has gradually shifted from desktop to mobile smart devices, a need has arisen to convert original images displayed on desktops (such as page-based comics) into target images (such as strip comics) suitable for mobile smart devices. Traditional conversion methods often directly migrate the original images to mobile devices, preserving their original layout and style. However, since mobile devices are mostly portrait-oriented, directly migrating the original images not only causes page layout issues but also results in some image content not being displayed, severely impacting the user's reading experience. Therefore, this disclosure addresses the problem of how to reasonably convert original images displayed on desktops into target images suitable for display on mobile smart devices.

[0054] Based on the above research, this disclosure provides an image conversion scheme. By performing preliminary detection on the original image, a first image region corresponding to each image grid can be detected, achieving a preliminary division of the image grids included in the original image. By expanding the region of the first image region to obtain a second image region, and then combining the first and second image regions for secondary detection, accurate division of the region corresponding to each image grid can be achieved, thereby determining the accurate image grid image. Finally, the detected image grid images are used to generate the target image, which ensures the consistency between the image grid images in the converted target image and the image grid images in the original image, and generates a target image suitable for mobile devices, thereby achieving reasonable layout and display of the original image on mobile devices. In addition, after obtaining the image grid images, the image grid images can be rearranged (i.e., the layout of the original image can be adjusted) so that the rearranged layout matches the display device. Then, by displaying the rearranged image on the display device, the layout rationality of the display can be improved.

[0055] The shortcomings of the above solutions are the result of the inventor's practical experience and careful research. Therefore, the discovery process of the above problems and the solutions proposed in this disclosure below should be considered as the inventor's contribution to this disclosure.

[0056] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0057] To facilitate understanding of this embodiment, a detailed description of the image conversion method disclosed in this disclosure will be provided first. The execution subject of the image conversion method provided in this disclosure is generally a terminal device or other processing device with certain computing power. The terminal device may be a user equipment (UE), mobile device, user terminal, terminal, personal digital assistant device (PDA), handheld device, computer device, etc. In some possible implementations, the image conversion method can be implemented by the processor calling computer-readable instructions stored in the memory.

[0058] The image conversion method provided in this disclosure embodiment will be described below using a computer device as an example.

[0059] like Figure 1 The flowchart shown is a method for image conversion provided in an embodiment of this disclosure, which may include the following steps:

[0060] S101: Obtain the original image to be converted.

[0061] Here, the original image can be a comic book image in any image format, such as an RGB format comic book image. The reading medium can be print media or a desktop (such as a personal computer, PC). An original image can include multiple image panels, each corresponding to a single image panel, i.e., a comic book storyboard. The size and shape of different image panels can vary. The reading order of the image panels in the original image can be from left to right, from top to bottom, or from right to left, from top to bottom. For example, the original image can also be called a page comic image.

[0062] An original image can be divided into multiple rows, and each row can contain one or more image cells. For example... Figure 2 The diagram shown is a schematic of an original image provided in an embodiment of this disclosure. The original image is divided into three rows: the first row includes an image cell 1, the second row includes image cells 2 and 3, and the third row includes image cells 4 and 5.

[0063] S102: Detect the first image region corresponding to the image grid from the original image; the original image includes multiple image grids.

[0064] Here, the first image region is the image region corresponding to the initially detected image grid in the page image. The area of ​​the first image region corresponding to an image grid can be greater than or equal to the actual image area of ​​that image grid. Specifically, the first image region can be a rectangular area.

[0065] In practice, image grid recognition can be performed on the acquired original image to determine each image grid included in the original image. Then, for each image grid, the image region corresponding to that image grid can be identified from the original image, and that image region is taken as the first image region corresponding to that image grid.

[0066] In one embodiment, for S102, the original image can be input into a pre-trained image grid detection network, and the image grid detection network can be used to perform image detection on the original image to determine the first image region corresponding to each image grid in the original image.

[0067] Here, the image grid detection network is a neural network used to detect the first image region corresponding to each image grid in the original image. Specifically, the original image can be input into the image grid detection network, and after the image grid detection network's recognition processing, it outputs the first image region corresponding to each image grid in the original image.

[0068] Thus, since the trained neural network has reliable prediction accuracy, the original image can be detected by using the pre-trained image grid detection network, and the accurate first image region corresponding to each image grid can be determined.

[0069] In one embodiment, since the original image is a cartoon image, each image cell may typically include a dialog box for displaying the corresponding character. These dialog boxes often have irregular shapes, so the edges of the dialog boxes may affect the recognition result when identifying the first image region. Therefore, to improve the accuracy of the identified first image region, the above-mentioned S102 can be implemented according to the following steps:

[0070] S102-1: Identify the dialog box in the original image, segment the dialog box from the original image, and obtain the updated original image.

[0071] Here, dialog boxes, also known as speech bubbles, are used to display dialogue information from anime characters or monologues in the original image. An original image can include one or more dialog boxes, and one image cell in the original image can correspond to one or more dialog boxes. Dialog boxes in the original image can reside on a different layer than image cells.

[0072] In practice, a bubble recognition algorithm can be used to identify bubbles in the original image, thus determining the individual dialog boxes included in the image. Each dialog box represents a speech bubble. Then, a bubble segmentation algorithm can be used to segment each identified dialog box from the original image. After segmenting the dialog boxes, the original image can be updated, resulting in an updated image that does not include the dialog boxes.

[0073] In one embodiment, S102-1 can be implemented according to the following steps:

[0074] S102-1-1: Determine the mask information of the dialog box in the original image.

[0075] Here, the dialog box mask information is used to characterize the layer on which the dialog box is located, the area and location information of the dialog box, and the image grid information to which the dialog box belongs. The image grid information indicates the image grid in which the dialog box is located.

[0076] In practice, a bubble segmentation algorithm can be used to determine the region range and location information of all dialog boxes included in the original image (which can be represented by a bubble mask). Then, a connected domain algorithm can be used to distinguish each dialog box, determining the layer in which each dialog box is located and the image grid information to which each dialog box belongs. The layer in which the dialog box is located can include the layer containing the dialog box graphic and the layer containing the text within the dialog box. Based on this, the mask information of each dialog box is obtained.

[0077] S102-1-2: Based on the mask information of the dialog boxes, the dialog boxes located in each layer are segmented from the original image to obtain an updated original image.

[0078] Here, the updated original image is the original image excluding the dialog box.

[0079] For example, based on the area range and location indicated by the mask information of the dialog box, the dialog box in each layer can be removed from the original image. For instance, the pixel value of the pixel at the corresponding location of the dialog box can be set to a preset value. Based on this, an updated original image can be obtained.

[0080] In one embodiment, S102-1-2 can be implemented according to the following steps:

[0081] S102-1-2-1: Based on the mask information of the dialog box, the dialog boxes located in each layer are removed from the original image to obtain the image to be repaired; wherein, in the image to be repaired, the pixel value of the pixel to be repaired at the corresponding position of the dialog box is a preset value.

[0082] Here, the image to be repaired is the original image with the dialog boxes in the layers removed. In the image to be repaired, for each layer, the pixel at the location corresponding to each dialog box in that layer is the pixel to be repaired, and the pixel value of the pixel to be repaired is a preset value. The preset value can be a fixed value, such as 0 or 1.

[0083] For example, for each layer, the position of each dialog box in the layer can be determined based on the area range and location indicated by the mask information of each dialog box in the layer. Then, the value of each pixel to be repaired at that position can be set to a preset value, thus enabling the deletion of the dialog box in the layer.

[0084] S102-1-2-2: Based on the pixel values ​​of all pixels in the image to be repaired, excluding the pixel to be repaired, adjust the pixel values ​​of the pixel to be repaired to obtain the updated original image.

[0085] Here, with the pixels to be repaired set to preset values, the pixel values ​​in the area corresponding to the dialog box may differ significantly from those in other parts of the original image, affecting the user's viewing experience. Therefore, it is necessary to adjust the pixel values ​​of each pixel to be repaired to reduce the difference between the pixel values ​​in the dialog box area and other areas, making the colors of the original image after removing the dialog box appear more natural.

[0086] In practice, a comic book restoration algorithm can be used to adjust the pixel values ​​of the pixels to be restored in the image to obtain an updated original image. For example, using a comic book restoration algorithm, the average pixel value is determined based on the pixel values ​​of all pixels in the image except the pixel to be restored. This average pixel value can then be used as the pixel value for each pixel to be restored, thereby adjusting the pixel values ​​of each pixel and ultimately obtaining the updated original image.

[0087] Alternatively, for each pixel to be repaired, a comic book restoration algorithm can be used to determine the weight of each pixel based on the distances between it and all other pixels. Then, a target pixel value can be determined according to the weights and values ​​of each pixel. This target pixel value is then used to adjust the pixel value of the pixel to be repaired, resulting in a pixel with the target pixel value. In this way, based on the adjusted pixel values ​​of the pixels to be repaired and all other pixels, an updated original image can be obtained.

[0088] In this way, by recognizing the original image with the dialog box removed, the shape of the dialog box can be effectively prevented from affecting the network detection results, thereby improving the accuracy of recognizing the first image region and the subsequently determined corner positions.

[0089] S102-2: Detect the first image region corresponding to the image grid from the updated original image.

[0090] For example, an image grid detection network can be used to perform image detection on the updated original image to obtain the first image region corresponding to each image grid in the updated original image.

[0091] like Figure 3 The diagram shown is a schematic representation of a first image region provided in an embodiment of this disclosure, wherein, when the image region is as follows... Figure 2 The original image shown is input into the image grid detection network, which then outputs the first detection box for the first image region corresponding to each image grid. Figure 3 In the diagram, the first detection box corresponding to the first image region for each image grid is marked with a dashed line. Understandably, in practice, the first detection box corresponding to each image grid can be marked with a detection box of a first preset color, such as green or yellow. Since each first image region is identified from the updated original image, Figure 3 None of the first image areas in the document contain dialog boxes.

[0092] S103: Expand the region of the first image region to obtain the second image region corresponding to the image grid.

[0093] Here, the second image region is the image region obtained by expanding the scope of the first image region, excluding the dialog box. Compared to the first image region, the second image region has a larger image range and contains richer image information. For example, the second image region corresponding to an image cell, compared to the corresponding first image region, can typically include more boundary information of other adjacent image cells.

[0094] For example, targeting Figure 3 Compared to the first image region corresponding to image grid 2, the second image region corresponding to image grid 3 can include more information related to the edges of image grid 3. Since the shape of image grids is usually irregular, the more edge-related information is available in the second image region for image recognition, the more auxiliary reference information can be used for recognition. Therefore, recognizing the image grid corresponding to image grid 2 based on more auxiliary reference information can improve the accuracy of the recognized image grid edges.

[0095] For example, for each first image region, a second image region can be obtained by expanding the region in the length and / or width directions of the first image region in the updated original image according to the length ratio of the first image region and a preset expansion distance. Specifically, for any target direction (length or width), if the expanded distance is equal to the maximum distance of the original image in the target direction, the expansion in the target direction can be stopped, and this distance is taken as the final distance of the second image region in the target direction.

[0096] S104: Obtain an image grid based on the first image region and the second image region.

[0097] Here, the image grid image is the image corresponding to the image grid in the original image.

[0098] For example, the regional feature information of the first image region and the regional feature information of the second image region can be convolved, and the image grid image can be determined using the result of the convolution process.

[0099] In one embodiment, the above-described S104 can be implemented according to the following steps:

[0100] S104-1: Based on the region mask information of the first image region, perform image grid detection in the second image region to determine the corner positions of the image grid image.

[0101] Here, the region mask information is used to represent the location information of the first image region and the layer information of the image grid corresponding to the first image region. The corner point position is the intersection point between the boundary lines in the image boundary of the image grid image. Since the image grid image can have a quadrilateral shape, the number of corner points corresponding to the image grid image can be 4.

[0102] In practice, for each second image region, image grid detection can be performed within the second image region based on the region mask information of the corresponding first image region to determine the corner positions of the image grid images. Since the first image region is the initially detected image region associated with the image grid, using the region mask information of the first image region to determine the image grid images allows for location determination based on the position of the first image region indicated by the region mask information, effectively avoiding the problem of the determined corner positions deviating from the actual positions.

[0103] In one embodiment, for S104-1, the region mask information of each first image region and the corresponding second image region can be input into a pre-trained position regression neural network. The position regression neural network is used to process the region mask information and the second image region to obtain the corner positions of the output image grid.

[0104] Here, the position regression neural network is a type of neural network used to regress the corner positions corresponding to image grids. Since image grids can have various shapes, such as regular rectangles, squares, or irregular quadrilaterals, after detecting the first image region of the rectangle corresponding to each image grid using the image grid detection network, shape regression is performed using the position regression neural network. This allows for the accurate determination of the corner positions of image grids with irregular quadrilateral shapes by using the region mask information of the first image region and referring to the edge information of adjacent image grids in the second image region, as well as the reasonable removal of redundant parts in the second image region, thereby improving the accuracy of the determined image grids.

[0105] In practice, each second image region can be cropped from the original image to obtain the cropped image corresponding to the second image region. Then, for each second image region, the cropped image of the second image region and the mask information of the corresponding first image region can be input into the position regression neural network. The position regression neural network processes the input information and then regresses the corner positions of the four corner points of the image grid corresponding to the second image region.

[0106] S104-2: Based on the determined corner positions and the segmented dialog boxes, obtain the image grid image.

[0107] In practice, S104-2 can be implemented according to the following steps:

[0108] S104-2-1: Based on the determined positions of each corner point, obtain an image grid that does not contain dialog boxes.

[0109] For example, for each image grid, the four corner positions regressed can be used to crop out an image grid that does not contain a dialog box from the second image area.

[0110] S104-2-2: Based on the image grid corresponding to each dialog box, paste each dialog box back to the corresponding image grid image that does not contain a dialog box, to obtain an image grid image that carries the dialog box.

[0111] For example, after obtaining the individual image grids that do not contain dialog boxes, the image grid corresponding to each dialog box segmented from the original image can be determined. Each segmented dialog box is then pasted back to its corresponding image grid, thus obtaining individual image grids carrying dialog boxes. In this way, by pasting the dialog boxes back to their corresponding image grids that do not contain dialog boxes, and subsequently using the image grids carrying dialog boxes to generate the target image, the integrity of each image grid in the generated target image can be guaranteed, achieving a lossless conversion of the original image into a target image that can be reasonably displayed on a mobile device.

[0112] like Figure 4 The image shown is a schematic diagram of an image grid provided in an embodiment of this disclosure, wherein the image is generated using... Figure 3 The second image region, determined by the first image region shown, along with the region mask information of the first image region, are input into a location regression neural network. The network outputs the corner positions of the image grid. Then, based on the output intersection positions and the segmented dialog boxes, the image grid image can be obtained. Each image grid image can be identified using a second detection box. Figure 4 In the diagram, the first detection box of the first image region corresponding to each image grid is marked with a dashed line, and the second detection box of the image grid corresponding to each image grid is marked with a solid line. Each image grid also contains a dialog box. Understandably, in specific implementations, the second detection box corresponding to each image grid can be marked with a detection box of a second preset color, such as red or blue, and the second preset color is different from the first preset color.

[0113] Thus, because the trained neural network has reliable prediction accuracy, using the trained position regression neural network to regress corner positions can yield accurate corner locations. Then, using the corner positions and the segmented dialog boxes to determine the image grid improves both the accuracy and integrity of the resulting image grid. Using two different neural networks to segment the image grid effectively enhances the accuracy of each resulting image grid.

[0114] Optionally, the image grid detection network and the location regression neural network can each be a branch of a pre-trained target neural network. After obtaining the original image, it can be input into the target neural network, where the image grid detection network branch determines the first image region corresponding to the output image grid. Then, the region processing branch in the target neural network can be used to expand the region of the first image region to obtain the second image region corresponding to the image grid. Finally, the location regression neural network branch in the target neural network can be used to output the individual image grids based on the region mask information of each first image region and the corresponding second image region.

[0115] S105: Generate the target image corresponding to the original image based on the image grid image.

[0116] Here, the target image can be viewed on a mobile smart terminal, such as a smartphone. A target image may contain multiple image panels, each corresponding to a single image panel, which represents a comic strip panel. The image panels can be read from top to bottom, with the target image divided into multiple rows, each containing only one image panel. For example, the target image may be a PSD format image, where PSD is a proprietary format of the graphic design software Photoshop. For example, the target image may be referred to as a strip comic image.

[0117] In practice, after detecting each image grid in the original image, each image grid can be arranged in a preset direction (such as the vertical direction) (with only one image grid arranged in a row) to obtain the target image. In this way, the original image can be converted into a target image that can be displayed on a mobile device.

[0118] For example, after obtaining the individual image grids, the layout of the original image can be directly modified using these grids to obtain a rearranged layout that matches the display device. To illustrate this, let's take modifying a traditional newspaper layout to create a version that can be displayed on a mobile device. Specifically, the various modules in the traditional newspaper can be identified, and then, based on the size of the mobile display area, these modules can be rearranged. This ensures that the information in each rearranged module is fully displayed on the mobile device, and that the modules within the rearranged layout are coordinated without gaps. In this way, by modifying the layout and displaying the modified image on a mobile device, the rationality of the displayed content can be improved.

[0119] In one embodiment, S105 can be implemented according to the following steps:

[0120] S105-1: Determine the arrangement order of each image grid in the original image in a preset direction based on the reading order of the image grid in the original image.

[0121] Here, the image grid reading order is the pre-specified reading order of each image grid in the original image, such as from left to right or from top to bottom. The preset direction can be the reading direction corresponding to the mobile device, such as the vertical direction or the horizontal direction.

[0122] In practice, the original image can be traversed according to the image grid reading order. Based on the image grids traversed each time, the reading order of each image grid in the original image can be determined. Then, this reading order can be used as the arrangement order of each image grid in a preset direction.

[0123] For example, determining the arrangement order of each image grid in a preset direction can be performed using an image grid detection neural network. Specifically, after inputting the original image into the image grid detection neural network, the network can traverse the first image region corresponding to each identified image grid according to the image grid reading order, thereby determining the reading order of each image grid in the original image, and then outputting the arrangement order of each image grid in the preset direction based on this reading order. The arrangement order of each image grid in the preset direction can be identified by an Index.

[0124] exist Figure 3 In the diagram, image grid 1 can be the first position, identified by Index 1; image grid 2 can be the second position, identified by Index 2; image grid 3 can be the third position, identified by Index 3; image grid 4 can be the fourth position, identified by Index 4; and image grid 5 can be the fifth position, identified by Index 5.

[0125] S105-2: Arrange the image grids corresponding to each image grid in a preset direction according to the arrangement order to generate the target image.

[0126] For example, the image grids corresponding to each image grid in the original image can be arranged sequentially from top to bottom in the vertical direction to obtain the arranged target image.

[0127] In one embodiment, S105-2 can be implemented according to the following steps:

[0128] S105-2-1: Adjust the image grid images corresponding to each image grid to image grid images with the same width or the same length; wherein, the aspect ratio of the image grid images remains unchanged after adjustment compared to before adjustment.

[0129] Here, adjusting the image grids to have the same width or the same length is called resizing the image grids. Since the image grids can be irregular quadrilaterals, adjusting the width of each image grid to be the same means that the maximum width of the image grids is the same; adjusting the length of each image grid to be the same means that the maximum length of the image grids is the same.

[0130] For example, when the mobile device is displayed in portrait mode, the width of the mobile device's screen used to display the original image (i.e., the horizontal length of the mobile device) can be used as the target length. Then, for each image grid, it can be adjusted to have the target length according to its aspect ratio. Here, the length direction of the image grid is the horizontal direction.

[0131] Alternatively, you can first determine the target image grid with the longest length based on the length of the image grid corresponding to each image grid (i.e., the length in the horizontal direction), and then determine whether the length of the target image grid is less than or equal to the width of the mobile screen. If so, then take the length of the target image grid as the target length, and for each image grid except the target image grid, adjust the image grid to have the target length according to the aspect ratio of the image grid.

[0132] In this way, by adjusting the size of each image grid according to its aspect ratio, each image grid can be adjusted to have a target length. This allows for obtaining image grids of the same length that are compatible with mobile devices without distortion.

[0133] For example, when the mobile device is displayed in landscape mode, the length of the mobile device's screen (i.e., the length in the vertical direction) used to display the original image can be used as the target width. Then, for each image grid, the image grid can be adjusted to have the target width according to its aspect ratio. Here, the width direction of the image grid is the vertical direction.

[0134] Alternatively, you can first determine the target image grid with the largest width based on the width (i.e., the length in the vertical direction) of each image grid. Then, determine whether the width of the target image grid is less than or equal to the length of the mobile screen. If so, take the width of the target image grid as the target width. Then, for each image grid except the target image grid, adjust it to have the target width according to its aspect ratio.

[0135] In this way, by adjusting the size of each image grid according to its aspect ratio, each image grid can be adjusted to have a target width. This allows for the production of image grids with the same width that are compatible with mobile devices, while ensuring that the image grids are not distorted.

[0136] S105-2-2: Arrange image grids of the same width or length in a preset direction according to the arrangement order to generate the target image.

[0137] Here, when the mobile device is displayed in portrait mode, the default orientation can be vertical; when the mobile device is displayed in landscape mode, the default orientation can be horizontal.

[0138] For example, if all the image grids have the same length, the image grids with the target length can be arranged vertically from top to bottom according to the arrangement order corresponding to each image grid, thus obtaining the target image. If all the image grids have the same width, the image grids with the target width can be arranged horizontally from left to right according to the arrangement order corresponding to each image grid, thus obtaining the target image.

[0139] Optionally, since the image grid contains dialog boxes, the dialog boxes will also adjust adaptively when the image grid is resized. However, since the size of the dialog box usually only occupies a specific proportion of the area of ​​the image grid, if the dialog box is adjusted synchronously according to the resizing ratio of the image grid, there may be overall distortion of the dialog box or distortion of some content within the dialog box. To ensure that the dialog boxes carried in the image grid are not distorted after resizing, in specific implementations, the size of the dialog boxes in each image grid can be adjusted individually, taking into account the requirements for the clarity of the text display within the dialog box and the information display requirements of the image grid outside the dialog box. For example, in one embodiment, for each dialog box corresponding to an image grid, the dialog box can be resized according to its aspect ratio and the degree of resizing of the corresponding image grid, resulting in a resized dialog box. The resizing ratio of the dialog box can be different from that of the image grid; for example, the reduction in the size of the dialog box can be less than the reduction in the size of the image grid to facilitate the display of text within the dialog box. Next, the resized dialog box is pasted back onto the corresponding resized image grid image, thereby improving the consistency between the dialog box and the image grid image, resulting in an image grid image carrying the distortion-free dialog box. Finally, the target image can be generated using the image grid images carrying the distortion-free dialog box. For example... Figure 5 The image shown is a schematic diagram of a target image provided in an embodiment of this disclosure, wherein... Figure 5The target image is obtained by using the image conversion method provided in the embodiments of this disclosure. Figure 2 The image shown is obtained after conversion of the original image. Figure 5 Each image grid in the image has a target length, and the relative positional relationship between the dialog box in each image grid and the corresponding image grid in the original image is consistent with the relative positional relationship between the dialog box and the corresponding image grid in the original image.

[0140] Alternatively, one could first determine if the size of the resized image grid corresponding to the dialog box is larger than the size of the dialog box. If so, the dialog box can be directly pasted back to any position within the resized image grid. If not, the dialog box can be resized according to its proportions until the ratio between the resized dialog box and the resized image grid reaches a set ratio, resulting in a resized dialog box. Then, this resized dialog box can be pasted back to any position within the resized image grid, resulting in an image grid carrying the undistorted dialog box. Finally, the target image can be generated using the image grids containing the undistorted dialog box.

[0141] In this way, by performing preliminary detection on the original image, the first image region corresponding to each image grid can be detected, achieving a preliminary division of the image grids included in the original image. By expanding the region of the first image region to obtain the second image region, and then combining the first and second image regions for secondary detection, the region corresponding to each image grid can be accurately divided, thus determining the accurate image grid image. Finally, the detected image grid images are used to generate the target image, which not only ensures the consistency of the image grid images in the converted target image with those in the original image, but also generates a target image suitable for mobile devices, thereby achieving reasonable layout and display of the original image on mobile devices. In addition, after obtaining the image grid images, the image grid images can be rearranged (i.e., the layout of the original image can be adjusted) so that the rearranged layout matches the display device. Then, by using the display device to display the rearranged image, the layout rationality of the display can be improved.

[0142] like Figure 6 The diagram illustrates a specific implementation of an image conversion method provided in this disclosure. After acquiring the original image to be converted, a bubble segmentation operation can be performed to determine each dialog box in the original image. Specifically, a bubble segmentation algorithm can be used to determine each dialog box in the original image, and then a connected domain algorithm can be used to determine the layer where each dialog box is located and the image grid information to which each dialog box belongs. Thus, the mask information of each dialog box can be obtained.

[0143] Next, perform the comic restoration operation. Specifically, delete the dialog boxes located in each layer, and then use the comic restoration algorithm to adjust the pixel values ​​of the pixels to be restored, thereby obtaining the original image with the dialog boxes removed.

[0144] Next, an image grid detection operation can be performed. That is, the image grid detection network can be used to perform image detection on the updated original image to determine the first image region corresponding to each image grid. Then, the first image region is expanded to obtain the second image region corresponding to the image grid.

[0145] Next, a comic irregular quadrilateral regression operation can be performed, that is, using a positional regression neural network, based on the region mask information of each first image region, to detect the corner positions of the corresponding image grid in the corresponding second image region. Then, based on the determined corner positions and the segmented dialog box, the image grid image can be obtained. At the same time, the arrangement order of each image grid in a preset direction can be determined according to the reading order of the image grids in the original image. This step can be performed at any time after the image grids are detected, and the specific timing of this step is not limited in this embodiment.

[0146] Finally, based on each image grid, the target image corresponding to the original image can be generated and output.

[0147] about Figure 6 The specific implementation process of each step shown can be referred to the above embodiments, and will not be repeated here.

[0148] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0149] Based on the same inventive concept, this disclosure also provides an image conversion device corresponding to the image conversion method. Since the principle of the device in this disclosure for solving the problem is similar to that of the image conversion method described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0150] like Figure 7 The diagram shown is a schematic representation of an image conversion apparatus provided in an embodiment of this disclosure, comprising:

[0151] The acquisition module 701 is used to acquire the original image to be converted;

[0152] The first detection module 702 is used to detect a first image region corresponding to an image grid from the original image; the original image includes multiple image grids;

[0153] Adjustment module 703 is used to expand the region of the first image region to obtain the second image region corresponding to the image grid;

[0154] The second detection module 704 is used to obtain an image grid image based on the first image region and the second image region;

[0155] The generation module 705 is used to generate a target image corresponding to the original image based on the image grid image.

[0156] In one possible implementation, the first detection module 702, when detecting the first image region corresponding to the image grid from the original image, is configured to:

[0157] Identify the dialog box in the original image, segment the dialog box out of the original image, and obtain an updated original image;

[0158] From the updated original image, the first image region corresponding to the image grid is detected.

[0159] In one possible implementation, the second detection module 704, when obtaining the image grid image based on the first image region and the second image region, is used to:

[0160] Based on the region mask information of the first image region, image grid detection is performed in the second image region to determine the corner positions of the image grid image;

[0161] Based on the determined corner positions and the segmented dialog box, the image grid is obtained.

[0162] In one possible implementation, the second detection module 704, when obtaining the image grid image based on the determined corner point positions and the segmented dialog box, is used to:

[0163] Based on the determined positions of each corner point, an image grid image that does not contain the dialog box is obtained;

[0164] Based on the image grid corresponding to each dialog box, each dialog box is pasted back to the corresponding image grid image that does not contain the dialog box, thus obtaining an image grid image carrying the dialog box.

[0165] In one possible implementation, the first detection module 702, when identifying the dialog box in the original image and segmenting the dialog box from the original image to obtain an updated original image, is used to:

[0166] Determine the mask information of the dialog box in the original image;

[0167] Based on the mask information of the dialog box, the dialog box located in each layer is deleted from the original image to obtain the image to be repaired; wherein, in the image to be repaired, the pixel value of the pixel to be repaired at the position corresponding to the dialog box is a preset value;

[0168] Based on the pixel values ​​of all pixels in the image to be repaired, excluding the pixel to be repaired, the pixel values ​​of the pixel to be repaired are adjusted to obtain the updated original image.

[0169] In one possible implementation, the generation module 705, when generating the target image corresponding to the original image based on the image grid image, is configured to:

[0170] Based on the reading order of the image grids in the original image, determine the arrangement order of each image grid in the original image in a preset direction;

[0171] According to the arrangement order, the image grid images corresponding to each of the image grids are arranged sequentially in the preset direction to generate the target image.

[0172] In one possible implementation, the generation module 705, when generating the target image by arranging the image grid images corresponding to each image grid in the preset direction according to the arrangement order, is used to:

[0173] The image grids corresponding to each of the aforementioned image grids are adjusted to have the same width or the same length; wherein, the aspect ratio of the image grids remains unchanged after adjustment compared to before adjustment;

[0174] The target image is generated by arranging the image grids of the same width or length sequentially in the preset direction according to the arrangement order.

[0175] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0176] Based on the same technical concept, embodiments of this application also provide a computer device. (Refer to...) Figure 8 The diagram shown is a structural schematic of a computer device provided in an embodiment of this application, comprising:

[0177] The system includes a processor 81, a memory 82, and a bus 83. The memory 82 stores machine-readable instructions executable by the processor 81. The processor 81 executes these machine-readable instructions, performing the following steps: S101: Acquire the original image to be converted; S102: Detect a first image region corresponding to an image grid from the original image; the original image includes multiple image grids; S103: Perform region expansion processing on the first image region to obtain a second image region corresponding to the image grid; S104: Obtain an image grid image based on the first and second image regions; and S105: Generate a target image corresponding to the original image based on the image grid image.

[0178] The aforementioned memory 82 includes a main memory 821 and an external memory 822. The main memory 821, also known as internal memory, is used to temporarily store the computational data in the processor 81, as well as the data exchanged with external memory such as a hard disk. The processor 81 exchanges data with the external memory 82 through the main memory 821. When the computer device is running, the processor 81 and the memory 82 communicate through the bus 83, so that the processor 81 executes the execution instructions mentioned in the above method embodiments.

[0179] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the image conversion method described in the above-described method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0180] The computer program product of the image conversion method provided in this disclosure includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the steps of the image conversion method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0181] The computer program product can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0182] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0183] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0184] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0185] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0186] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the user through pop-up information or by asking the user to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0187] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. An image conversion method, characterized in that, include: Obtain the original image to be converted; The first image region corresponding to the image grid is detected from the original image; the original image includes multiple image grids; The first image region is expanded to obtain the second image region corresponding to the image grid. Based on the first image region and the second image region, an image grid image is obtained; as well as Based on the image grid, generate the target image corresponding to the original image; The step of detecting the first image region corresponding to the image grid from the original image includes: Perform image grid recognition on the original image to determine each image grid included in the original image; and For each image cell, the corresponding image region is identified from the original image, and the corresponding image region is used as the first image region corresponding to each image cell.

2. The method according to claim 1, characterized in that, The step of detecting the first image region corresponding to the image grid from the original image includes: Identify the dialog box in the original image, segment the dialog box out of the original image, and obtain an updated original image; From the updated original image, the first image region corresponding to the image grid is detected.

3. The method according to claim 2, characterized in that, Based on the first image region and the second image region, an image grid image is obtained, including: Based on the region mask information of the first image region, image grid detection is performed in the second image region to determine the corner positions of the image grid image; Based on the determined corner positions and the segmented dialog box, the image grid is obtained.

4. The method according to claim 3, characterized in that, Based on the determined corner positions and the segmented dialog box, the image grid is obtained, including: Based on the determined positions of each corner point, an image grid image that does not contain the dialog box is obtained; Based on the image grid corresponding to each dialog box, each dialog box is pasted back to the corresponding image grid image that does not contain the dialog box, thus obtaining an image grid image carrying the dialog box.

5. The method according to claim 2, characterized in that, The process of identifying the dialog box in the original image, segmenting the dialog box from the original image, and obtaining an updated original image includes: Determine the mask information of the dialog box in the original image; Based on the mask information of the dialog box, the dialog box located in each layer is deleted from the original image to obtain the image to be repaired; wherein, in the image to be repaired, the pixel value of the pixel to be repaired at the position corresponding to the dialog box is a preset value; Based on the pixel values ​​of all pixels in the image to be repaired, excluding the pixel to be repaired, the pixel values ​​of the pixel to be repaired are adjusted to obtain the updated original image.

6. The method according to claim 1, characterized in that, The step of generating a target image corresponding to the original image based on the image grid image includes: Based on the reading order of the image grids in the original image, determine the arrangement order of each image grid in the original image in a preset direction; According to the arrangement order, the image grid images corresponding to each of the image grids are arranged sequentially in the preset direction to generate the target image.

7. The method according to claim 6, characterized in that, According to the stated arrangement order, the image grid images corresponding to each of the image grids are arranged sequentially in the preset direction to generate the target image, including: The image grids corresponding to each of the aforementioned image grids are adjusted to have the same width or the same length; wherein, the aspect ratio of the image grids remains unchanged after adjustment compared to before adjustment; The target image is generated by arranging the image grids of the same width or length sequentially in the preset direction according to the arrangement order.

8. An image conversion device, characterized in that, include: The acquisition module is used to acquire the original image to be converted; The first detection module is used to detect a first image region corresponding to an image grid from the original image; the original image includes multiple image grids. An adjustment module is used to expand the region of the first image region to obtain the second image region corresponding to the image grid. The second detection module is used to obtain an image grid image based on the first image region and the second image region; as well as A generation module is used to generate a target image corresponding to the original image based on the image grid image; The first detection module is further configured as follows: Perform image grid recognition on the original image to determine each image grid included in the original image; and For each image cell, the corresponding image region is identified from the original image, and the corresponding image region is used as the first image region corresponding to each image cell.

9. A computer device, characterized in that, include: The processor and the memory, wherein the memory stores machine-readable instructions executable by the processor, the processor is configured to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the processor performs the steps of the image conversion method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer device, performs the steps of the image conversion method as described in any one of claims 1 to 7.

Images