Image processing method, image processing device, electronic device, and storage medium

Abstract

An image processing method, an image processing device, an electronic device, and a non-transitory computer-readable storage medium. The image processing method includes: acquiring an original image; processing the original image to obtain a preprocessed image, wherein the preprocessed image includes at least two first lines, and the at least two first lines are sequentially arranged in parallel along the same direction; processing the preprocessed image using a distortion processing model to obtain an intermediate image, wherein the intermediate image includes at least two second lines, and the at least two second lines are sequentially arranged in parallel along the same direction, and the at least two second lines have a one-to-one correspondence with the at least two first lines; and remapping the original image based on the mapping relationship between the preprocessed image and the intermediate image to obtain an output image.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to an image processing method, an image processing apparatus, an electronic device, and a non-transitory computer-readable storage medium. Background Art

[0002] With the development of digital technology, objects can be scanned or photographed to convert them into electronic images, which can be easily stored and transmitted over the internet. Furthermore, electronic images can be identified using image recognition technology to obtain the information contained therein. However, in the process of scanning or photographing objects to obtain electronic images, it is inevitable that the content in the resulting electronic images may be tilted, distorted, or deformed. Such tilting, distortion, or deformation can adversely affect the analysis and processing of the electronic images, for example, causing inaccurate recognition results, and also impacting the user's browsing experience. Summary of the Invention

[0003] At least one embodiment of the present disclosure provides an image processing method, comprising: acquiring an original image; processing the original image to obtain a preprocessed image, wherein the preprocessed image includes at least two first lines, and the at least two first lines are arranged in sequence and side by side along the same direction; processing the preprocessed image through a distortion processing model to obtain an intermediate image, wherein the intermediate image includes at least two second lines, and the at least two second lines are arranged in sequence and side by side along the same direction, and the at least two second lines correspond one-to-one to the at least two first lines; based on the mapping relationship between the preprocessed image and the intermediate image, remapping the original image to obtain an output image.

[0004] For example, in the image processing method provided by at least one embodiment of the present disclosure, the mapping relationship between the preprocessed image and the intermediate image includes a mapping relationship between the at least two first lines and the at least two second lines and a mapping relationship between the area between the at least two first lines in the preprocessed image and the area between the at least two second lines in the intermediate image.

[0005] For example, in the image processing method provided by at least one embodiment of the present disclosure, the original image is remapped based on the mapping relationship between the preprocessed image and the intermediate image to obtain an output image, including: based on the mapping relationship between the preprocessed image and the intermediate image, determining preprocessing mapping information corresponding to the preprocessed image by an interpolation method, wherein the preprocessing mapping information is used to indicate mapping parameters of at least part of the pixels in the preprocessed image; based on the preprocessing mapping information, determining mapping information corresponding to the area corresponding to the original image in the preprocessed image; scaling the mapping information of the area corresponding to the original image to determine the mapping information corresponding to the original image; and remapping the original image based on the mapping information corresponding to the original image to obtain the output image.

[0006] For example, in the image processing method provided by at least one embodiment of the present disclosure, at least part of the pixels in the preprocessed image include pixels in the area between the at least two first lines in the preprocessed image and pixels on the at least two first lines.

[0007] For example, in the image processing method provided by at least one embodiment of the present disclosure, the original image is processed to obtain a preprocessed image, including: binarizing the original image to obtain an input image; scaling the input image to obtain a scaled image; filling the scaled image to obtain a filled image; and dividing the filled image into regions to obtain the preprocessed image.

[0008] For example, in the image processing method provided by at least one embodiment of the present disclosure, the scaled image includes a first scaled image edge and a second scaled image edge opposite to each other, the pre-processed image includes a first pre-processed image edge and a second pre-processed image edge opposite to each other, the first pre-processed image edge corresponds to the first scaled image edge, and the second pre-processed image edge corresponds to the second scaled image edge, the at least two first lines are arranged between the first pre-processed image edge and the second pre-processed image edge along a direction from the first pre-processed image edge to the second pre-processed image edge, and the scaled image is filled to obtain the filled image, including: filling a first filling area on a side of the first scaled image edge away from the second scaled image edge and filling a second filling area on a side of the second scaled image edge away from the first scaled image edge, to obtain the filled image, wherein the two opposite sides of the first filling area are the first scaled image edge and the first pre-processed image edge, and the two opposite sides of the second filling area are the second scaled image edge and the second pre-processed image edge.

[0009] For example, in the image processing method provided by at least one embodiment of the present disclosure, the size of the first filling area is the same as the size of the second filling area.

[0010] For example, in the image processing method provided in at least one embodiment of the present disclosure, the original image is processed to obtain a preprocessed image, including: binarizing the original image to obtain an input image; filling the input image to obtain a filled image; scaling the filled image to obtain a scaled image; and dividing the scaled image into regions to obtain the preprocessed image.

[0011] For example, in the image processing method provided by at least one embodiment of the present disclosure, the at least two first lines are at least two equally dividing lines that equally divide the pre-processed image along the same direction.

[0012] For example, in the image processing method provided by at least one embodiment of the present disclosure, the distortion processing model is a neural network-based model.

[0013] For example, in the image processing method provided by at least one embodiment of the present disclosure, the image content in the original image is distorted.

[0014] For example, the image processing method provided by at least one embodiment of the present disclosure also includes: training the distortion processing model, wherein training the distortion processing model includes: generating a training image, wherein the training image includes at least two training lines, and the at least two training lines are arranged in sequence and side by side along the same direction; based on the training image, generating a target image corresponding to the training image, wherein the target image includes at least two target training lines, and the at least two target training lines are arranged in sequence and side by side along the same direction, and the at least two target training lines correspond one-to-one to the at least two training lines; based on the training image and the target image, training the distortion processing model to be trained to obtain the trained distortion processing model.

[0015] For example, in the image processing method provided by at least one embodiment of the present disclosure, the distortion processing model to be trained is trained based on the training image and the target image to obtain the trained distortion processing model, including: processing the training image through the distortion processing model to be trained to obtain an output training image, wherein the output training image includes at least two output lines, and the at least two output lines are arranged in parallel in sequence along the same direction, and the at least two output lines correspond one-to-one to the at least two training lines; based on the output training image and the target image, the parameters of the distortion processing model to be trained are adjusted; when the loss function corresponding to the distortion processing model to be trained meets a predetermined condition, the trained distortion processing model is obtained, and when the loss function corresponding to the distortion processing model to be trained does not meet the predetermined condition, the training image and the target image are continued to be input to repeat the above training process.

[0016] For example, in the image processing method provided by at least one embodiment of the present disclosure, generating the training image includes: generating an input training image; scaling the input training image to obtain a scaled input training image; filling the scaled input training image to obtain a filled input training image; warping the filled input training image to obtain a warped input training image; and dividing the warped input training image into regions to obtain the training image including the at least two training lines.

[0017] For example, in the image processing method provided in at least one embodiment of the present disclosure, a target image corresponding to the training image is generated based on the training image, including: performing reverse warping on the training image based on the warping parameters corresponding to the warping to obtain the target image.

[0018] For example, in the image processing method provided in at least one embodiment of the present disclosure, generating an input training image includes: acquiring an original training image; and performing binarization processing on the original training image to obtain the input training image.

[0019] For example, in the image processing method provided by at least one embodiment of the present disclosure, the at least two training lines are at least two equally dividing lines that equally divide the training image along the same direction.

[0020] At least one embodiment of the present disclosure further provides an image processing device, comprising: an image acquisition module, configured to acquire an original image; a first processing module, configured to process the original image to obtain a preprocessed image, wherein the preprocessed image includes at least two first lines, and the at least two first lines are arranged in sequence and side by side along the same direction; a second processing module, configured to process the preprocessed image through a distortion processing model to obtain an intermediate image, wherein the intermediate image includes at least two second lines, and the at least two second lines are arranged in sequence and side by side along the same direction, and the at least two second lines correspond one-to-one to the at least two first lines; a mapping module, configured to remap the original image based on the mapping relationship between the preprocessed image and the intermediate image to obtain an output image.

[0021] At least one embodiment of the present disclosure further provides an electronic device, comprising: a memory, which non-transitorily stores computer-executable instructions; and a processor, configured to execute the computer-executable instructions, wherein the computer-executable instructions, when executed by the processor, implement the image processing method according to any embodiment of the present disclosure.

[0022] At least one embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the image processing method according to any embodiment of the present disclosure can be implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, rather than limiting the present disclosure.

[0024] Figure 1 A schematic flowchart of an image processing method provided in at least one embodiment of the present disclosure;

[0025] Figure 2 A schematic diagram of an original image provided for at least one embodiment of the present disclosure;

[0026] Figure 3 A schematic diagram of a pre-processed image provided by at least one embodiment of the present disclosure;

[0027] Figure 4A A schematic diagram of a scaled image provided by at least one embodiment of the present disclosure;

[0028] Figure 4B A schematic diagram of a filled image provided by at least one embodiment of the present disclosure;

[0029] Figure 5 A schematic diagram of an intermediate image provided for at least one embodiment of the present disclosure;

[0030] Figure 6 A schematic diagram of an output image provided for at least one embodiment of the present disclosure;

[0031] Figure 7 A flowchart of a model training method provided for at least one embodiment of the present disclosure;

[0032] Figure 8A A schematic diagram of an original training image provided for at least one embodiment of the present disclosure;

[0033] Figure 8B A schematic diagram of a padded training image provided by at least one embodiment of the present disclosure;

[0034] Figure 8C A schematic diagram of a distorted training image provided by at least one embodiment of the present disclosure;

[0035] Figure 8D A schematic diagram of a training image provided for at least one embodiment of the present disclosure;

[0036] Figure 8E A schematic diagram of a target image provided for at least one embodiment of the present disclosure;

[0037] Figure 9 A schematic block diagram of an image processing device provided in at least one embodiment of the present disclosure;

[0038] Figure 10 A schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure;

[0039] Figure 11 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure;

[0040] Figure 12 A schematic diagram of a hardware environment provided for at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0042] Unless otherwise defined, the technical or scientific terms used in this disclosure should have the usual meanings understood by persons of ordinary skill in the field to which this disclosure belongs. The words "first", "second" and similar terms used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Words such as "include" or "comprise" mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0043] In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits detailed descriptions of some known functions and components.

[0044] Currently, neural network models can be used to identify electronic images and obtain the information contained in them. These images can be photographed or scanned by the user. During the acquisition process, the content in the electronic image may inevitably be distorted or deformed due to factors such as the shooting angle, resulting in inaccurate results from the neural network model.

[0045] At least one embodiment of the present disclosure provides an image processing method. The image processing method includes: acquiring an original image; processing the original image to obtain a preprocessed image, wherein the preprocessed image includes at least two first lines, and the at least two first lines are sequentially arranged in parallel along the same direction; processing the preprocessed image using a distortion processing model to obtain an intermediate image, wherein the intermediate image includes at least two second lines, and the at least two second lines are sequentially arranged in parallel along the same direction, and the at least two second lines have a one-to-one correspondence with the at least two first lines; and remapping the original image based on a mapping relationship between the preprocessed image and the intermediate image to obtain an output image.

[0046] In the image processing method provided in the embodiment of the present disclosure, the preprocessed image is first processed using a distortion processing model, and then the original image is remapped to obtain an output image based on the mapping relationship between the input and output of the distortion processing model, that is, the mapping relationship between the preprocessed image and the intermediate image, thereby achieving correction of the original image, effectively solving the problem of image distortion, improving the accuracy of the recognition results obtained based on the output image, improving the efficiency of image recognition, enhancing the readability of the image, and improving the user's experience of viewing the output image.

[0047] At least one embodiment of the present disclosure further provides an image processing apparatus, an electronic device, and a non-transitory computer-readable storage medium.

[0048] The image processing method provided in the embodiments of the present disclosure can be applied to the image processing device provided in the embodiments of the present disclosure, which can be configured on an electronic device. The electronic device can be a personal computer, a mobile terminal, etc. The mobile terminal can be a hardware device such as a mobile phone or a tablet computer with various operating systems. In other words, the image processing method can be executed by a personal computer, a mobile terminal, etc.

[0049] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings, but the present disclosure is not limited to these specific embodiments.

[0050] Figure 1 A schematic flowchart of an image processing method provided by at least one embodiment of the present disclosure is provided. Figure 2 A schematic diagram of an original image provided for at least one embodiment of the present disclosure.

[0051] like Figure 1 As shown, the image processing method provided by the embodiment of the present disclosure includes the following steps S10 to S13.

[0052] First, in step S10, an original image is acquired.

[0053] For example, the original image is an image obtained by photographing or scanning an object, and the object includes at least one of various characters, various symbols, and various graphics. The characters may include Chinese (for example, Chinese characters or pinyin), English, Japanese, French, Korean, Latin, numbers, etc. The symbols may include mathematical symbols and punctuation marks, etc. The mathematical symbols include plus signs, minus signs, greater than signs, less than signs, percent signs, etc. The punctuation marks may include periods, commas, greetings, etc. The graphics may include straight lines, curves, circles, rectangles, heart shapes, various pictures, etc., such as Figure 2 As shown, the original image 100 may include Chinese characters, numbers, graphics of houses (eg, Xiaohong's house, school, etc.), graphics of people, and the like.

[0054] For example, the original image can be any type of image, and the object can be, for example, a business card, a test paper, a workbook, a contract, an invoice, etc. Thus, the original image can be an image of a shopping list, an image of a restaurant receipt, an image of a test paper, an image of a workbook, an image of a contract, etc. For example, characters, symbols, and graphics can be handwritten, printed, or machine-generated.

[0055] For example, in some embodiments, the image content in the original image is distorted, that is, the objects in the original image are deformed, and the objects in the original image are inconsistent with their actual shapes. For example, characters in the same row of objects are tilted, distorted, or deformed. For example, distortion may include one or more of translation, rotation, scaling, affine transformation, perspective transformation, and cylindrical transformation. For example, Figure 2 As shown, in some embodiments, the original image 100 may be an image obtained by photographing a page of an exercise book (e.g., a math exercise book, etc.), and the text in the original image 100 is distorted. For example, in the page of the exercise book, the lines connecting the centers of the characters in “Whole tens plus single digits and corresponding subtractions” are located on the same straight line. However, in the original image 100, “Whole tens plus single digits and corresponding subtractions” are distorted, and the lines connecting the centers of the characters in “Whole tens plus single digits and corresponding subtractions” are not located on the same straight line, but are located on a curve (an irregular or regular curve).

[0056] For example, the shape of the original image can be any suitable shape such as a rectangle, etc. The shape and size of the original image can be set by the user according to actual conditions, and the embodiments of the present disclosure do not limit this.

[0057] For example, the original image may be an image captured by an image acquisition device (e.g., a digital camera or a mobile phone camera). The original image may be a grayscale image, a black and white image, or a color image. It should be noted that the original image refers to a visual representation of an object, such as a picture of the object. For another example, the original image may be obtained by scanning or other methods. For example, the original image may be an image directly captured by the image acquisition device, or an image obtained by preprocessing the captured image. For example, to prevent the impact of data quality or data imbalance of the image directly captured by the image acquisition device on subsequent processing, the image processing method may further include preprocessing the image directly captured by the image acquisition device before processing the original image. Preprocessing may, for example, include cropping, gamma correction, or noise reduction filtering of the image directly captured by the image acquisition device. Preprocessing can eliminate irrelevant information or noise in the original image, thereby facilitating subsequent processing of the original image.

[0058] Figure 3 A schematic diagram of a pre-processed image provided by at least one embodiment of the present disclosure. For example, Figure 3 The preprocessed images shown are for Figure 2 The image obtained by processing the original image shown.

[0059] like Figure 1As shown, in step S11: the original image is processed to obtain a pre-processed image.

[0060] For example, in some embodiments, step S11 includes: binarizing the original image to obtain an input image; scaling the input image to obtain a scaled image; filling the scaled image to obtain a filled image; and dividing the filled image into regions to obtain a preprocessed image.

[0061] For example, in some other embodiments, step S11 includes: gray-scaling the original image to obtain an input image; scaling the input image to obtain a scaled image; filling the scaled image to obtain a filled image; and dividing the filled image into regions to obtain a preprocessed image.

[0062] For example, binarization or grayscale processing of the original image can reduce the amount of data required for subsequent processing and increase processing speed. Binarization or grayscale processing is used to remove interfering pixels in the original image, retaining only the content that needs to be processed, such as characters, graphics, or images.

[0063] For example, the binarization method may include a threshold method, a bimodal method, a P parameter method, an OTSU method, a maximum entropy method, an iterative method, and the like.

[0064] For example, grayscale processing methods include component method, maximum method, average method and weighted average method.

[0065] It should be noted that the order of binarization / grayscale processing, scaling processing and filling processing can be adjusted arbitrarily and is not limited to the above description. For example, scaling processing can be performed first, then filling processing, and finally binarization / grayscale processing.

[0066] For example, the size of the input image and the size of the original image can be the same. The size of the padded image is larger than the size of the scaled image, and the size of the padded image is equal to the size of the pre-processed image. For example, when the scaling process is a reduction process, the size of the scaled image is smaller than the size of the input image, and when the scaling process is an expansion process, the size of the scaled image is larger than the size of the input image.

[0067] In the embodiments of the present disclosure, binarization / grayscale processing can reduce the amount of data processing, thereby improving the processing speed of image processing; scaling processing can unify the image size to facilitate model processing; and filling processing can prevent the content corresponding to the object in the pre-processed image after the distortion operation from exceeding the picture area of ​​the pre-processed image, thereby avoiding the loss of image content and ensuring the integrity of the image content.

[0068] It should be noted that, in the image processing method provided in the embodiment of the present disclosure, binarization / grayscale processing may not be performed, thereby reducing the processing flow.

[0069] Figure 3 The pre-processed image shown is the image after grayscale processing. Figure 3 As shown, the pre-processed image 200 includes a first pre-processed image side PB1, a second pre-processed image side PB2, a third pre-processed image side PB3, and a fourth pre-processed image side PB4. The first pre-processed image side PB1 and the second pre-processed image side PB2 are opposite sides, and the third pre-processed image side PB3 and the fourth pre-processed image side PB4 are opposite sides. For example, the pre-processed image 200 may be rectangular. In this case, the first pre-processed image side PB1 and the second pre-processed image side PB2 are parallel to each other and parallel to the X1 direction; the third pre-processed image side PB3 and the fourth pre-processed image side PB4 are parallel to each other and parallel to the Y1 direction; and the first pre-processed image side PB1 and the third pre-processed image side PB3 are perpendicular to each other. For example, the X1 direction is the width of the pre-processed image 200, and the Y1 direction is the height of the pre-processed image 200.

[0070] For example, the preprocessed image includes at least two first lines, the at least two first lines are arranged in sequence and side by side along the same direction, and the at least two first lines are arranged between the first preprocessed image edge and the second preprocessed image edge of the preprocessed image along the direction from the first preprocessed image edge to the second preprocessed image edge.

[0071] For example, in some examples, such as Figure 3 As shown, the pre-processed image 200 may include at least two first lines L1, which are arranged in parallel along the same direction (e.g., the Y1 direction, i.e., the height direction of the pre-processed image 200). The at least two first lines L1 are parallel to each other and to the X1 direction. The at least two first lines L1 are located between the first pre-processed image side PB1 and the second pre-processed image side PB2.

[0072] It should be noted that, in the embodiment of the present disclosure, the arrangement of the at least two first lines L1 is not limited to Figure 3 As shown, in some embodiments, at least two first lines L1 may also be arranged along the X1 direction. In this case, at least two first lines L1 are parallel to the Y1 direction and located between the third preprocessed image side PB3 and the fourth preprocessed image side PB4.

[0073] For example, in some embodiments, Figure 3As shown, the at least two first lines L1 are at least two equally dividing lines that equally divide the pre-processed image 200 along the same direction (eg, Y1 direction), that is, the distance h3 between any two adjacent first lines L1 is a fixed value.

[0074] For example, the number of the at least two first lines L1 can be set according to actual conditions. For example, in some examples, Figure 3 As shown, the number of at least two first lines L1 can be 23. In this case, the pre-processed image 200 can be equally divided into 24 parts along the third pre-processed image side PB3, thereby obtaining 23 first lines L1. For example, if the length of the third pre-processed image side PB3 is 768 pixels, the distance h3 between any two adjacent first lines L1 is 32 pixels.

[0075] It should be noted that the number of first lines L1 can be less or more. For example, the number of first lines L1 can be within the numerical range of 12 to 48, for example, 12 or 48. The more first lines L1 there are, the more accurate the final output image will be, but the amount of data processing will be greater.

[0076] exist Figure 3 In order to clearly illustrate the first line L1 , the first line L1 is represented by a thicker line. The width of the first line L1 can be set according to actual conditions, for example, it can be 1 to 2 pixels.

[0077] Figure 4A A schematic diagram of a scaled image provided by at least one embodiment of the present disclosure; Figure 4B A schematic diagram of a filled image provided by at least one embodiment of the present disclosure. Figure 4B The filled image shown is for Figure 4A The scaled image shown is obtained by filling.

[0078] For example, in some embodiments, Figure 4A As shown, the scaled image 300 includes a first scaled image edge CB1 and a second scaled image edge CB2 that are opposite to each other. The first pre-processed image edge corresponds to the first scaled image edge CB1, and the second pre-processed image edge corresponds to the second scaled image edge CB2. That is, in the pre-processed image, the first pre-processed image edge and the first scaled image edge CB1 are located on the same side, for example Figure 4A On the upper side shown, the second pre-processed image edge and the second scaled image edge CB2 are located on the same side, for example Figure 4A Underside shown.

[0079] In step S11, the scaled image is filled to obtain a filled image, including: filling a first filling area on a side of the first scaled image away from the second scaled image side and filling a second filling area on a side of the second scaled image away from the first scaled image side, to obtain a filled image.

[0080] For example, the padded image includes the scaled image, a first padded region, and a second padded region. The pre-processed image includes the padded image and at least one first line. In the pre-processed image, two opposing sides of the first padded region are the first scaled image side and the first pre-processed image side, and two opposing sides of the second padded region are the second scaled image side and the second pre-processed image side.

[0081] For example, Figure 4B As shown, in some embodiments, the first padding region 310 is padded (e.g., stitched) to the side of the first scaled image edge CB1 of the scaled image 300 that is away from the second scaled image edge CB2, and the second padding region 320 is padded (e.g., stitched) to the side of the second scaled image edge CB2 of the scaled image 300 that is away from the first scaled image edge CB1. The padded image 2000 includes a complete area consisting of the scaled image 300, the first padding region 310, and the second padding region 320.

[0082] For example, Figure 4B As shown, the filled image 2000 includes a first filled image edge FB1 and a second filled image edge FB2 relative to each other. After the filled image 2000 is divided into regions to obtain a preprocessed image, the first filled image edge FB1 is the first preprocessed image edge, and the second filled image edge FB2 is the second preprocessed image edge.

[0083] For example, Figure 4A As shown, the scaled image 300 also includes a third scaled image side CB3 and a fourth scaled image side CB4 that are opposite to each other. For example, the scaled image 300 can be rectangular. In this case, the first scaled image side CB1 and the second scaled image side CB2 are parallel to each other and to the X2 direction; the third scaled image side CB3 and the fourth scaled image side CB4 are parallel to each other and to the Y2 direction; and the first scaled image side CB1 and the third scaled image side CB3 are perpendicular to each other. For example, the X2 direction is the width of the scaled image 300, and the Y2 direction is the height of the scaled image 300.

[0084] For example, Figure 4BAs shown, the first filling area 310 can be a rectangle, and the second filling area 320 can also be a rectangle. The length of the side of the first filling area 310 parallel to the Y2 direction can be h1, and the length of the side of the second filling area 320 parallel to the Y2 direction can be h2. Figure 4A and Figure 4B As shown, the length of the first scaled image side CB1 is w1, the length of the side of the first padding area 310 parallel to the X2 direction is w1, and the length of the side of the second padding area 320 parallel to the X2 direction is w1. For example, the size of the first padding area 310 and the size of the second padding area 320 are the same, in which case h1 equals h2.

[0085] For example, in some embodiments, h1 may be 64 pixels. For example, if the size of the scaled image 300 may be 576 (pixels)*640 (pixels), the size of the padded image 2000 may be 576 (pixels)*768 (pixels).

[0086] For example, the pixel value of each pixel in the first filling area 310 and the second filling area 320 can be set according to actual conditions, for example, both are 0, and the present disclosure does not limit this.

[0087] In some embodiments of the present disclosure, scaling may be performed first, followed by filling. However, the present disclosure is not limited thereto. In other embodiments, filling may be performed first, followed by scaling. Specific filling parameters corresponding to the filling process (i.e., the sizes of the first filling area and the second filling area, etc.) may be set according to actual conditions and are not limited by the present disclosure.

[0088] For example, the filled image may be divided into regions along the height direction of the filled image using at least two first lines to obtain a pre-processed image.

[0089] For example, in some other embodiments, step S11 includes: binarizing the original image to obtain an input image; filling the input image to obtain a filled image; scaling the filled image to obtain a scaled image; and dividing the scaled image into regions to obtain a preprocessed image; or, step S11 includes: grayscale processing the original image to obtain an input image; filling the input image to obtain a filled image; scaling the filled image to obtain a scaled image; and dividing the scaled image into regions to obtain a preprocessed image.

[0090] It should be noted that the filling process can be determined according to the distortion direction of the image content in the original image. For example, if the image content in the original image is distorted in the length direction, then in the filling process, a filling area is filled on each of the two sides in the length direction of the image; if the image content in the original image is distorted in the width direction, then in the filling process, a filling area is filled on each of the two sides in the width direction of the image; if the image content in the original image is distorted in both the length and width directions, then in the filling process, a filling area is filled on each of the two sides in the length direction of the image, and a filling area is also filled on each of the two sides in the width direction of the image.

[0091] Figure 5 A schematic diagram of an intermediate image provided for at least one embodiment of the present disclosure. Figure 5 The middle image 400 is a diagram of the model after the warping process. Figure 3 The image obtained after processing the preprocessed image shown.

[0092] like Figure 1 As shown, in step S12, the pre-processed image is processed by the distortion processing model to obtain an intermediate image.

[0093] For example, the distortion processing model can be implemented using machine learning technology (e.g., deep learning technology). For example, in some embodiments, the distortion processing model can be a neural network-based model. The distortion processing model can use a pix2pixHD (pixel to pixel HD) model. The pix2pixHD model uses a multi-level generator (coarse-to-fine generator) and a multi-scale discriminator (multi-scale discriminator) to distort the preprocessed image to generate a distorted intermediate image. The generator of the pix2pixHD model includes a global generator network and a local enhancer network. The global generator network adopts a U-Net structure. The features output by the global generator network are fused with the features extracted by the local enhancer network and serve as input information for the local enhancer network. The local enhancer network outputs the distorted intermediate image. For example, the distortion processing model can also use other models, such as a U-Net model, etc., which is not limited by the present disclosure. The training process for the distortion processing model is described below and will not be repeated here.

[0094] For example, the intermediate image includes at least two second lines, the at least two second lines are sequentially arranged in parallel along the same direction, and the at least two second lines correspond one-to-one to the at least two first lines. Figure 5 As shown, the intermediate image 400 includes at least two second lines L2, and the at least two second lines L2 are arranged in parallel in the same direction (for example, the Y3 direction, that is, the height direction of the intermediate image 400), and the extension direction of the at least two second lines L2 is the X3 direction. Figure 5 At least two second lines L2 are shown Figure 3 The at least two first lines L1 shown correspond to each other. The second line L2 is a line after the first line L1 is twisted, as shown in FIG. Figure 5 As shown, each second line L2 is a regular or irregular curve, and the shapes of the second lines L2 are different. It should be noted that one or more second lines L2 can also be straight lines, and the present disclosure does not specifically limit the shape and other properties of the second lines L2.

[0095] It should be noted that the X1, X2, and X3 directions are parallel to each other, and the Y1, Y2, and Y3 directions are also parallel to each other. In some embodiments, the X1, X2, and X3 directions are all width directions of the image, for example, the width direction of the image is parallel to the horizontal direction. The Y1, Y2, and Y3 directions are all height directions of the image, for example, the height direction of the image is parallel to the vertical direction.

[0096] Figure 6 A schematic diagram of an output image provided for at least some embodiments of the present disclosure. Figure 6 The output image shown is the image processing method provided by the embodiment of the present disclosure. Figure 2 The image obtained after processing the original image shown.

[0097] like Figure 1 As shown, in step S13, based on the mapping relationship between the preprocessed image and the intermediate image, the original image is remapped to obtain an output image.

[0098] For example, the mapping relationship between the preprocessed image and the intermediate image includes a mapping relationship between at least two first lines and at least two second lines and a mapping relationship between an area between at least two first lines in the preprocessed image and an area between at least two second lines in the intermediate image.

[0099] In addition, it should be noted that the mapping relationship between the area between the at least two first lines in the preprocessed image and the area between the at least two second lines in the intermediate image needs to be determined based on the mapping relationship between the at least two first lines and the at least two second lines.

[0100] For example, in some embodiments, step S13 may include: determining mapping information corresponding to the original image based on the mapping relationship between the preprocessed image and the intermediate image; and remapping the original image based on the mapping information corresponding to the original image to obtain an output image.

[0101] For example, in step S13, based on the mapping relationship between the preprocessed image and the intermediate image, the mapping information corresponding to the original image is determined, including: based on the mapping relationship between the preprocessed image and the intermediate image, determining the preprocessed mapping information corresponding to the preprocessed image through an interpolation method; based on the preprocessed mapping information, determining the mapping information corresponding to the area in the preprocessed image corresponding to the original image; and scaling the mapping information of the area corresponding to the original image to determine the mapping information corresponding to the original image.

[0102] For example, the pre-processing mapping information is used to indicate mapping parameters of at least some pixels in the pre-processing image. At least some pixels in the pre-processing image include pixels in a region between at least two first lines in the pre-processing image and pixels on at least two first lines. Figure 3 As shown, the preprocessed image 200 includes region A1 and region A2, which are not located between two first lines L1, and at least part of the pixels in the preprocessed image 200 include all pixels in the preprocessed image except region A1 and region A2.

[0103] It should be noted that the pre-processing mapping information may also indicate mapping parameters of all pixels in the pre-processing image, which is not limited in the present disclosure.

[0104] In the present disclosure, the original image is remapped according to the mapping relationship between the input and output of the distortion processing model (i.e., based on the mapping relationship between the preprocessed image and the intermediate image), thereby correcting the distorted original image to obtain an output image, effectively solving the problem of image distortion, improving the accuracy of the recognition results obtained based on the output image, improving the efficiency of image recognition, enhancing the readability of the image, and improving the user's experience of viewing the output image.

[0105] For example, the area between any two adjacent second lines in the intermediate image can correspond to the area between the corresponding two adjacent first lines in the preprocessed image, and each second line in the intermediate image can correspond to the corresponding first line in the preprocessed image, so that the preprocessing mapping information corresponding to the preprocessed image can be determined by an interpolation method based on the mapping relationship between the preprocessed image and the intermediate image. Figure 3 and Figure 5As shown, the area between any two adjacent first lines L1 (e.g., first line L11 and first line L12) in the pre-processed image 200 and the area between two second lines L2 (e.g., second line L21 and second line L22) in the intermediate image 400 corresponding to the two first lines L1, respectively, are mapped to each other. In other words, the area between the first line L11 and the first line L12 in the pre-processed image 200 needs to be mapped to the area between the second line L21 and the second line L22 in the intermediate image 400. The first line L1 in the pre-processed image 200 and the second line L2 corresponding to the first line L1 in the intermediate image 400 are also mapped to each other. For example, the first line L11 and the first line L12 in the pre-processed image 200 need to be mapped to the second line L21 and the second line L22 in the intermediate image 400.

[0106] For example, the interpolation method may include nearest neighbor interpolation, bilinear interpolation, bicubic spline interpolation, bicubic interpolation, Lanczos interpolation, etc. The present disclosure does not limit the interpolation method.

[0107] For example, the mapping information corresponding to the original image may include mapping parameters corresponding to all pixels in the original image, that is, the number of mapping parameters in the mapping information corresponding to the original image may be the same as the number of all pixels in the original image. For example, the mapping parameter corresponding to a pixel may represent the coordinate value of the position to which the pixel is mapped; or it may represent the offset between the coordinate value of the pixel and the coordinate value of the position to which the pixel is mapped.

[0108] It should be noted that the coordinate value of the pixel can be expressed as the coordinate value in the coordinate system corresponding to the original image, the coordinate origin of the coordinate system corresponding to the original image is a pixel point of the original image (for example, the pixel point corresponding to the center of the original image or the pixel point in the upper left corner of the original image), and the two coordinate axes of the coordinate system corresponding to the original image are the width and height of the original image. The coordinate value of the position to which the pixel is mapped can be expressed as the coordinate value in the coordinate system corresponding to the output image, the coordinate origin of the coordinate system corresponding to the output image is the pixel point in the output image corresponding to the coordinate origin of the coordinate system corresponding to the original image, and the two coordinate axes of the coordinate system corresponding to the output image are the width and height of the output image.

[0109] For example, based on the mapping relationship between the preprocessed image and the intermediate image as a reference, the mapping parameters corresponding to each pixel in the original image can be determined, thereby obtaining mapping information corresponding to the original image. Based on the mapping information corresponding to the original image, the position to which each pixel is mapped after correcting image distortion can be determined, thereby achieving mapping processing.

[0110] For example, in step S13, remapping the original image based on the mapping information corresponding to the original image to obtain the output image may include: calling the remapping function (i.e., remap function) in opencv to remap the original image based on the mapping information corresponding to the original image to obtain the output image. Figure 6 As shown, in the output image 500, the lines connecting the centers of the characters in "a whole ten plus a single-digit number and the corresponding subtraction" are located on the same straight line, so that the text is straightened, thereby effectively correcting the distortion of the original image, solving the problem of image distortion, improving the accuracy of the recognition results obtained based on the output image, improving the efficiency of image recognition, enhancing the readability of the image, and improving the user's experience of viewing the output image.

[0111] For example, in some embodiments of the present disclosure, the image processing method further includes: training a distortion processing model.

[0112] At least one embodiment of the present disclosure further provides a model training method for implementing the above-mentioned operation of training the distortion processing model. Figure 7 A flowchart of a model training method provided for at least one embodiment of the present disclosure.

[0113] In some embodiments, the model training method may include training a distortion processing model, e.g., Figure 7 As shown, training the distortion processing model includes the following steps S20 to S22.

[0114] Step S20: Generate a training image. For example, the training image includes at least two training lines, and the at least two training lines are sequentially arranged in parallel along the same direction.

[0115] Step S21: Based on the training image, a target image corresponding to the training image is generated. For example, the target image includes at least two target training lines, the at least two target training lines are arranged in parallel along the same direction, and the at least two target training lines correspond to the at least two training lines in a one-to-one manner.

[0116] Step S22: Based on the training image and the target image, the distortion processing model to be trained is trained to obtain a trained distortion processing model.

[0117] For example, in some embodiments, step S20 may include: generating an input training image; scaling the input training image to obtain a scaled input training image; filling the scaled input training image to obtain a filled input training image; warping the filled input training image to obtain a warped input training image; and dividing the warped input training image into regions to obtain a training image including at least two training lines.

[0118] For example, in some other embodiments, step S20 may include: generating an input training image; performing filling processing on the input training image to obtain a filled input training image; performing scaling processing on the filled input training image to obtain a scaled input training image; performing warping processing on the scaled input training image to obtain a warped input training image; and performing region division on the warped input training image to obtain a training image including at least two training lines.

[0119] It should be noted that, in step S20, the order of the filling process and the scaling process can be set according to actual conditions, and the present disclosure does not limit this. In the following description of the present disclosure, the scaling process is performed first and then the filling process is described as an example.

[0120] For example, in step S20, generating an input training image may include: obtaining an original training image; and performing binarization or grayscale processing on the original training image to obtain an input training image. Binarization or grayscale processing on the original training image can remove interference (noise) in the original training image and reduce the amount of data processing in subsequent training. It should be noted that binarization or grayscale processing is not a required step, and the original training image can also be directly filled, scaled, and region-partitioned to obtain the training image.

[0121] Figure 8A A schematic diagram of an original training image provided for at least one embodiment of the present disclosure.

[0122] For example, the original training image can be an image that has not been distorted, such as Figure 8A As shown, in the original training image 810 , all texts are not distorted.

[0123] Figure 8B A schematic diagram of a padded training image provided for at least one embodiment of the present disclosure. Figure 8B The padded training images shown can be used for Figure 8A The original training images shown are scaled and padded.

[0124] For example, the input training image can be scaled and padded to a fixed size. Unifying the size of the image can facilitate the image processing by the distortion processing model to be trained. Figure 8BAs shown, the input training image can be first scaled to obtain a scaled training image 830. The size of scaled training image 830 can be 576*640 (pixels). Scaled training image 830 includes an image side CTB1 and an image side CTB2 that are opposite to each other. A training padding region 831 is padded on the side of image side CTB1 of scaled training image 830 that is away from image side CTB2, and a training padding region 832 is padded on the side of image side CTB2 of scaled training image 830 that is away from image side CTB1, thereby obtaining a padded training image 820. Padded training image 820 can include an area consisting of training padding region 831, scaled training image 830, and training padding region 832. The padding process can prevent the content from exceeding the screen after the distortion operation. For example, the size of training padding region 831 and training padding region 832 can be the same, and the size of training padding region 831 can be 576*64 (pixels), so that the size of padded training image 820 can be 576*768 (pixels).

[0125] It should be noted that, for detailed descriptions of the filling process and the scaling process, reference may be made to the descriptions of the filling process and the scaling process in the above embodiment of the image processing method, and repeated descriptions will be omitted.

[0126] Figure 8C A schematic diagram of a distorted training image provided by at least one embodiment of the present disclosure is provided. Figure 8C The distorted training images shown can be used for Figure 8B The padded training image shown is the image after distortion.

[0127] For example, the method of distortion processing is not limited. In some embodiments, opencv can be used to implement distortion processing. For example, first, a set of offsets is randomly generated, and then the offsets are Gaussian filtered to make the offsets smooth and continuous. The offsets after Gaussian filtering are used to generate a distortion parameter matrix (for example, map), and the remap function in opencv is called to remap the filled image to implement distortion processing, thereby obtaining a distorted training image.

[0128] Figure 8D A schematic diagram of a training image provided for at least one embodiment of the present disclosure. Figure 8D The training image 850 shown can be Figure 8C The warped image 840 is shown as the processed image.

[0129] For example, in some embodiments, Figure 8DAs shown, the training image 850 may include at least two training lines TL, which are at least two equally dividing lines that equally divide the training image 850 along the same direction (eg, the height direction Y4 of the training image 850). Figure 8D As shown, the training lines TL may be parallel to each other and extend along the width direction X4 of the training image 850. For example, Figure 8C and Figure 8D As shown in FIG, the distorted image 840 can be divided into equal parts and a dividing line can be drawn to obtain a training image 850. For example, the distorted image 840 can be divided into equal parts along its height direction. The number of the at least two training lines TL1 can be set according to actual conditions, such as Figure 8D As shown, the number of at least two training lines TL1 can be 23. However, the number of at least two training lines TL1 can be less or more, for example, 12 to 48, etc. The more the number of at least two training lines TL1 is, the more accurate the distortion processing model obtained by training is, but the amount of data processing is greater.

[0130] Figure 8E A schematic diagram of a target image provided for at least one embodiment of the present disclosure. Figure 8E The target image 860 shown can be a Figure 8D The training image 850 shown is an image after reverse warping.

[0131] For example, in some embodiments, step S21 includes: performing a reverse warping process on the training image based on a warping parameter corresponding to the warping process to obtain a target image.

[0132] For example, the purpose of the reverse warping process is to restore the image content portion of the training image 850 except the training line TL1 to the state before the warping process (ie Figure 8B The state of the filled image 820 shown. Figure 8E As shown, the target image 860 includes at least two target training lines TL2, the at least two target training lines TL2 are sequentially arranged along the height direction of the target image 860, and the at least two target training lines TL2 extend along the width direction of the target image 860. Figure 8D The at least two training lines TL1 in the training image 850 shown are in one-to-one correspondence, and the at least two target training lines TL2 are lines obtained by performing a reverse warping process on the at least two training lines TL1.

[0133] For example, in some embodiments, step S21 may include: processing the training image through the distortion processing model to be trained to obtain an output training image; adjusting the parameters of the distortion processing model to be trained based on the output training image and the target image; when the loss function corresponding to the distortion processing model to be trained meets a predetermined condition, obtaining a trained distortion processing model; when the loss function corresponding to the distortion processing model to be trained does not meet the predetermined condition, continuing to input the training image and the target image to repeat the above training process.

[0134] For example, the output training image includes at least two output lines, the at least two output lines are arranged in parallel in the same direction, and the at least two output lines correspond one-to-one to the at least two training lines. The at least two output lines can be the lines after the distortion processing model to be trained processes the at least two training lines.

[0135] For example, in step S21, the distortion processing model to be trained processes the image content and the training lines in the training image as a whole to obtain an output training image.

[0136] For example, in step S21, adjusting the parameters of the distortion processing model to be trained based on the output training image and the target image may include: calculating the loss value of the distortion processing model to be trained by the loss function corresponding to the distortion processing model to be trained based on the output training image and the target image; and adjusting the parameters of the distortion processing model to be trained based on the loss value.

[0137] For example, in one example, the predetermined condition corresponds to minimizing a loss function corresponding to the distortion processing model to be trained when a certain number of training images are input. In another example, the predetermined condition is that the distortion processing model to be trained has been trained for a predetermined number of times or training cycles, which can be in the millions, as long as the number of training images used for training is sufficiently large.

[0138] It should be noted that, in the operation of repeatedly executing the training process, different training images and their corresponding target images can be used to train the distortion processing model to be trained; in addition, the above training process can also be executed multiple times using the same training image and its corresponding target image.

[0139] At least one embodiment of the present disclosure further provides an image processing device, Figure 9 A schematic block diagram of an image processing device provided in at least one embodiment of the present disclosure.

[0140] For example, Figure 9 As shown, in some embodiments, the image processing device 900 may include an image acquisition module 901 , a first processing module 902 , a second processing module 903 and a mapping module 904 .

[0141] The image acquisition module 901 is configured to acquire the original image. Figure 1 As shown in step S10, the specific description of the functions implemented by the image acquisition module 901 can refer to the embodiment of the above-mentioned image processing method. Figure 1 The description of step S10 shown is repeated and will not be repeated.

[0142] For example, the image acquisition module 901 may include a camera, such as a camera of a smartphone, a camera of a tablet computer, a camera of a personal computer, a lens of a digital camera, or even a webcam.

[0143] The first processing module 902 is configured to process the original image to obtain a pre-processed image. For example, the pre-processed image includes at least two first lines, and the at least two first lines are arranged in parallel in the same direction. The first processing module 902 is used to implement Figure 1 For the detailed description of the functions implemented by the first processing module 902, please refer to the embodiment of the above-mentioned image processing method. Figure 1 The description of step S11 shown is repeated and will not be repeated.

[0144] The second processing module 903 is configured to process the pre-processed image using a distortion processing model to obtain an intermediate image. For example, the intermediate image includes at least two second lines, the at least two second lines are sequentially arranged in parallel along the same direction, and the at least two second lines correspond to the at least two first lines in a one-to-one manner. The second processing module 903 is used to implement Figure 1 For the detailed description of the functions implemented by the second processing module 903, please refer to the embodiment of the above-mentioned image processing method. Figure 1 The description of step S12 shown is repeated and will not be repeated.

[0145] The mapping module 904 is configured to remap the original image based on the mapping relationship between the pre-processed image and the intermediate image to obtain an output image. Figure 1 As shown in step S13, the detailed description of the functions implemented by the mapping module 904 can refer to the embodiment of the above-mentioned image processing method. Figure 1 The description of step S13 shown is repeated and will not be repeated.

[0146] For example, data communication may be performed among the image acquisition module 901 , the first processing module 902 , the second processing module 903 and the mapping module 904 .

[0147] For example, in some embodiments, the image processing apparatus 900 may further include a model training module configured to train a distortion processing model.

[0148] For example, in some embodiments, the model training module may include an image generation submodule and a training submodule.

[0149] For example, the image generation submodule is configured to: generate a training image; and based on the training image, generate a target image corresponding to the training image. For example, the training image includes at least two training lines, and the at least two training lines are sequentially arranged in parallel along the same direction; the target image includes at least two target training lines, and the at least two target training lines are sequentially arranged in parallel along the same direction, and the at least two target training lines correspond one-to-one to the at least two training lines. The image generation submodule is used to implement Figure 7 The detailed description of the functions implemented by the image generation submodule can refer to the embodiment of the above-mentioned image processing method. Figure 7 The descriptions of step S20 and step S21 are shown, and the repeated parts are not repeated here.

[0150] For example, the training submodule is configured to train the distortion processing model to be trained based on the training image and the target image to obtain a trained distortion processing model. The training submodule is used to implement Figure 7 For the detailed description of the functions implemented by the training submodule, please refer to the embodiment of the above-mentioned image processing method. Figure 7 The description of step S22 shown is repeated and will not be repeated.

[0151] In some examples, the training submodule is configured to process a training image using the warping model to be trained to obtain an output training image; adjust the parameters of the warping model to be trained based on the output training image and the target image; and obtain a trained warping model when the loss function corresponding to the warping model to be trained meets a predetermined condition. For example, the image generation submodule is further configured to continue generating at least one training image and a target image corresponding to the at least one training image when the loss function corresponding to the warping model to be trained does not meet the predetermined condition. The at least one training image and its corresponding target image are used to repeatedly execute the above training process.

[0152] For example, the image acquisition module 901, the first processing module 902, the second processing module 903, the mapping module 904 and / or the model training module include codes and programs stored in the memory; the processor can execute the codes and programs to implement some or all of the functions of the image acquisition module 901, the first processing module 902, the second processing module 903, the mapping module 904 and / or the model training module as described above. For example, the image acquisition module 901, the first processing module 902, the second processing module 903, the mapping module 904 and / or the model training module can be dedicated hardware devices used to implement some or all of the functions of the image acquisition module 901, the first processing module 902, the second processing module 903, the mapping module 904 and / or the model training module as described above. For example, the image acquisition module 901, the first processing module 902, the second processing module 903, the mapping module 904 and / or the model training module can be a circuit board or a combination of multiple circuit boards used to implement the functions as described above. In an embodiment of the present application, the circuit board or the combination of multiple circuit boards may include: (1) one or more processors; (2) one or more non-temporary memories connected to the processors; and (3) firmware stored in the memories that is executable by the processors.

[0153] It should be noted that the image processing device can achieve technical effects similar to those of the aforementioned image processing method, which will not be described in detail here.

[0154] At least one embodiment of the present disclosure further provides an electronic device, Figure 10 A schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure.

[0155] For example, Figure 10 As shown, electronic device 1000 may include a processor 1001 and a memory 1002. Memory 1002 non-transiently stores computer-executable instructions; processor 1001 is configured to execute the computer-executable instructions. When the computer-executable instructions are executed by processor 1001, the image processing method described in any embodiment of the present disclosure may be implemented. The specific implementation and related explanations of each step of the image processing method can be found in the above-mentioned embodiments of the image processing method and are not further elaborated here.

[0156] For example, Figure 10 As shown, electronic device 1000 may further include a communication interface 1003 and a communication bus 1004. Processor 1001, memory 1002, and communication interface 1003 communicate with each other via communication bus 1004. Components such as processor 1001, memory 1002, and communication interface 1003 may also communicate with each other via a network connection. This disclosure does not limit the type and function of the network.

[0157] For example, other implementations of the image processing method implemented by the processor 1001 executing the program stored in the memory 1002 are the same as the implementations mentioned in the aforementioned image processing method embodiment section and will not be repeated here.

[0158] For example, the communication bus 1004 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industrial Standard Architecture (EISA) bus. The communication bus 1004 may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0159] For example, the communication interface 1003 is used to implement communication between the electronic device 1000 and other devices.

[0160] For example, the processor 1001 and the memory 1002 may be provided on a server side (or a cloud), or on a client side (eg, a mobile device such as a mobile phone).

[0161] For example, processor 1001 can control other components in electronic device 1000 to perform desired functions. Processor 1001 can be a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The central processing unit (CPU) can be an X86 or ARM architecture, etc. The GPU can be separately integrated directly into the motherboard, or built into the motherboard's north bridge chip. The GPU can also be built into the central processing unit (CPU).

[0162] For example, the memory 1002 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, a flash memory, etc. One or more computer-executable instructions may be stored on the computer-readable storage medium, and the processor 1001 may execute the computer-executable instructions to implement various functions of the electronic device 1000. Various applications and various data may also be stored in the memory 1002.

[0163] It should be noted that the electronic device 1000 can achieve technical effects similar to those of the aforementioned image processing method, and repeated details will be omitted.

[0164] Figure 11 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. Figure 11 As shown, one or more computer-executable instructions 1101 may be non-transitory stored on a non-transitory computer-readable storage medium 1100. For example, when the computer-executable instructions 1101 are executed by a processor, one or more steps in the image processing method according to any embodiment of the present disclosure may be performed.

[0165] For example, the non-transitory computer-readable storage medium 1100 may be applied to the electronic device 1000 described above. For example, it may include the memory 1002 in the electronic device 1000 .

[0166] For example, the description of the non-transitory computer-readable storage medium 1100 may refer to the description of the memory 1002 in the embodiment of the electronic device 1000, and the repeated parts will be omitted.

[0167] Figure 12 This is a schematic diagram of a hardware environment provided by at least one embodiment of the present disclosure. The electronic device provided by the present disclosure can be applied in an Internet system.

[0168] use Figure 12 The computer system provided in can realize the functions of the image processing device and / or electronic device involved in the present disclosure. Such computer systems may include personal computers, laptops, tablet computers, mobile phones, personal digital assistants, smart glasses, smart watches, smart rings, smart helmets and any smart portable devices or wearable devices, etc. The specific system in this embodiment uses a functional block diagram to explain a hardware platform including a user interface. Such a computer device may be a general-purpose computer device or a computer device with a specific purpose. Both computer devices can be used to implement the image processing device and / or electronic device in this embodiment. The computer system may include any components that implement the information required to implement the image processing currently described. For example, the computer system can be implemented by a computer device through its hardware devices, software programs, firmware and their combination. For convenience, Figure 12 Only one computer device is depicted in the figure, but the computer functions related to the information required for image processing described in this embodiment can be implemented in a distributed manner by a group of similar platforms to disperse the processing load of the computer system.

[0169] like Figure 12As shown, the computer system may include a communication port 250, connected to which is a network for data communication ( Figure 12 The computer system may also include a processor group 220 (i.e., the processor described above) for executing program instructions. The processor group 220 may be composed of at least one processor (e.g., a CPU). The computer system may include an internal communication bus 210. The computer system may include different forms of program storage units and data storage units (i.e., the memory or storage medium described above), such as a hard disk 270, a read-only memory (ROM) 230, and a random access memory (RAM) 240, which can be used to store various data files used for computer processing and / or communication, as well as possible program instructions executed by the processor group 220. The computer system may also include an input / output 260, which is used to implement input / output data flow between the computer system and other components (e.g., a user interface 280, etc.).

[0170] Typically, the following devices can be connected to input / output 260: input devices including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices including, for example, a tape, a hard disk, etc.; and a communication interface.

[0171] Although Figure 12 A computer system is shown with various devices, but it should be understood that the computer system is not required to have all of the devices shown, and the computer system may alternatively have more or fewer devices.

[0172] Regarding this disclosure, the following points need to be explained:

[0173] (1) The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure. Other structures may refer to conventional designs.

[0174] (2) For the sake of clarity, the thickness and size of layers or structures in the drawings used to describe the embodiments of the present invention are exaggerated. It will be understood that when an element such as a layer, film, region, or substrate is referred to as being "on" or "under" another element, the element can be "directly on" or "under" the other element, or intervening elements may be present.

[0175] (3) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to form new embodiments.

[0176] Although the present disclosure has been described in detail above using general descriptions and specific embodiments, it will be apparent to those skilled in the art that modifications or improvements may be made based on the embodiments of the present disclosure. Therefore, such modifications or improvements, as long as they do not depart from the spirit of the present disclosure, are within the scope of protection claimed by the present disclosure.

Claims

1. An image processing method, comprising: Get the original image; Processing the original image to obtain a pre-processed image, wherein the pre-processed image includes at least two first lines, and the at least two first lines are sequentially arranged in parallel along the same direction; Processing the pre-processed image using a distortion processing model to obtain an intermediate image, wherein the intermediate image includes at least two second lines, the at least two second lines are sequentially arranged in parallel along the same direction, and the at least two second lines correspond one-to-one to the at least two first lines; remapping the original image based on a mapping relationship between the preprocessed image and the intermediate image to obtain an output image; The step of remapping the original image based on the mapping relationship between the preprocessed image and the intermediate image to obtain an output image comprises: determining, based on a mapping relationship between the preprocessed image and the intermediate image, preprocessing mapping information corresponding to the preprocessed image by an interpolation method, wherein the preprocessing mapping information is used to indicate mapping parameters of at least some pixels in the preprocessed image; Determining, based on the preprocessing mapping information, mapping information corresponding to a region in the preprocessing image corresponding to the original image; performing scaling processing on mapping information of an area corresponding to the original image to determine mapping information corresponding to the original image; The original image is remapped based on mapping information corresponding to the original image to obtain the output image.

2. The image processing method according to claim 1, wherein: The mapping relationship between the preprocessed image and the intermediate image includes a mapping relationship between the at least two first lines and the at least two second lines and a mapping relationship between an area between the at least two first lines in the preprocessed image and an area between the at least two second lines in the intermediate image.

3. The image processing method according to claim 1, wherein: At least part of the pixels in the pre-processed image include pixels in a region between the at least two first lines in the pre-processed image and pixels on the at least two first lines.

4. The image processing method according to any one of claims 1 to 3, wherein: Processing the original image to obtain a preprocessed image includes: Performing binarization processing on the original image to obtain an input image; Performing scaling processing on the input image to obtain a scaled image; Performing a filling process on the scaled image to obtain a filled image; The padded image is divided into regions to obtain the pre-processed image.

5. The image processing method according to claim 4, wherein: The scaled image includes a first scaled image side and a second scaled image side opposite to each other, the pre-processed image includes a first pre-processed image side and a second pre-processed image side opposite to each other, the first pre-processed image side corresponds to the first scaled image side, the second pre-processed image side corresponds to the second scaled image side, and the at least two first lines are arranged between the first pre-processed image side and the second pre-processed image side in a direction from the first pre-processed image side to the second pre-processed image side. Performing padding processing on the scaled image to obtain the padded image, comprising: Filling a first filling area on a side of the first zoomed image away from the second zoomed image and filling a second filling area on a side of the second zoomed image away from the first zoomed image, to obtain the filled image, The two opposite sides of the first filling area are the first zoomed image side and the first pre-processed image side, and the two opposite sides of the second filling area are the second zoomed image side and the second pre-processed image side. The image processing method according to claim 5 , wherein: The size of the first filling area is the same as the size of the second filling area.

7. The image processing method according to any one of claims 1 to 3, wherein: Processing the original image to obtain a preprocessed image includes: Performing binarization processing on the original image to obtain an input image; Performing padding processing on the input image to obtain a padded image; performing scaling processing on the padded image to obtain a scaled image; The scaled image is divided into regions to obtain the preprocessed image.

8. The image processing method according to any one of claims 1 to 3, wherein: The at least two first lines are at least two equally dividing lines that equally divide the pre-processed image along the same direction.

9. The image processing method according to any one of claims 1 to 3, wherein: The distortion processing model is a neural network-based model.

10. The image processing method according to any one of claims 1 to 3, wherein: The image content in the original image is distorted.

11. The image processing method according to any one of claims 1 to 3, further comprising: Training the distortion processing model, wherein training the distortion processing model comprises: generating a training image, wherein the training image comprises at least two training lines, and the at least two training lines are sequentially arranged in parallel along the same direction; generating a target image corresponding to the training image based on the training image, wherein the target image includes at least two target training lines, the at least two target training lines are sequentially arranged in parallel along the same direction, and the at least two target training lines correspond one-to-one to the at least two training lines; The distortion processing model to be trained is trained based on the training image and the target image to obtain the trained distortion processing model.

12. The image processing method according to claim 11, wherein: Training the distortion processing model to be trained based on the training image and the target image to obtain the trained distortion processing model, including: Processing the training image using the distortion processing model to be trained to obtain an output training image, wherein the output training image includes at least two output lines, the at least two output lines are sequentially arranged in parallel along the same direction, and the at least two output lines correspond one-to-one to the at least two training lines; Adjusting parameters of the distortion processing model to be trained based on the output training image and the target image; When the loss function corresponding to the distortion processing model to be trained meets a predetermined condition, the trained distortion processing model is obtained; when the loss function corresponding to the distortion processing model to be trained does not meet the predetermined condition, the training image and the target image are continued to be input to repeat the above training process.

13. The image processing method according to claim 11, wherein: Generating the training image includes: Generate input training images; performing scaling processing on the input training image to obtain a scaled input training image; Performing padding processing on the scaled input training image to obtain a padded input training image; performing a distortion process on the padded input training image to obtain a distorted input training image; The distorted input training image is divided into regions to obtain the training image including the at least two training lines.

14. The image processing method according to claim 13, wherein: Generating a target image corresponding to the training image based on the training image includes: performing a reverse warping process on the training image based on a warping parameter corresponding to the warping process to obtain the target image.

15. The image processing method according to claim 13, wherein: Generate input training images, including: Get the original training image; The original training image is binarized to obtain the input training image.

16. The image processing method according to claim 11, wherein: The at least two training lines are at least two equally dividing lines that equally divide the training image along the same direction.

17. An image processing apparatus, comprising: An image acquisition module is configured to acquire an original image; a first processing module configured to process the original image to obtain a pre-processed image, wherein the pre-processed image includes at least two first lines, and the at least two first lines are sequentially arranged in parallel along the same direction; a second processing module configured to process the pre-processed image using a distortion processing model to obtain an intermediate image, wherein the intermediate image includes at least two second lines, the at least two second lines are sequentially arranged in parallel along the same direction, and the at least two second lines correspond one-to-one to the at least two first lines; a mapping module configured to remap the original image based on a mapping relationship between the preprocessed image and the intermediate image to obtain an output image; The mapping module remaps the original image based on the mapping relationship between the preprocessed image and the intermediate image to obtain the output image, including the following operations: determining, based on a mapping relationship between the preprocessed image and the intermediate image, preprocessing mapping information corresponding to the preprocessed image by an interpolation method, wherein the preprocessing mapping information is used to indicate mapping parameters of at least some pixels in the preprocessed image; Determining, based on the preprocessing mapping information, mapping information corresponding to a region in the preprocessing image corresponding to the original image; performing scaling processing on mapping information of an area corresponding to the original image to determine mapping information corresponding to the original image; The original image is remapped based on mapping information corresponding to the original image to obtain the output image.

18. An electronic device comprising: a memory that non-transitorily stores computer-executable instructions; a processor configured to execute the computer-executable instructions, Wherein, when the computer executable instructions are executed by the processor, the image processing method according to any one of claims 1 to 16 is implemented.

19. A non-transitory computer-readable storage medium, wherein: The non-transitory computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the image processing method according to any one of claims 1 to 16.

Images