Image processing method, device, equipment and storage medium

Abstract

The embodiments of the present application provide an image processing method, device, equipment and storage medium. The method includes: identifying a target object in an image to be processed, obtaining coordinate information of a target quadrilateral, the target quadrilateral being used to characterize the shape of the target object in the image, determining whether the target quadrilateral has at least one set of parallel sides based on the coordinate information of the target quadrilateral, and determining the aspect ratio of the target object based on whether the target quadrilateral has at least one set of parallel sides, and then processing the image based on the aspect ratio. The method achieves obtaining the corresponding aspect ratio according to different shapes of the target quadrilateral, making the aspect ratio closer to the real aspect ratio of the target object, thereby making the target object have a better presentation effect in the image.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of image processing technology, and more specifically, to an image processing method, apparatus, device, and storage medium. Background Art

[0002] As the degree of electronicization continues to deepen, users often need to use electronic devices including mobile phones to capture images of some document objects. However, the captured images show a tilted view effect of the document objects in the images due to the tilt of the electronic device during shooting.

[0003] In this regard, the prior art often identifies a target quadrilateral in an image, calculates the aspect ratio of a document-like object based on the target quadrilateral, and then corrects the image based on the aspect ratio.

[0004] However, there is no better technical solution for accurately obtaining the aspect ratio in the prior art, resulting in poor display effect of the corrected image. Summary of the invention

[0005] The embodiments of the present application provide an image processing method, apparatus, device and storage medium.

[0006] In a first aspect, a method for processing an image is provided, comprising:

[0007] Identify the target object in the image to be processed and obtain the coordinate information of the target quadrilateral, where the target quadrilateral is used to represent the shape of the target object in the image;

[0008] Based on the coordinate information of the target quadrilateral, determining whether the target quadrilateral has at least one set of parallel sides;

[0009] determining an aspect ratio of the target object based on whether at least one set of parallel sides of the target quadrilateral exists;

[0010] Based on the aspect ratio, the image is processed.

[0011] In a second aspect, an image processing device is provided, comprising:

[0012] A recognition unit, used for recognizing a target object in the image to be processed, and obtaining coordinate information of a target quadrilateral, where the target quadrilateral is used for representing a shape of the target object in the image;

[0013] a processing unit, configured to determine whether the target quadrilateral has at least one set of parallel sides based on the coordinate information of the target quadrilateral;

[0014] The processing unit is also used to determine the aspect ratio of the target object based on whether the target quadrilateral has at least one set of parallel sides;

[0015] The processing unit is further configured to process the image based on the aspect ratio.

[0016] According to a third aspect, an electronic device is provided, comprising: a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method in the first aspect or its various implementations.

[0017] According to a fourth aspect, a computer-readable storage medium is provided for storing a computer program, wherein the computer program enables a computer to execute the method according to the first aspect or its various implementations.

[0018] According to a fifth aspect, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions enable a computer to execute the method according to the first aspect or its various implementations.

[0019] According to a sixth aspect, a computer program is provided, which enables a computer to execute the method according to the first aspect or its various implementations.

[0020] The technical solution of the first aspect above identifies and analyzes the target quadrilateral representing the shape of the target object in the image, determines whether the target quadrilateral has at least one set of parallel sides, and distinguishes between target quadrilaterals with at least one set of parallel sides and target quadrilaterals without at least one set of parallel sides, so as to adopt corresponding technical means to obtain the most realistic aspect ratio of the target object, so that the target object has a better presentation effect in the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of a process of an image correction scenario 100 provided in an embodiment of the present application;

[0022] Figure 2 A schematic diagram of a framework of an electronic device 200 provided in an embodiment of the present application;

[0023] Figure 3 A schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of an image 500 to be processed provided in an embodiment of the present application;

[0026] Figure 6 A schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application;

[0027] Figure 7A schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application;

[0028] Figure 8 A schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application;

[0029] Fig. 9 A schematic diagram of an image processing process provided in an embodiment of the present application;

[0030] Fig.10 A schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application;

[0031] Fig.11 A schematic block diagram of an image processing device according to an embodiment of the present application is shown;

[0032] Fig.12 A schematic structural diagram of an electronic device 1200 provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0033] The following will describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. For the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0034] At present, users often use mobile phones, tablets, computers, wearable devices and other terminal devices instead of scanners and other devices to collect images of document objects (the same as the target object below), which improves the convenience of obtaining document objects. It should be understood that document objects include but are not limited to documents, business cards, posters, bulletin boards, erasable whiteboards or blackboards, etc. Because it is difficult to control the relative shooting angle between the terminal device and the target object, the document object in the captured image often presents a tilted irregular quadrilateral due to perspective transformation, that is, the target quadrilateral. Therefore, it is necessary to correct the image for the tilt of the document object, so that the quadrilateral in the corrected image changes from an irregular quadrilateral to a rectangle, and presents it with the aspect ratio of the document object itself. However, in the actual image processing process, the aspect ratio of document objects is often set to a fixed value, or the aspect ratio of the target quadrilateral is used as the aspect ratio of the document object without considering the deformation of the document object. As a result, the aspect ratio used in image processing is quite different from the actual aspect ratio of the document object, causing the document objects presented in the corrected image to have problems such as distorted aspect ratio and difficult-to-read text.

[0035] The embodiments of the present application are applied to the above-mentioned scenarios, combined with Figure 1As shown, the embodiment of the present application acquires an image of the target object and obtains an image 110 to be processed. By identifying the target object in the image 110, a target quadrilateral used to characterize the shape of the target object in the image is obtained. The target object in the image 110 presents a tilted display effect. Correspondingly, the target quadrilateral 120 is an irregular quadrilateral. Furthermore, in order to accurately obtain the aspect ratio of the target object, the embodiment of the present application analyzes the shape of the target quadrilateral 120, and distinguishes different shapes of the quadrilateral 120 according to the shape of the target quadrilateral 120, for example, whether there is at least one set of parallel sides, and adopts different aspect ratio acquisition methods according to different shapes. The image is processed based on the aspect ratio of the target object, for example, a perspective transformation is performed to obtain an image 111, so that the transformed target quadrilateral 120 presented in the image 111 has the aspect ratio of the target object, thereby realizing the restoration of the shape of the target object and avoiding the problem of aspect ratio distortion of the target object presented in the image.

[0036] As an example and not limitation, after obtaining image 111, in order to further ensure that there is no aspect ratio distortion of the target object in image 111, the text in the target object can be recognized to obtain the average aspect ratio of the text, and the image can be further adjusted based on the average aspect ratio of the text so that the target object presented in the image has a better display effect.

[0037] The technical solution of the embodiment of the present application can be applied to various electronic devices to realize at least one of the verification, optimization and testing of the language algorithm model. The electronic device can be a terminal device, such as a mobile phone, a tablet computer, a computer, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a terminal device in industrial control, a terminal device in self-driving, a terminal device in remote medical, a terminal device in a smart city or a terminal device in a smart home, etc. The terminal device in the embodiment of the present application can also be a wearable device, which can also be called a wearable smart device, which is a general term for the intelligent design of daily wearables using wearable technology and the development of wearable devices, such as glasses, gloves, watches, clothing and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. The terminal device can be fixed or mobile.

[0038] Exemplarily, the electronic device in the embodiment of the present application can also be a server. When the electronic device is a server, it can receive an image captured by a terminal device, determine the aspect ratio of the target object in the image, and then process the image according to the aspect ratio of the target object.

[0039] Figure 2 Schematic diagram of the structure of an electronic device 200 provided in an embodiment of the present application. Figure 2 As shown, the electronic device 200 includes: an image acquisition unit 210 , an image recognition unit 220 and an image processing unit 230 , and the image processing unit 230 includes at least an aspect ratio calculation subunit 231 .

[0040] The image acquisition unit 210 is used to acquire the image to be processed, which should include a document class object. For example, it can receive an image acquired by an image acquisition device, or receive an image transmitted by other devices, or receive an image input by a user, which is not limited in the present embodiment.

[0041] The image recognition unit 220 receives the image to be processed sent by the image acquisition unit 210, and recognizes the target object in the image to obtain a target quadrilateral for representing the shape of the target object in the image. Figure 1 As shown, the target quadrilateral 120 can be obtained. Further, the coordinate information of the identified target quadrilateral 120 is sent to the image processing unit 230.

[0042] The image processing unit 230 receives the target quadrilateral sent by the image recognition unit 220 , and determines the aspect ratio of the target object through the aspect ratio calculation subunit 231 .

[0043] Furthermore, the image processing unit 230 performs perspective transformation on the image according to the aspect ratio, or directly outputs the aspect ratio to other devices, or sends the aspect ratio to a display unit for display.

[0044] As an example, after performing perspective transformation on the image according to the aspect ratio, the image processing unit 230 identifies and analyzes the text in the transformed image to determine whether the aspect ratio of the target object in the transformed image is within the aspect ratio. If the aspect ratio is distorted, the transformed image is adjusted according to the average aspect ratio of the text, or the aspect ratio of the target object used in the image transformation is modified according to the average aspect ratio of the text.

[0045] Optionally, the image processing unit 230 further includes a focal length determining subunit 232, which is used to determine the focal length of the image so that the aspect ratio calculating subunit 231 can calculate the aspect ratio of the target object in combination with the focal length of the image.

[0046] The present application is described in detail below through several embodiments.

[0047] Figure 3 A flowchart of an image processing method provided in an embodiment of the present application.

[0048] In order to obtain the accurate aspect ratio of the target object and better present the target object in the image, the embodiment of the present application determines how to obtain the aspect ratio of the target object more accurately based on whether the target quadrilateral has at least one set of parallel sides, and then obtains the aspect ratio in a corresponding manner.

[0049] like Figure 3 As shown, the image processing method includes:

[0050] S310: Identify the target object in the image to be processed and obtain coordinate information of the target quadrilateral.

[0051] The target quadrilateral is used to characterize the shape of the target object in the image. Generally speaking, due to the relative shooting angle between the image acquisition device and the target object during image acquisition, the target object appears as a tilted or deformed figure in the image, and accordingly, the target quadrilateral should be an irregular quadrilateral.

[0052] In this step, the target object in the image can be identified by any image recognition method to obtain the coordinate information of the target quadrilateral. Exemplarily, the coordinate information includes the coordinates of the four vertices of the quadrilateral.

[0053] S320: Based on the coordinate information of the target quadrilateral, determine whether the target quadrilateral has at least one set of parallel sides.

[0054] Exemplary, combined Figure 4 As shown, step S320 may specifically include the following steps:

[0055] S321: Determine the target detection value based on the coordinate information of the target quadrilateral.

[0056] It should be understood that the target detection value is used to determine whether a quadrilateral has at least one set of parallel sides.

[0057] Exemplary, combined Figure 5 As shown, the target quadrilateral 501 in the image 500 has four vertices: a first vertex A, a second vertex B, a third vertex C, and a fourth vertex D. The four vertices may be in a clockwise order or a counterclockwise order. Figure 5Shown are four vertices A, B, C, and D arranged in counterclockwise order, and each vertex corresponds to a vector, which are vectors of the coordinates of a preset reference point pointing to the coordinates of each vertex. The reference point is any point in the plane coordinate system where the image is located. Optionally, the reference point can be the origin in the plane coordinate system. For example, the vector corresponding to vertex A is m1, the vector corresponding to vertex B is m2, the vector corresponding to vertex C is m3, and the vector corresponding to vertex D is m4.

[0058] Optionally, the target detection value includes a first detection value and a second detection value, and each detection value is used to determine that a corresponding set of opposite sides are parallel sides. Exemplarily, by formula The first detection value k2 is calculated, where m1×m4·m3 can be described as the first value, and m1×m4·m3 can be described as the second value. The second detection value k3 is calculated, wherein m1×m4·m2 can be described as a third value, and m1×m4·m2 can be described as a fourth value.

[0059] S322: Based on the target detection value and a preset threshold, determine whether the target quadrilateral has at least one set of parallel sides.

[0060] Exemplarily, for each detection value in the target detection value, determine whether the difference between the detection value and the preset threshold is less than 1. If there is at least one detection value in the target detection value whose difference between the detection value and the preset threshold is less than 1, it indicates that the target quadrilateral has at least one set of parallel sides.

[0061] For example, the formula |k2-∈|<1 is used to determine whether the difference between the first detection value k2 and the preset threshold ∈ is less than 1, and the formula |k3-∈|<1 is used to determine whether the difference between the second detection value k2 and the preset threshold ∈ is less than 1. Optionally, ∈=0.0001.

[0062] S330: Determine the aspect ratio of the target object based on whether the target quadrilateral has at least one set of parallel sides.

[0063] In this step, the aspect ratio of the target object is determined in a corresponding manner based on whether the target quadrilateral has at least one set of parallel sides. For example, when the target quadrilateral does not have at least one set of parallel sides, the focal length of the image is calculated based on the coordinate information of the target quadrilateral, and the aspect ratio is determined based on the focal length and the coordinate information of the target quadrilateral; or, when the target quadrilateral has at least one set of parallel sides, the focal length of the image is determined based on the parameter information of the camera, and the aspect ratio is determined based on the focal length and the coordinate information of the target quadrilateral; or, when the target quadrilateral has at least one set of parallel sides, the aspect ratio is determined directly based on the coordinate information of the quadrilateral.

[0064] S340: Process the image based on the aspect ratio.

[0065] Exemplarily, based on the determined aspect ratio, the image may be perspective transformed so that the target object is presented in the transformed image according to the aspect ratio, thereby obtaining a better display effect.

[0066] Based on the above process, the example of the present application identifies and analyzes the target quadrilateral that represents the shape of the target object in the image, determines whether the target quadrilateral has at least one set of parallel sides, and distinguishes between target quadrilaterals that have at least one set of parallel sides and target quadrilaterals that do not have at least one set of parallel sides, so as to adopt corresponding technical means to obtain the most realistic aspect ratio of the target object, so that the target object has a better presentation effect in the image.

[0067] Based on the above embodiments, the embodiments of the present application are combined with Figure 6 The content shown provides an exemplary description of how to determine the aspect ratio of a target object based on whether the target quadrilateral has at least one set of parallel sides.

[0068] Figure 6 The following is a flow chart of an image processing method provided in an embodiment of the present application. Figure 6 As shown, after determining whether the target quadrilateral has at least one set of parallel sides, the aspect ratio of the target object can be determined by at least the following four implementation methods.

[0069] Method 1: When the target quadrilateral does not have at least one set of parallel sides, the focal length of the image can be determined according to the coordinate information of the target quadrilateral, and the aspect ratio of the target object can be determined according to the focal length of the image and the coordinate information of the target quadrilateral.

[0070] It should be understood that the focal length of the image determined based on the coordinate information of the target quadrilateral has a stronger correlation with the target quadrilateral, that is, it can more truly reflect the focal length when the image of the target object is captured. Furthermore, the aspect ratio determined based on the focal length and coordinate information is closer to the actual aspect ratio of the target object.

[0071] Combination Figure 6 As shown, after determining that the target quadrilateral does not have at least one set of parallel sides, the following steps are performed:

[0072] S331: Determine a first focal length based on the coordinate information of the target quadrilateral.

[0073] Exemplarily, based on the first detection value k2, the vector m1 corresponding to the first vertex A and the vector m2 corresponding to the second vertex B, the first vector n2 is determined, for example, the first vector n2 is calculated by the formula n2=k2m2-m1; based on the second detection value k3, the vector m1 corresponding to the first vertex A and the vector m3 corresponding to the third vertex C, the second vector n3 is determined, for example, the second vector n3 is calculated by the formula n3=k3m3-m1; based on the first coordinate component n of the first vector n2 2x and the third coordinate component n 2z , the first coordinate component n of the second vector n3 3x and the third coordinate component n 3z , and the first coordinate component u0 of the center coordinate of the image, determine the third value t1, where the center coordinate of the image is the coordinate of the center position of the image, for example, by the formula The third value t1 is calculated based on the second coordinate component n of the first vector n2. 2y and the third coordinate component n 2z , the second coordinate component n of the second vector n3 3y and the third coordinate component n 3z , and the second coordinate component v0 of the center coordinate of the image, determine the fourth value, for example, by the formula Calculate the fourth value t2; determine the first focal length f based on the third value t1, the fourth value t2, the first vector n2 and the second vector n3. For example, the third value t1, the fourth value t2, the third coordinate component n of the first vector n2 2z and the third coordinate component n of the second vector n3 3z , determine the first focal length f, for example, by the formula The first focal length f is calculated.

[0074] S332: Determine the focal length of the image based on the first focal length.

[0075] Generally speaking, the first focal length can be directly determined as the focal length of the image. In order to avoid abnormalities in the calculated focal length, it is possible to detect whether the first focal length is abnormal. After confirming that there is no abnormality in the first focal length, the first focal length is determined as the focal length of the image.

[0076] S333: Determine the aspect ratio of the target object based on the focal length of the image and the coordinate information of the target quadrilateral.

[0077] Exemplarily, based on the focal length determined in the above steps and the center coordinates of the image, a first matrix is ​​determined, and then based on the first matrix, the first vector and the second vector, the aspect ratio of the target object is determined.

[0078] For example, by the formula Get the first matrix A- T A- 1 , and then through the formula Determine the aspect ratio α of the target object.

[0079] Optionally, when the target quadrilateral does not have at least one set of parallel sides, the focal length f of the image can be set to a default focal length, or the focal length parameters set by the image acquisition device can be directly read, or the focal length information can be obtained from the image information carried by the image, or the aspect ratio of the target object can be calculated directly based on the coordinate information of the target quadrilateral without obtaining the focal length of the image. Compared with the first method in which the focal length of the image is calculated based on the coordinate information of the target quadrilateral, any of the aforementioned possible implementation methods has poorer accuracy in calculating the aspect ratio, but has higher processing efficiency.

[0080] The above-mentioned method 1 is the preferred implementation method when the target quadrilateral does not have at least one set of parallel sides. When the target quadrilateral has at least one set of parallel sides, it is impossible to obtain the accurate focal length based on the coordinate information of the target quadrilateral. Therefore, the following methods 2, 3 and 4 are provided to achieve the focal length of the image when the target quadrilateral has at least one set of parallel sides.

[0081] Method 2: When the target quadrilateral has at least one set of parallel sides, the focal length of the image is determined based on the parameter information of the image, and the aspect ratio of the target object is determined according to the focal length of the image and the coordinate information of the target quadrilateral.

[0082] Combination Figure 6 As shown, after determining that the target quadrilateral has at least one set of parallel sides, the following process is included:

[0083] S334: Obtain parameter information of the image.

[0084] Optionally, parameter information set in the image acquisition device can be read, and the parameter information includes the focal length when the image is acquired; or, parameter information carried by the image can be obtained, and the parameter information includes the focal length when the image is acquired. If the image is cropped after acquisition, the parameter information also includes cropping information.

[0085] S335: Determine the focal length of the image based on the parameter information.

[0086] Exemplarily, it is determined whether the parameter information contains cropping information. If it contains cropping information, the focal length in the parameter information is corrected based on the cropping information to obtain the focal length of the image; if it does not contain cropping information, or the parameter information is read from an image acquisition device, the focal length in the parameter information is directly used as the focal length of the image.

[0087] S333: Determine the aspect ratio of the target object based on the focal length of the image and the coordinate information of the target quadrilateral.

[0088] Method 3: When the target quadrilateral has at least one set of parallel sides, the default focal length is determined as the focal length of the image, and the aspect ratio of the target object is determined based on the focal length of the image and the coordinate information of the target quadrilateral. Figure 6 Step S335 and step S333 in the embodiment. The default focal length is preset.

[0089] Method 4: When the target quadrilateral has at least one set of parallel sides, the focal length of the image is not obtained, and the aspect ratio of the target object is directly determined according to the coordinate information of the target quadrilateral.

[0090] Compared with method 2 or method 3, method 4 has lower accuracy but higher processing efficiency.

[0091] Combination Figure 6 As shown, after determining that the target quadrilateral has at least one set of parallel sides, the following steps are performed:

[0092] S336: Determine the aspect ratio of the target object based on the coordinate information of the target quadrilateral.

[0093] Exemplarily, the ratio of the sum of the lengths of any two adjacent sides of the target quadrilateral to the sum of the lengths of the two diagonals of the target quadrilateral is used as the aspect ratio of the target object. Figure 5 As shown, according to the two adjacent side lines AB and CD, and the two diagonals AD and BC, by the formula The aspect ratio α is calculated.

[0094] In summary, when the target quadrilateral does not have at least one set of parallel sides, the embodiment of the present application determines the most accurate focal length based on the coordinate information of the target quadrilateral, thereby determining the aspect ratio of the target object based on the focal length and the coordinate information of the target quadrilateral, and the aspect ratio is closer to the actual aspect ratio of the target object, so that the target object has a better presentation effect in the corrected image. Furthermore, when the target quadrilateral has at least one set of parallel sides, the focal length is obtained according to the default focal length, or the focal length is obtained from the parameter information carried by the image, or the parameter information is read from the image acquisition device to obtain the focal length, which is more accurate than the focal length determined according to the coordinate information of the target quadrilateral. In addition, in order to improve processing efficiency, when the target quadrilateral has at least one set of parallel sides, the embodiment of the present application can also directly determine the aspect ratio according to the coordinate information of the target quadrilateral.

[0095] In order to obtain an accurate aspect ratio, the embodiment of the present application needs to ensure that the focal length of the image is within a reasonable focal length range. Based on any of the above embodiments, Figure 7The following is a flow chart of an image processing method provided in an embodiment of the present application. Figure 7 As shown, step S332 determines the focal length of the image based on the first focal length, including:

[0096] S3321: Determine whether the first focal length is within a preset range.

[0097] Exemplarily, the preset range includes an upper threshold and a lower threshold, and the preset range is calculated based on the image width w and the preset field of view angle θ. fov Determine the upper and lower thresholds, such as the formula As shown, the upper threshold should be the preset field of view angle θ fov A tangent operation is performed on half of the image width w, and then multiplied by half of the image width w to obtain the upper threshold, and the opposite of the upper threshold is used as the lower threshold.

[0098] Determine whether the first focal length is within a preset range, that is, determine whether the first focal length f1 is greater than a lower threshold and less than an upper threshold, that is, f1 2 Is it less than and greater than zero.

[0099] If the first focal length is within the preset range, execute step S3322; if the first focal length is not within the preset range, execute step S3323 or S3324 or S3325.

[0100] S3322: Determine the first focal length as the focal length of the image.

[0101] S3323: Obtain parameter information of the image, and determine the focal length of the image based on the parameter information.

[0102] S3324: Determine the default focal length as the focal length of the image.

[0103] After step S3322 or step S3323 or step S3324 is executed, a step of determining the aspect ratio of the target object based on the focal length of the image and the coordinate information of the target quadrilateral is executed.

[0104] S3325: Determine the aspect ratio of the target object based on the coordinate information of the target quadrilateral.

[0105] The above steps S3323, S3324 and S3325 have similar implementation methods and technical effects as the corresponding steps in the above embodiments, and will not be repeated here.

[0106] Optionally, in a specific implementation process, determining the focal length of an image based on parameter information may include: determining a second focal length of the image based on the parameter information, determining whether the second focal length is within a preset range, if the second focal length is within the preset range, determining the second focal length as the focal length of the image, otherwise determining a default focal length as the focal length of the image.

[0107] Figure 8 A flowchart of an image processing method provided in an embodiment of the present application.

[0108] Based on any of the above embodiments, the embodiments of the present application are based on Figure 3 The embodiments shown are used as examples. Figure 8 As shown, after determining the aspect ratio of the target object, it also includes:

[0109] S410: Determine whether the difference between the aspect ratio and the preset aspect ratio is less than a preset error.

[0110] If the difference between the aspect ratio and the preset aspect ratio is less than the preset error, then S420 is executed: the aspect ratio of the target object is determined to be the preset aspect ratio; otherwise, S430 is executed: the aspect ratio of the target object is not changed. The image is processed based on the finally determined aspect ratio of the image.

[0111] For example, the preset aspect ratio is a common aspect ratio of a document object. For example, the preset aspect ratio is set to or or For the two values ​​of the target object with the same aspect ratio under different placement methods, the preset error is set to 0.05. or Determine whether the aspect ratio α of the target object is close to the preset aspect ratio or exist or When the aspect ratio α is close to or At this point, you can directly set the aspect ratio of the target object to or Otherwise, the value of the aspect ratio α is still used as the aspect ratio of the target object.

[0112] Optionally, after determining the aspect ratio of the target object, it is determined whether the aspect ratio α of the target object is abnormal, so as to avoid errors in the calculation process or the acquisition process that lead to an incorrect value of the aspect ratio α. 2 is greater than zero, and / or is α less than α t and greater than α t The reciprocal of Among them, α t is the maximum reasonable aspect ratio, the optional α t =10.

[0113] Fig. 9 A schematic diagram of an image processing process provided by an embodiment of the present application. Fig. 9As shown, before the image is processed, the target object appears to be tilted or deformed in the image, and accordingly, the text in the target object also has a corresponding deformation. After the image is perspective transformed according to the determined aspect ratio, the tilt or deformation of the text content is corrected, but the text may still be flat or narrow, resulting in the text being not beautiful enough or even inconvenient for users to read. Therefore, it is necessary to identify the text area and obtain the average aspect ratio of the text based on the text area, and continue to process the image to adjust the aspect ratio of the text in the target object or the target object itself to achieve a better display effect.

[0114] Fig.10 A flowchart of an image processing method provided in an embodiment of the present application is shown in FIG. Figure 3 The embodiments shown are used as examples. Fig.10 As shown, after the image is processed, it also includes:

[0115] S350: Identify the target object in the image to obtain text information.

[0116] It should be understood that the target object is a document-type object, including a plurality of characters. By identifying the target object in the image, text information can be obtained, and the text information at least includes a plurality of text regions, and each text region includes at least one character.

[0117] S360: Determine an average aspect ratio of the text based on the text information.

[0118] Exemplarily, the average aspect ratio of the text of the target object presented in the image is determined according to the aspect ratio of each text area.

[0119] Exemplarily, the average aspect ratio of the text may be determined by combining the line spacing and character spacing outside the text area with the aspect ratio of the text area.

[0120] S370: Based on the average aspect ratio of the text and the preset aspect ratio of the text, when the difference between the average aspect ratio of the text and the preset aspect ratio of the text is greater than a preset value, adjusting the aspect ratio of the target object according to the average aspect ratio of the text.

[0121] It should be understood that, according to different application scenarios, the preset text aspect ratio is the real aspect ratio of the text in the target object, or a universal text aspect ratio, which is preferably convenient for users to read or truly restore the target object.

[0122] When the difference between the average aspect ratio of the text and the preset aspect ratio of the text is greater than a preset value, the image is processed according to the preset aspect ratio of the text, or the aspect ratio of the target object is adjusted according to the average aspect ratio of the text, and the image is perspectively transformed again according to the adjusted aspect ratio of the target object.

[0123] Optionally, when the difference between the average aspect ratio of the text and the preset aspect ratio of the text is greater than a preset value, a prompt message can be sent to the user, allowing the user to choose whether to optimize the display effect of the text in the image, and in response to the optimization instruction input by the user, the image can be optimized, or the image processing process can be ended.

[0124] Combination of the above Figures 1 to 10 , describes in detail the method embodiment of the present application, and the following is combined with Fig.11 , the device embodiments of the present application are described in detail. It should be understood that the device embodiments and the method embodiments correspond to each other, and similar descriptions can refer to the method embodiments.

[0125] Fig.11 FIG. 1 is a schematic block diagram of an image processing device according to an embodiment of the present application. Fig.11 As shown, the image processing device 1100 includes:

[0126] An identification unit 1110 is used to identify a target object in the image to be processed and obtain coordinate information of a target quadrilateral, where the target quadrilateral is used to represent the shape of the target object in the image;

[0127] A processing unit 1120 is used to determine whether the target quadrilateral has at least one set of parallel sides based on the coordinate information of the target quadrilateral;

[0128] The processing unit 1120 is further configured to determine the aspect ratio of the target object based on whether the target quadrilateral has at least one set of parallel sides;

[0129] The processing unit 1120 is further configured to process the image based on the aspect ratio.

[0130] The image processing device 1100 in the embodiment of the present application includes a recognition unit 1110 and a processing unit 1120, which recognizes and analyzes a target quadrilateral representing the shape of a target object in an image, determines whether the target quadrilateral has at least one set of parallel sides, and distinguishes between a target quadrilateral with at least one set of parallel sides and a target quadrilateral without at least one set of parallel sides, so as to adopt corresponding technical means to obtain the most realistic aspect ratio of the target object, so that the target object has a better presentation effect in the image.

[0131] Optionally, the processing unit 1120 is specifically configured to:

[0132] Determine the target detection value based on the coordinate information of the target quadrilateral;

[0133] Based on the target detection value and a preset threshold, it is determined whether the target quadrilateral has at least one set of parallel sides.

[0134] Optionally, the processing unit 1120 is specifically configured to:

[0135] Based on the vector corresponding to each of the four vertices, the target detection value is determined; the vector corresponding to each vertex is a vector from the coordinates of a preset reference point to the coordinates of the vertex.

[0136] Optionally, the processing unit 1120 is specifically configured to:

[0137] Perform a cross product of the vector corresponding to the first vertex and the vector corresponding to the fourth vertex, and then perform a dot product with the vector corresponding to the third vertex to obtain a first value;

[0138] Perform a cross product of the vector corresponding to the second vertex and the vector corresponding to the fourth vertex, and then perform a dot product with the vector corresponding to the third vertex to obtain a second value;

[0139] The ratio of the first value to the second value is used as a first detection value;

[0140] Perform a cross product of the vector corresponding to the first vertex and the vector corresponding to the fourth vertex, and then perform a dot product with the vector corresponding to the second vertex to obtain a third value;

[0141] Perform a cross product of the vector corresponding to the third vertex and the vector corresponding to the fourth vertex, and then perform a dot product with the vector corresponding to the second vertex to obtain a fourth value;

[0142] using a ratio of the third value to the fourth value as a second detection value;

[0143] The four vertices of the quadrilateral include, in order in a clockwise direction or a counterclockwise direction: a first vertex, a second vertex, a third vertex and a fourth vertex, and the target detection value includes a first detection value and a second detection value.

[0144] Optionally, the processing unit 1120 is specifically configured to:

[0145] For each detection value in the target detection value, determine whether the difference between the detection value and a preset threshold value is less than 1;

[0146] If the difference between at least one of the target detection values ​​and the preset threshold is less than 1, then the target quadrilateral has at least one set of parallel sides;

[0147] Otherwise, the quadrilateral does not have at least one set of parallel sides.

[0148] Optionally, the processing unit 1120 is specifically configured to:

[0149] Determining the focal length of the image based on whether the target quadrilateral has at least one set of parallel sides;

[0150] The aspect ratio of the target object is determined based on the focal length of the image and the coordinate information of the target quadrilateral.

[0151] Optionally, the processing unit 1120 is specifically configured to:

[0152] If the target quadrilateral does not have at least one set of parallel sides, determining a first focal length based on the coordinate information of the target quadrilateral; determining a focal length of the image based on the first focal length;

[0153] If the target quadrilateral has at least one set of parallel sides, then obtaining parameter information of the image, and determining the focal length of the image based on the parameter information; or determining a default focal length as the focal length of the image;

[0154] The parameter information includes: the focal length when capturing the image, and / or cropping information for cropping the image.

[0155] Optionally, the processing unit 1120 is specifically configured to:

[0156] Determine a first vector based on the first detection value, the vector corresponding to the first vertex, and the vector corresponding to the second vertex;

[0157] Determine a second vector based on the second detection value, the vector corresponding to the first vertex, and the vector corresponding to the third vertex;

[0158] determining a third value based on the first coordinate component and the third coordinate component of the first vector, the first coordinate component and the third coordinate component of the second vector, and the first coordinate component of the center coordinate of the image;

[0159] determining a fourth value based on the second coordinate component and the third coordinate component of the first vector, the second coordinate component and the third coordinate component of the second vector, and the second coordinate component of the center coordinate of the image;

[0160] Determining a first focal length based on the third value, the fourth value, the first vector, and the second vector;

[0161] Among them, the coordinate information of the target quadrilateral includes the coordinates of the four vertices of the quadrilateral. The four vertices include, in sequence, the first vertex, the second vertex, the third vertex and the fourth vertex in a clockwise or counterclockwise direction. The vector corresponding to each vertex is a vector pointing from the coordinates of a preset reference point to the coordinates of the vertex. The first detection value and the second detection value are both determined based on the vector corresponding to each of the four vertices.

[0162] Optionally, the processing unit 1120 is specifically configured to:

[0163] Determine whether the parameter information includes clipping information;

[0164] If the parameter information includes cropping information, the focal length in the parameter information is corrected based on the cropping information to obtain the focal length of the image;

[0165] If the parameter information does not include cropping information, the focal length in the parameter information is used as the focal length of the image.

[0166] Optionally, the processing unit 1120 is specifically configured to:

[0167] Determine an aspect ratio of the target object based on the first vector, the second vector and the first matrix;

[0168] Among them, the coordinate information of the target quadrilateral includes the coordinates of the four vertices of the quadrilateral, and the four vertices include, in sequence, the first vertex, the second vertex, the third vertex and the fourth vertex in a clockwise or counterclockwise direction. The vector corresponding to each vertex is a vector of the coordinates of the preset reference point pointing to the coordinates of the vertex. The first vector is determined based on the first detection value, the vector corresponding to the first vertex and the vector corresponding to the second vertex; the second vector is determined based on the second detection value, the vector corresponding to the first vertex and the vector corresponding to the third vertex; the first matrix is ​​determined based on the center coordinates of the image and the focal length of the image; the first detection value and the second detection value are both determined based on the vector corresponding to each of the four vertices; the first matrix is ​​determined based on the focal length and the center coordinates of the image.

[0169] Optionally, the processing unit 1120 is specifically configured to:

[0170] Based on the first focal length and the preset range, when it is determined that the first focal length is within the preset range, the first focal length is determined as the focal length of the image.

[0171] Optionally, the processing unit 1120 is specifically configured to:

[0172] When it is determined that the first focal length is not within the preset range, parameter information of the image is acquired, and based on the parameter information, the focal length of the image is determined; or, a default focal length is determined as the focal length of the image.

[0173] Optionally, the processing unit 1120 is specifically configured to:

[0174] When it is determined that the first focal length is not within the preset range, the aspect ratio of the target object is determined based on the coordinate information of the target quadrilateral.

[0175] Optionally, the processing unit 1120 is further configured to:

[0176] Based on the width of the image and the preset field of view, an upper threshold and a lower threshold of the preset range are determined.

[0177] Optionally, the processing unit 1120 is specifically configured to:

[0178] A tangent operation is performed on half of the preset field of view angle, and then multiplied by half of the width of the image to obtain an upper threshold, and the opposite of the upper threshold is used as the lower threshold.

[0179] Optionally, the processing unit 1120 is specifically configured to:

[0180] If the target quadrilateral has at least one set of parallel sides, the aspect ratio of the target object is determined based on the coordinate information of the target quadrilateral.

[0181] Optionally, the processing unit 1120 is specifically configured to:

[0182] The ratio of the sum of the lengths of any two adjacent sides of the target quadrilateral to the sum of the lengths of the two diagonals of the target quadrilateral is taken as the aspect ratio of the target object.

[0183] Optionally, the processing unit 1120 is further configured to:

[0184] Based on the aspect ratio, the preset aspect ratio and the preset error, when the difference between the aspect ratio and the preset aspect ratio is less than the preset error, the aspect ratio of the target object is determined to be the preset aspect ratio.

[0185] Optionally, the processing unit 1120 is further configured to:

[0186] Recognize the target object in the image to obtain text information, where the target object includes multiple characters;

[0187] Based on the text information, determine the average aspect ratio of the text;

[0188] Based on an average aspect ratio of the text and a preset aspect ratio of the text, when a difference between the average aspect ratio of the text and the preset aspect ratio of the text is greater than a preset value, the aspect ratio of the target object is adjusted according to the average aspect ratio of the text.

[0189] The data processing device provided in the above embodiment can execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0190] Fig.12 A schematic structural diagram of an electronic device 1200 provided in an embodiment of the present application is shown. Fig.12 The electronic device shown includes a processor 1210, which can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0191] Alternatively, if Fig.11As shown, the electronic device 1200 may further include a memory 1220. The processor 1210 may call and run a computer program from the memory 1220 to implement the method in the embodiment of the present application.

[0192] The memory 1220 may be a separate device independent of the processor 1210 , or may be integrated into the processor 1210 .

[0193] Alternatively, if Fig.11 As shown, the electronic device 1200 may further include a transceiver 1230, and the processor 1210 may control the transceiver 1230 to communicate with other devices, specifically, may send information or data to other devices, or receive information or data sent by other devices.

[0194] The transceiver 1230 may include a transmitter and a receiver. The transceiver 1230 may further include an antenna, and the number of the antennas may be one or more.

[0195] Optionally, the electronic device 1200 can implement the corresponding processes in the various methods of the embodiments of the present application, which will not be described here for the sake of brevity.

[0196] It should be understood that the processor of the embodiment of the present application may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and performed. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, and other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0197] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0198] It should be understood that the above-mentioned memory is exemplary but not restrictive. For example, the memory in the embodiments of the present application may also be static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synch link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM), etc. That is to say, the memory in the embodiments of the present application is intended to include but not limited to these and any other suitable types of memory.

[0199] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.

[0200] Optionally, the computer-readable storage medium can be applied to the electronic device in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the electronic device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0201] An embodiment of the present application also provides a computer program product, including computer program instructions.

[0202] Optionally, the computer program product can be applied to the electronic device in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes implemented by the electronic device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0203] The embodiment of the present application also provides a computer program.

[0204] Optionally, the computer program can be applied to the electronic device in the embodiments of the present application. When the computer program runs on a computer, the computer executes the corresponding processes implemented by the electronic device in the various methods of the embodiments of the present application. For the sake of brevity, they are not described here.

[0205] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0206] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0207] In the several embodiments provided in the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0208] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0209] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0210] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. In view of such an understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0211] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for processing an image, characterized in that: include: Identify a target object in the image to be processed to obtain coordinate information of a target quadrilateral, where the target quadrilateral is used to characterize a shape of the target object in the image; Based on the coordinate information of the target quadrilateral, determining whether the target quadrilateral has at least one set of parallel sides; determining a focal length of the image based on whether the target quadrilateral has at least one set of parallel sides; Determining the aspect ratio of the target object based on the focal length of the image and the coordinate information of the target quadrilateral; The image is processed based on the aspect ratio.

2. The method according to claim 1, characterized in that The determining, based on the coordinate information of the target quadrilateral, whether the target quadrilateral has at least one set of parallel sides comprises: Determining a target detection value based on the coordinate information of the target quadrilateral; Based on the target detection value and a preset threshold, it is determined whether the target quadrilateral has at least one set of parallel sides.

3. The method according to claim 2, characterized in that The coordinate information of the target quadrilateral includes the coordinates of the four vertices of the quadrilateral, and the target detection value is determined based on the coordinate information of the target quadrilateral, including: Based on the vector corresponding to each of the four vertices, the target detection value is determined; the vector corresponding to each vertex is a vector in which the coordinates of a preset reference point point to the coordinates of the vertex.

4. The method according to claim 3, characterized in that The target detection value includes a first detection value and a second detection value, and the four vertices of the target quadrilateral include, in a clockwise direction or a counterclockwise direction, a first vertex, a second vertex, a third vertex, and a fourth vertex, respectively. Then, determining the target detection value based on a vector corresponding to each vertex of the four vertices includes: Perform a cross product of the vector corresponding to the first vertex and the vector corresponding to the fourth vertex, and then perform a dot product of the vector corresponding to the third vertex to obtain a first value; Perform a cross product of the vector corresponding to the second vertex and the vector corresponding to the fourth vertex, and then perform a dot product of the vector corresponding to the third vertex to obtain a second value; taking the ratio of the first value to the second value as a first detection value; Perform a cross product of the vector corresponding to the first vertex and the vector corresponding to the fourth vertex, and then perform a dot product of the vector corresponding to the second vertex to obtain a third value; Perform a cross product of the vector corresponding to the third vertex and the vector corresponding to the fourth vertex, and then perform a dot product of the vector corresponding to the second vertex to obtain a fourth value; The ratio of the third value to the fourth value is used as the second detection value.

5. The method according to any one of claims 2 to 4, characterized in that: The determining, based on the target detection value and a preset threshold, whether the target quadrilateral has at least one set of parallel sides comprises: For each detection value in the target detection value, determining whether the difference between the detection value and a preset threshold value is less than 1; If the difference between at least one of the target detection values ​​and a preset threshold value is less than 1, then the target quadrilateral has at least one set of parallel sides; Otherwise, the target quadrilateral does not have at least one set of parallel sides.

6. The method according to claim 1, characterized in that The determining the focal length of the image based on whether the target quadrilateral has at least one set of parallel sides comprises: If the target quadrilateral does not have at least one set of parallel sides, determining a first focal length based on the coordinate information of the target quadrilateral; and determining a focal length of the image based on the first focal length; If the target quadrilateral has at least one set of parallel sides, acquiring parameter information of the image, and determining the focal length of the image based on the parameter information; or determining a default focal length as the focal length of the image; The parameter information includes: the focal length when the image is captured, and / or cropping information for cropping the image.

7. The method according to claim 6, characterized in that The coordinate information of the target quadrilateral includes the coordinates of the four vertices of the quadrilateral, and the four vertices include: a first vertex, a second vertex, a third vertex and a fourth vertex in a clockwise direction or a counterclockwise direction, and the vector corresponding to each vertex is a vector from the coordinate of a preset reference point to the coordinate of the vertex. Then, determining the first focal length based on the coordinate information of the target quadrilateral includes: Determine a first vector based on the first detection value, the vector corresponding to the first vertex, and the vector corresponding to the second vertex; Determine a second vector based on the second detection value, the vector corresponding to the first vertex, and the vector corresponding to the third vertex; determining a third value based on the first coordinate component and the third coordinate component of the first vector, the first coordinate component and the third coordinate component of the second vector, and the first coordinate component of the center coordinates of the image; determining a fourth value based on the second coordinate component and the third coordinate component of the first vector, the second coordinate component and the third coordinate component of the second vector, and the second coordinate component of the center coordinate of the image; determining a first focal length based on the third value, the fourth value, the first vector, and the second vector; The first detection value and the second detection value are both determined based on a vector corresponding to each of the four vertices.

8. The method according to claim 6, characterized in that The determining the focal length of the image based on the parameter information includes: Determining whether the parameter information includes the cropping information; If the parameter information includes the cropping information, correcting the focal length in the parameter information based on the cropping information to obtain the focal length of the image; If the parameter information does not include the cropping information, the focal length in the parameter information is used as the focal length of the image.

9. The method according to claim 1, characterized in that: The coordinate information of the target quadrilateral includes the coordinates of the four vertices of the quadrilateral, and the four vertices include: a first vertex, a second vertex, a third vertex and a fourth vertex in a clockwise direction or a counterclockwise direction, and the vector corresponding to each vertex is a vector from the coordinate of a preset reference point to the coordinate of the vertex. Then, the aspect ratio of the target object is determined based on the focal length of the image and the coordinate information of the target quadrilateral, including: Determining an aspect ratio of the target object based on the first vector, the second vector and the first matrix; Among them, the first vector is determined based on the first detection value, the vector corresponding to the first vertex and the vector corresponding to the second vertex; the second vector is determined based on the second detection value, the vector corresponding to the first vertex and the vector corresponding to the third vertex; the first matrix is ​​determined based on the central coordinates of the image and the focal length of the image; the first detection value and the second detection value are both determined based on the vector corresponding to each of the four vertices; the first matrix is ​​determined based on the focal length and the central coordinates of the image.

10. The method according to any one of claims 6 to 8, characterized in that: The determining the focal length of the image based on the first focal length includes: Based on the first focal length and a preset range, when it is determined that the first focal length is within the preset range, the first focal length is determined as the focal length of the image.

11. The method according to claim 10, characterized in that When it is determined that the first focal length is not within the preset range, the method further includes: acquiring parameter information of the image, and determining the focal length of the image based on the parameter information; or, A default focal length is determined as the focal length of the image.

12. The method according to claim 10, characterized in that The determining the aspect ratio of the target object based on the focal length of the image and the coordinate information of the target quadrilateral includes: When it is determined that the first focal length is not within the preset range, the aspect ratio of the target object is determined based on the coordinate information of the target quadrilateral.

13. The method according to claim 10, characterized in that The method further comprises: An upper threshold and a lower threshold of the preset range are determined based on the width of the image and a preset field of view.

14. The method according to claim 13, characterized in that The step of determining an upper threshold and a lower threshold of the preset range based on the width of the image and the preset field of view angle includes: A tangent operation is performed on half of the preset field of view angle, and then multiplied by half of the width of the image to obtain the upper threshold, and the opposite of the upper threshold is used as the lower threshold.

15. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: If the target quadrilateral has at least one set of parallel sides, the aspect ratio of the target object is determined based on the coordinate information of the target quadrilateral.

16. The method according to claim 15, characterized in that The step of determining the aspect ratio of the target object based on the coordinate information of the target quadrilateral includes: The ratio of the sum of the lengths of any two adjacent side lines of the target quadrilateral to the sum of the lengths of the two diagonal lines of the target quadrilateral is used as the aspect ratio of the target object.

17. The method according to any one of claims 1 to 4, characterized in that: After determining the aspect ratio of the target object based on the focal length of the image and the coordinate information of the target quadrilateral, the method further includes: Based on the aspect ratio, a preset aspect ratio and a preset error, when a difference between the aspect ratio and the preset aspect ratio is less than the preset error, determining that the aspect ratio of the target object is the preset aspect ratio.

18. The method according to any one of claims 1 to 4, characterized in that: After processing the image based on the aspect ratio, the method further includes: Identify a target object in the image to obtain text information, wherein the target object includes a plurality of characters; Based on the text information, determining an average aspect ratio of the text; Based on the average aspect ratio of the characters and a preset aspect ratio of the characters, when the difference between the average aspect ratio of the characters and the preset aspect ratio of the characters is greater than a preset value, the aspect ratio of the target object is adjusted according to the average aspect ratio of the characters.

19. An image processing device, characterized in that: include: an identification unit, used for identifying a target object in the image to be processed, and obtaining coordinate information of a target quadrilateral, wherein the target quadrilateral is used for representing a shape of the target object in the image; a processing unit, configured to determine whether the target quadrilateral has at least one set of parallel sides based on the coordinate information of the target quadrilateral; The processing unit is further configured to determine the focal length of the image based on whether the target quadrilateral has at least one set of parallel sides; and determine the aspect ratio of the target object based on the focal length of the image and the coordinate information of the target quadrilateral; The processing unit is further configured to process the image based on the aspect ratio.

20. The device according to claim 19, characterized in that The processing unit is specifically used for: Determining a target detection value based on the coordinate information of the target quadrilateral; Based on the target detection value and a preset threshold, it is determined whether the target quadrilateral has at least one set of parallel sides.

21. The device according to claim 20, characterized in that The coordinate information of the target quadrilateral includes the coordinates of the four vertices of the quadrilateral, and the processing unit is specifically used for: Based on the vector corresponding to each of the four vertices, the target detection value is determined; the vector corresponding to each vertex is a vector in which the coordinates of a preset reference point point to the coordinates of the vertex.

22. The device according to claim 21, characterized in that The target detection value includes a first detection value and a second detection value, and the four vertices of the quadrilateral include, in a clockwise direction or a counterclockwise direction, a first vertex, a second vertex, a third vertex, and a fourth vertex; the processing unit is specifically used for: Perform a cross product of the vector corresponding to the first vertex and the vector corresponding to the fourth vertex, and then perform a dot product with the vector corresponding to the third vertex to obtain a first value; Perform a cross product of the vector corresponding to the second vertex and the vector corresponding to the fourth vertex, and then perform a dot product with the vector corresponding to the third vertex to obtain a second value; taking the ratio of the first value to the second value as a first detection value; Perform a cross product of the vector corresponding to the first vertex and the vector corresponding to the fourth vertex, and then perform a dot product of the vector corresponding to the second vertex to obtain a third value; Perform a cross product of the vector corresponding to the third vertex and the vector corresponding to the fourth vertex, and then perform a dot product of the vector corresponding to the second vertex to obtain a fourth value; The ratio of the third value to the fourth value is used as the second detection value.

23. The device according to any one of claims 20 to 22, characterized in that The processing unit is specifically used for: For each detection value in the target detection value, determining whether the difference between the detection value and a preset threshold value is less than 1; If the difference between at least one of the target detection values ​​and a preset threshold value is less than 1, then the target quadrilateral has at least one set of parallel sides; Otherwise, the target quadrilateral does not have at least one set of parallel sides.

24. The device according to claim 19, characterized in that The processing unit is specifically used for: If the target quadrilateral does not have at least one set of parallel sides, determining a first focal length based on the coordinate information of the target quadrilateral; and determining a focal length of the image based on the first focal length; If the target quadrilateral has at least one set of parallel sides, acquiring parameter information of the image, and determining the focal length of the image based on the parameter information; or determining a default focal length as the focal length of the image; The parameter information includes: the focal length when the image is captured, and / or cropping information for cropping the image.

25. The device according to claim 24, characterized in that The processing unit is specifically used for: Determine a first vector based on the first detection value, the vector corresponding to the first vertex, and the vector corresponding to the second vertex; Determine a second vector based on the second detection value, the vector corresponding to the first vertex, and the vector corresponding to the third vertex; determining a third value based on the first coordinate component and the third coordinate component of the first vector, the first coordinate component and the third coordinate component of the second vector, and the first coordinate component of the center coordinates of the image; determining a fourth value based on the second coordinate component and the third coordinate component of the first vector, the second coordinate component and the third coordinate component of the second vector, and the second coordinate component of the center coordinate of the image; determining the first focal length based on the third value, the fourth value, the first vector, and the second vector; Among them, the coordinate information of the target quadrilateral includes the coordinates of the four vertices of the quadrilateral, and the four vertices include, in sequence, a first vertex, a second vertex, a third vertex and a fourth vertex in a clockwise or counterclockwise direction. The vector corresponding to each vertex is a vector pointing from the coordinates of a preset reference point to the coordinates of the vertex, and the first detection value and the second detection value are both determined based on the vector corresponding to each of the four vertices.

26. The device according to claim 24, characterized in that The processing unit is specifically used for: Determining whether the parameter information includes the cropping information; If the parameter information includes the cropping information, correcting the focal length in the parameter information based on the cropping information to obtain the focal length of the image; If the parameter information does not include the cropping information, the focal length in the parameter information is used as the focal length of the image.

27. The device according to claim 19, characterized in that The processing unit is specifically used for: Determining an aspect ratio of the target object based on the first vector, the second vector and the first matrix; Among them, the coordinate information of the target quadrilateral includes the coordinates of the four vertices of the quadrilateral, and the four vertices include, in sequence in a clockwise or counterclockwise direction: a first vertex, a second vertex, a third vertex and a fourth vertex, and the vector corresponding to each vertex is a vector of the coordinates of a preset reference point pointing to the coordinates of the vertex, and the first vector is determined based on the first detection value, the vector corresponding to the first vertex and the vector corresponding to the second vertex; the second vector is determined based on the second detection value, the vector corresponding to the first vertex and the vector corresponding to the third vertex; the first matrix is ​​determined based on the center coordinates of the image and the focal length of the image; the first detection value and the second detection value are both determined based on the vector corresponding to each of the four vertices; the first matrix is ​​determined based on the focal length and the center coordinates of the image.

28. The device according to any one of claims 24 to 26, characterized in that The processing unit is specifically used for: Based on the first focal length and a preset range, when it is determined that the first focal length is within the preset range, the first focal length is determined as the focal length of the image.

29. The device according to claim 28, characterized in that The processing unit is specifically used for: When it is determined that the first focal length is not within the preset range, parameter information of the image is acquired, and based on the parameter information, the focal length of the image is determined; or, a default focal length is determined as the focal length of the image.

30. The device according to claim 29, characterized in that The processing unit is specifically used for: When it is determined that the first focal length is not within the preset range, the aspect ratio of the target object is determined based on the coordinate information of the target quadrilateral.

31. The device according to claim 28, characterized in that The processing unit is also used for: An upper threshold and a lower threshold of the preset range are determined based on the width of the image and a preset field of view.

32. The device according to claim 31, characterized in that The processing unit is specifically used for: A tangent operation is performed on half of the preset field of view angle, and then multiplied by half of the width of the image to obtain the upper threshold, and the opposite of the upper threshold is used as the lower threshold.

33. The device according to any one of claims 19 to 22, characterized in that The processing unit is specifically used for: If the target quadrilateral has at least one set of parallel sides, the aspect ratio of the target object is determined based on the coordinate information of the target quadrilateral.

34. The device according to claim 33, characterized in that The processing unit is specifically used for: The ratio of the sum of the lengths of any two adjacent side lines of the target quadrilateral to the sum of the lengths of the two diagonal lines of the target quadrilateral is used as the aspect ratio of the target object.

35. The device according to any one of claims 19 to 22, characterized in that The processing unit is also used for: Based on the aspect ratio, a preset aspect ratio and a preset error, when a difference between the aspect ratio and the preset aspect ratio is less than the preset error, determining that the aspect ratio of the target object is the preset aspect ratio.

36. The device according to any one of claims 19 to 22, characterized in that The processing unit is also used for: Identify a target object in the image to obtain text information, wherein the target object includes a plurality of characters; Based on the text information, determining an average aspect ratio of the text; Based on the average aspect ratio of the characters and a preset aspect ratio of the characters, when the difference between the average aspect ratio of the characters and the preset aspect ratio of the characters is greater than a preset value, the aspect ratio of the target object is adjusted according to the average aspect ratio of the characters.

37. An electronic device, characterized in that: include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 18.

38. A computer-readable storage medium, characterized in that: Used to store a computer program, the computer program causing a computer to execute the method according to any one of claims 1 to 18.

39. A computer program product, characterized in that The method comprises computer program instructions which cause a computer to execute the method as claimed in any one of claims 1 to 18.

Images