Image processing method, device, equipment and storage medium
By identifying quadrilaterals in an image and calculating their credibility, the problem of identifying target quadrilaterals in multiple quadrilaterals or blurry situations in an image is solved, achieving accurate image correction and good display.
Patent Information
- Application Number
- CN202080103272.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-10-27
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-10-27
AI Technical Summary
Existing technologies cannot accurately identify target quadrilaterals when the image contains multiple quadrilaterals or when the captured image is blurry, resulting in poor image correction effects.
By identifying quadrilaterals in an image, obtaining their coordinate information, and using a Gaussian function to calculate the credibility of the quadrilaterals, the target quadrilateral is determined based on the credibility, and image correction is performed.
Accurate identification of target quadrilaterals improves the image correction effect, making the target object appear front-facing in the image and providing a better display effect.
Smart Images

Figure CN115885314B_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present application relate to the field of image processing technology, and more specifically, to an image processing method, apparatus, device, and storage medium. Background Art
[0002] As the degree of electronicization continues to deepen, users often need to use electronic devices including mobile phones to capture images of some document objects. However, the captured images show a tilted view effect of the document objects in the images due to the tilt of the electronic device during shooting.
[0003] In this regard, the prior art often identifies a target quadrilateral in an image and corrects the image based on the coordinate information of the target quadrilateral.
[0004] However, when the image contains multiple quadrilaterals or the captured image is blurry, the prior art cannot accurately identify the target quadrilateral, resulting in poor correction effect on the captured image. Summary of the invention
[0005] The embodiments of the present application provide an image processing method, apparatus, device and storage medium.
[0006] In a first aspect, a method for processing an image is provided, comprising:
[0007] Acquire an image to be identified, where the image includes a target object;
[0008] Identify at least one quadrilateral in the image and obtain coordinate information of each quadrilateral;
[0009] For each quadrilateral, the credibility of the quadrilateral is calculated based on the coordinate information of the quadrilateral through the operation of the Gaussian function. The credibility is used to characterize the evaluation result of the quadrilateral as the target area, and the target area is used to characterize the shape of the target object in the image;
[0010] At least one target quadrilateral is determined according to the credibility of each quadrilateral.
[0011] In a second aspect, an image processing device is provided, comprising:
[0012] An image acquisition unit, used for acquiring an image to be identified, the image including a target object;
[0013] An image recognition unit, used to recognize at least one quadrilateral in the image and obtain coordinate information of each quadrilateral;
[0014] An image processing unit is used to calculate the credibility of each quadrilateral according to the coordinate information of the quadrilateral through Gaussian operation, and the credibility is used to characterize the evaluation result of the quadrilateral as a target area, and the target area is used to characterize the shape of the target object in the image;
[0015] The image processing unit is further used to determine at least one target quadrilateral according to the credibility of each quadrilateral.
[0016] According to a third aspect, an electronic device is provided, comprising: a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method in the first aspect or its various implementations.
[0017] According to a fourth aspect, a computer-readable storage medium is provided for storing a computer program, wherein the computer program enables a computer to execute the method according to the first aspect or its various implementations.
[0018] According to a fifth aspect, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions enable a computer to execute the method according to the first aspect or any implementation thereof.
[0019] According to a sixth aspect, a computer program is provided, which enables a computer to execute the method according to the first aspect or its various implementations.
[0020] The technical solution of the first aspect above, by identifying at least one quadrilateral in the image, obtains the coordinate information of each quadrilateral, and realizes the preliminary screening of the quadrilaterals; and based on the coordinate information of each quadrilateral, obtains the credibility of the quadrilateral as the target area through Gaussian operation, and by setting the preset parameters in the Gaussian function, the credibility of the quadrilateral close to the target can be made higher; and then selects one or more quadrilaterals with the highest credibility as the target quadrilateral. The embodiment of the present application can accurately identify the target quadrilateral that can characterize the morphology of the target object, and provides a basis for the subsequent correction of the morphology of the target object to obtain a better presentation effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 A schematic diagram of a process of an image correction scenario 100 provided in an embodiment of the present application;
[0022] Figure 2 A schematic diagram of a framework of an electronic device 200 provided in an embodiment of the present application;
[0023] Figure 3 A schematic diagram of a flow chart of an image processing method 300 provided in an embodiment of the present application;
[0024] Figure 4A schematic diagram of a candidate graph 400 provided in an embodiment of the present application;
[0025] Figure 5 A schematic diagram of a flow chart of an image processing method 500 provided in an embodiment of the present application;
[0026] Figure 6 A schematic diagram of a quadrilateral 600 to be identified provided in an embodiment of the present application;
[0027] Figure 7 A flowchart of an image processing method 700 provided in an embodiment of the present application;
[0028] Figure 8 A schematic block diagram of an image processing device according to an embodiment of the present application is shown;
[0029] Fig. 9 A schematic block diagram of an image processing device according to an embodiment of the present application is shown;
[0030] Fig.10 A schematic block diagram of an image processing device according to an embodiment of the present application is shown;
[0031] Fig.11 It is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0032] The following will describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. For the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0033] At present, users often use mobile phones, tablets, computers, wearable devices and other terminal devices instead of scanners and other devices to collect images of document objects (the same as the target object below), which improves the convenience of obtaining document objects. It should be understood that document objects include but are not limited to documents, business cards, posters, bulletin boards, erasable whiteboards or blackboards, etc. Because it is difficult to control the relative shooting angle between the terminal device and the target object, the document object in the captured image often presents a tilted irregular quadrilateral due to perspective transformation. Therefore, it is necessary to correct the image for the tilt of the document object. For example, by obtaining the tilt degree of the target quadrilateral related to the document object in the image, and correcting the image according to the tilt degree of the target quadrilateral. However, in the actual image acquisition process, if the image acquisition capability of the terminal device is poor, the surrounding light is weak, or there are many interferences around the target object, it is difficult to identify the quadrilateral in the image, or it is difficult to determine the target quadrilateral from the multiple quadrilaterals identified.
[0034] The embodiment of the present application is applied to the above-mentioned scene. In order to make the corrected image have a better display effect and accurately obtain the target quadrilateral in the image, combined with Figure 1 As shown, the image 101 obtained after the target object is captured by the embodiment of the present application includes multiple line segments. By identifying these line segments, quadrilaterals in the image are obtained, for example, quadrilaterals 102 and quadrilaterals 103, and by analyzing the credibility of each quadrilateral as a target area, the target quadrilateral is determined to be quadrilateral 102 based on the credibility of each quadrilateral. It should be understood that the target area is the boundary of the shape presented by the target object in the image. The embodiment of the present application achieves the technical effect of correctly obtaining the target quadrilateral in the image.
[0035] The number of target quadrilaterals may be one or more. For example, when simultaneously capturing images of multiple target objects, a target quadrilateral corresponding to each target object should be identified; or, for the same target object, multiple target quadrilaterals with high credibility may be determined. As an example but not a limitation, a final target quadrilateral may be determined based on multiple target quadrilaterals through algorithm screening, or a target quadrilateral selected by a user from multiple target quadrilaterals may be received as the final target quadrilateral.
[0036] As an example, according to the determined at least one target quadrilateral, a transformation matrix is obtained, and the image is perspectively transformed according to the transformation matrix to obtain Figure 1 As shown in the transformed image 110 , the regular quadrilateral 112 in the transformed image 110 is the effect of the transformation of the quadrilateral 102 . The target object is displayed positively in the image 110 after the transformation of the quadrilateral 112 , which has a good display effect.
[0037] The technical solution of the embodiment of the present application can be applied to various electronic devices to realize at least one of the verification, optimization and testing of the language algorithm model. The electronic device can be a terminal device, such as a mobile phone, a tablet computer, a computer, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a terminal device in industrial control, a terminal device in self-driving, a terminal device in remote medical, a terminal device in a smart city or a terminal device in a smart home, etc. The terminal device in the embodiment of the present application can also be a wearable device, which can also be called a wearable smart device, which is a general term for the intelligent design of daily wearables using wearable technology and the development of wearable devices, such as glasses, gloves, watches, clothing and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. The terminal device can be fixed or mobile.
[0038] Exemplarily, the electronic device in the embodiment of the present application may also be a server. When the electronic device is a server, it can receive an image captured by a terminal device and determine a target quadrilateral in the image.
[0039] Figure 2 Schematic diagram of the structure of an electronic device 200 provided in an embodiment of the present application. Figure 2 As shown, the electronic device 200 includes: an image acquisition unit 210 , an image recognition unit 220 and an image processing unit 230 , and the image processing unit 230 includes at least a credibility calculation subunit 231 .
[0040] The image acquisition unit 210 is used to acquire an image to be recognized, which should contain a document object. For example, an image acquired by an image acquisition device, or an image transmitted by other devices, or an image input by a user can be received, and this embodiment of the application does not limit this.
[0041] The image recognition unit 220 receives the image to be recognized sent by the image acquisition unit 210, and recognizes at least one quadrilateral in the image. Figure 1 As shown, quadrilateral 102 and quadrilateral 103 can be obtained, and further, the identified quadrilateral is sent to the image processing unit 230.
[0042] The image processing unit 230 receives at least one quadrilateral sent by the image recognition unit 220, and determines the credibility of each quadrilateral through the credibility calculation subunit 231. Figure 1 As shown in the example, the reliability of quadrilateral 102 is higher than the reliability of quadrilateral 103 .
[0043] Further, the image processing unit 230 determines a quadrilateral with the highest credibility as the target quadrilateral according to the credibility of each quadrilateral, or after determining the credibility ranking, selects n quadrilaterals starting from the one with the highest credibility as the target quadrilateral.
[0044] As an example, the image processing unit 230 performs subsequent operations based on the determined at least one target quadrilateral. For example, at least one target quadrilateral is sent to an image display unit (not shown in the figure) for display, and the user can determine the recognition effect of the electronic device on the quadrilateral recognition based on the displayed at least one target quadrilateral, or the user can select a target quadrilateral as the final target quadrilateral through the human-computer interaction interface; or, based on at least one target quadrilateral, determine the transformation matrix, and perform perspective transformation on the image based on the transformation matrix. Exemplarily, when the number of target quadrilaterals is two or more, the image processing unit 230 selects a final target quadrilateral from two or more target quadrilaterals, or obtains the final target quadrilateral by weighted averaging two or more target quadrilaterals.
[0045] The present application is described in detail below through several embodiments.
[0046] Figure 3 A flowchart of an image processing method 300 provided in an embodiment of the present application.
[0047] In order to ensure that a quadrilateral that is closest to the inclination of the target object is obtained, the embodiment of the present application confirms the credibility of the quadrilateral as the target area and obtains the target quadrilateral based on the credibility.
[0048] like Figure 3 As shown, the image processing method includes:
[0049] S301: Acquire an image to be recognized.
[0050] It should be understood that the electronic device can capture images of the target object through an image capture device to obtain an image to be recognized, or can receive images input by a user, or receive images sent by other devices.
[0051] Exemplarily, after acquiring the image to be recognized, the image may be preprocessed to facilitate subsequent recognition operations.
[0052] For example, the image is converted in color space. The images collected by the image acquisition device are mostly RGB images, that is, color images formed by changing and superimposing the three color channels of red R (Red), green G (Green), and blue B (Blue). In order to facilitate the subsequent recognition operation of the image and improve the processing efficiency, the RGB image can be converted in color space in advance, such as converted into a grayscale image or an HSI image, etc. The HSI image is an image characterized by hue H (Hue), saturation S (Saturation) and brightness I (Intensity).
[0053] Optionally, preprocessing the image to be recognized further includes: performing edge detection on the image by using any algorithm, such as a Canny algorithm or a Holistically-Nested Edge Detection (HED) algorithm.
[0054] Optionally, preprocessing the image to be recognized further includes: scaling the image to a preset size.
[0055] It should be understood that this embodiment does not limit the execution order of any of the above preprocessing processes.
[0056] S302: Identify at least one quadrilateral in the image and obtain coordinate information of each quadrilateral.
[0057] The coordinate information of the quadrilateral includes the coordinates of the four vertices of the quadrilateral.
[0058] Exemplarily, in the process of identifying a quadrilateral in an image, multiple line segments in the image are first identified, and every four line segments in the multiple line segments are combined into a candidate shape to obtain all possible candidate shapes, and then each candidate shape is analyzed to see whether it is the quadrilateral to be identified.
[0059] Figure 4 A schematic diagram of an image 400 to be identified provided in an embodiment of the present application. Exemplarily, in this embodiment, determining whether a candidate graphic satisfies a first preset condition, when the candidate graphic satisfies the first preset condition, the candidate graphic is a quadrilateral to be identified, specifically includes the following possible implementations:
[0060] 1. The first preset condition is that two first angles of the candidate figure are both smaller than a first preset value, and the first angle is the angle between any two non-adjacent line segments among the four line segments.
[0061] Determine whether the two first angles of the candidate figure are less than a first preset value, if both first angles are less than the first preset value, the candidate figure is a quadrilateral to be identified. It should be understood that the first angle is the angle of the two angles of the two line segments that is less than 90 degrees.
[0062] Combination Figure 4 As shown, assuming that the candidate figure is a candidate figure composed of line segments AB, BC, CD and AD, it is determined whether the angle between line segment AB and line segment CD is less than a first preset value, and whether the angle between line segment AD and line segment BC is less than the first preset value. If the angle between line segment AB and line segment CD, and the angle between line segment AD and line segment BC are both less than the first preset value, then the candidate figure is the quadrilateral to be identified.
[0063] Optionally, the first preset value is 30 degrees.
[0064] 2. The first preset condition is that the four second angles of the candidate figure are all greater than the second preset value, and the second angle is the angle between any two adjacent line segments of the four line segments. Optionally, the first preset condition can be that the four second angles of the candidate figure are all less than the difference between 180 degrees and the second preset value.
[0065] Combination Figure 4 As shown, assuming that the candidate graph is a candidate graph composed of line segments AB, BC, CD and AD, it is determined whether ∠A, ∠B, ∠C and ∠D are all greater than the second preset value. Exemplarily, the second preset value is 60 degrees, and it is determined whether ∠A, ∠B, ∠C and ∠D are all greater than 60 degrees, that is, it is determined whether ∠A, ∠B, ∠C and ∠D are all less than 120 degrees.
[0066] When ∠A, ∠B, ∠C and ∠D are all greater than the second preset value, the candidate figure is a quadrilateral to be identified.
[0067] 3. The first preset condition is that the area ratio of the candidate graphic is greater than the fourth preset value, and the area ratio is the ratio of the area of the candidate graphic to the area of the image.
[0068] Optionally, the fourth preset value is one sixth.
[0069] Exemplarily, the first preset condition may also be a combination of any two or three of the above three examples. For example, when the two first angles of the candidate figure are both smaller than the first preset value, and the four second angles of the candidate figure are both larger than the second preset value, the candidate figure is a quadrilateral to be identified; or when the two first angles of the candidate figure are both smaller than the first preset value, and the four second angles of the candidate figure are both larger than the second preset value, and the area ratio of the candidate figure is larger than the fourth preset value, the candidate figure is a quadrilateral to be identified.
[0070] Optionally, for a candidate graph consisting of four unclosed line segments, for example Figure 4 The candidate figure for the upper left corner of quadrilateral ABCD in the figure determines the endpoints of the two line segments that are not closed (such as Figure 4Whether the distance between the endpoints V and W in the two line segments is less than the preset distance, when the distance between the endpoints of the two line segments is less than the preset distance, determine whether the candidate figure is the quadrilateral to be identified according to any of the above embodiments, otherwise determine that the candidate figure is not the quadrilateral to be identified.
[0071] Exemplarily, after determining that the candidate figure is a quadrilateral to be identified, the coordinates of the four vertices of the quadrilateral are determined, that is, Figure 4 The coordinates of the four points A, B, C, and D.
[0072] The quadrilateral obtained after screening through any of the above examples is closer to the target quadrilateral to be obtained, which narrows the range for subsequent determination of the target quadrilateral and improves processing efficiency.
[0073] As an example, in order to avoid interference from noise and the like, before every four line segments of the plurality of line segments are combined in the above embodiment to obtain at least one candidate graph, the identified line segments may be optimized.
[0074] Exemplarily, it is determined whether there are at least two line segments in the same straight line among the multiple line segments identified, and the at least two line segments in the same straight line are merged. Optionally, through the coordinate information of each line segment, it is possible to identify that at least two line segments in the same straight line have the following possible positional relationships: one, there is partial overlap, two, one line segment is covered by another line segment, and three, the line segments are separated and the distance is less than a fourth preset value, which is the distance between the closest endpoints of the two line segments. The embodiment of the present application merges the line segments in the same straight line that meet the above three possible positional relationships, for example, the overlapping parts of the partially overlapping line segments are merged, the line segments covered by other line segments are deleted, and the line segments whose line segment distance is less than the fourth preset value are connected to another line segment by translating one line segment, or the position between the line segments is supplemented to achieve the merging of the line segments.
[0075] S303: For each quadrilateral, the credibility of the quadrilateral is calculated through Gaussian operation according to the coordinate information of the quadrilateral.
[0076] The credibility is used to characterize the evaluation result of the quadrilateral as the target area, and the target area is used to characterize the shape of the target object in the image.
[0077] In this step, an evaluation is performed based on whether the quadrilateral can be used as a target area to obtain the credibility of the quadrilateral. Exemplarily, at least one feature of the quadrilateral can be determined based on the coordinate information of the quadrilateral, and based on the at least one feature of the quadrilateral, a score of the quadrilateral as a target area is determined by Gaussian operation, that is, the credibility of the quadrilateral is obtained.
[0078] Optionally, the feature of the quadrilateral includes at least one of an area ratio feature, a first vertical angle feature, a second vertical angle feature, an adjacent angle feature, or an intermediate position feature. The area ratio feature is used to characterize the ratio of the area of the quadrilateral to the area of the image; the first vertical angle feature is used to characterize the angle between the midline of the quadrilateral and the vertical line of the image; the second vertical angle feature is used to characterize the angle between the midline of the quadrilateral and the projected gravity vector in the image; the adjacent angle feature is used to characterize the angular relationship between adjacent angles of the quadrilateral; and the intermediate position feature is used to characterize the coordinates of the intermediate position of the quadrilateral.
[0079] It should be understood that the Gaussian function used in the Gaussian operation process contains three preset parameters: weight, mean value and variance. The mean value is used to characterize the eigenvalue that the corresponding feature is expected to reach. The closer the eigenvalue is to the mean value, the higher the calculated credibility. For example, when the ratio of the area of the quadrilateral to the area of the image is 0.8, it is most likely the target area, and the mean value is preset to 0.8.
[0080] S304: Determine at least one target quadrilateral according to the credibility of each quadrilateral.
[0081] In this step, at least one identified quadrilateral is screened according to the credibility of the quadrilateral to obtain at least one target quadrilateral.
[0082] Possible implementation method 1: sort at least one quadrilateral according to the credibility of each quadrilateral, and use the n quadrilaterals with the highest credibility as at least one target quadrilateral, where n≥1. It should be understood that the credibility of the n quadrilaterals can be the same or different, and the credibility of the n quadrilaterals is higher than the credibility of other quadrilaterals that are not selected as target quadrilaterals.
[0083] Possible implementation manner 2: determining whether the credibility of each quadrilateral in at least one quadrilateral is greater than a preset threshold, and determining the quadrilateral whose credibility is greater than the preset threshold as the target quadrilateral.
[0084] In the embodiment of the present application, at least one quadrilateral in the image is identified to obtain the coordinate information of each quadrilateral, thereby achieving preliminary screening of the quadrilaterals; and based on the coordinate information of each quadrilateral, a Gaussian operation is performed to obtain the credibility of the quadrilateral as a target area, and by setting the preset parameters in the Gaussian function, the credibility of the quadrilateral close to the target quadrilateral can be made higher; and then one or more quadrilaterals with the highest credibility are selected as the target quadrilateral. The embodiment of the present application can accurately identify the target quadrilateral that can characterize the morphology of the target object, providing a basis for subsequent correction of the morphology of the target object to obtain a better presentation effect.
[0085] On the basis of the above embodiments, in order to accurately evaluate the credibility of the quadrilateral as the target area, the embodiment of the present application can evaluate the credibility of each feature of the quadrilateral, obtain the credibility corresponding to each feature, and then sum the credibility of each feature to obtain the credibility of the quadrilateral.
[0086] Figure 5 The flowchart of an image processing method 500 provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, based on the area ratio feature, the first vertical angle feature, the adjacent angle feature and the middle position feature of the quadrilateral and the preset parameters corresponding to each feature, the credibility g corresponding to each feature is obtained respectively, and the credibility g corresponding to each feature is summed up to obtain the credibility c of the quadrilateral.
[0087] It should be noted that, for the sake of clarity, in this embodiment, the preset parameter corresponding to the area ratio feature is called the first preset parameter, the preset parameter corresponding to the first vertical angle feature is called the second preset parameter, the preset parameter corresponding to the adjacent angle feature is called the third preset parameter, and the preset parameter corresponding to the intermediate position feature is called the fourth preset parameter.
[0088] Exemplarily, the credibility g_1 corresponding to the area ratio feature is calculated: the area of the quadrilateral is determined according to the coordinate information of the quadrilateral, and the ratio of the area of the quadrilateral to the area of the image is used as the characteristic value of the area ratio feature, and then the credibility corresponding to the area ratio feature is calculated according to the Gaussian function. Figure 6 A schematic diagram of an image 600 to be identified provided in an embodiment of the present application. Figure 6 As shown, according to the coordinates of the four vertices of the quadrilateral, namely A, B, C, D and the pixel values of the image, namely Figure 6 The length h and width w of the image in the formula Calculate the characteristic value x of the area ratio feature 1 , where AC and BD are the vectors of the diagonals of the quadrilateral.
[0089] Furthermore, according to the calculated x 1 , through the corresponding Gaussian function Get the credibility g corresponding to the area ratio feature 1 , where α 1 , μ 1 and σ 1 is the first preset parameter corresponding to the area ratio feature, α 1 Used to characterize the credibility g corresponding to the area ratio feature 1 The weight, μ 1 is the expected value of the mean value used to characterize the area ratio, σ 1 is the variance.
[0090] Optional, α 1 =1,μ 1 =0.7,σ 1 =0.2.
[0091] Exemplarily, the credibility g corresponding to the first vertical angle feature is calculated 2 :According to the coordinate information of the quadrilateral, the vectors of the two midlines of the quadrilateral are determined, and the angle between the vector of each midline and the vertical line is determined respectively, and the smallest angle of the two angles is used as the eigenvalue of the first vertical angle feature. It should be noted that the midline of the quadrilateral is the opposite side of the quadrilateral, that is, the connecting line of the midpoints of two non-adjacent sidelines, and the vertical line is a straight line perpendicular to the horizontal sideline of the image. The horizontal sideline refers to the sideline in the horizontal direction when the image is placed in the positive direction.
[0092] Combination Figure 6 As shown, N is the midpoint of line segment AB, M is the midpoint of line segment CD, P is the midpoint of line segment BC, Q is the midpoint of line segment AD, and the two medians of quadrilateral ABCD are MN and PQ. It should be understood that vector MN can be obtained by It can be obtained that the vector PQ can be obtained by Get, and based on the formula Get the first vertical angle feature x 2 , where x=0 represents the y-axis in the two-dimensional coordinate system of the image, that is, the vertical line perpendicular to the horizontal edge of the image. express Figure 6 ∠b in express Figure 6 ∠a in.
[0093] Furthermore, according to the calculated x 2 , through the corresponding Gaussian function Get the credibility g corresponding to the first vertical angle feature 2 , where α 2 , μ 2 and σ 2 is the second preset parameter corresponding to the first vertical angle feature, α 2 Used to characterize the credibility g corresponding to the first vertical angle feature 2 The weight, μ 2 is the average value used to characterize the expected value of the first vertical angle, σ 2 is the variance.
[0094] Exemplarily, the credibility g corresponding to the adjacent angle features is calculated 3 :According to the coordinate information of the quadrilateral, determine the angles of the four vertices of the quadrilateral, combined with Figure 6 As shown, assuming ∠A=θ 1 , ∠B=θ2 , ∠C=θ 3 , ∠D=θ 4 , further, perform a difference operation on the vertex angles of every two vectors of the quadrilateral to obtain four adjacent angle differences, and then sum the four adjacent angle differences to obtain the eigenvalue x of the adjacent angle feature 3 , that is, x 3 =|θ 1 -θ 2 |+|θ 2 -θ 3 |+|θ 3 -θ 4 |+|θ 4 -θ 1 |.
[0095] Furthermore, according to the calculated x 3 , through the corresponding Gaussian function Get the credibility g corresponding to the adjacent angle features 3 , where α 3 , μ 3 and σ 3 is the third preset parameter corresponding to the adjacent angle feature, α 3 Used to characterize the credibility of the vector angle feature corresponding to g 3 The weight, μ 3 is the average value used to characterize the expected value of adjacent angles, σ 3 is the variance.
[0096] For example, the reliability g corresponding to the intermediate position feature is calculated 4 :According to the coordinate information of the quadrilateral, determine the coordinates of the middle position of the quadrilateral. It should be noted that the coordinates of the middle position are the average of the coordinates of the four vertices of the quadrilateral. Figure 6 As shown, the eigenvalue of the middle position feature, that is, the coordinates of the middle position of the quadrilateral
[0097] Furthermore, according to the calculated x 4 , through the corresponding Gaussian function Get the credibility g corresponding to the middle position feature 4 , where α 4 , μ 4 and σ 4 is the fourth preset parameter corresponding to the middle position feature, α 4 Used to characterize the credibility of the vector angle feature corresponding to g 4 The weight, μ 4 is the average value used to characterize the expected value of the middle position, σ 4 is the variance.
[0098] By the formula c = Σg i (x i ,α i ,μ i ,σ i ), i=1, 2, 3, 4, and the credibility of the above features is summed to obtain the credibility c of the quadrilateral. It should be understood that the features of the quadrilateral listed in this embodiment are only examples, and more or fewer features may be included in actual applications.
[0099] In a possible implementation manner, the calculation of the credibility in any of the above embodiments may also be combined with the credibility of the second vertical angle feature.
[0100] Exemplarily, the credibility g of the second vertical angle feature is calculated 5 : Obtain the posture information of the electronic device when capturing an image, the posture information includes position information and / or posture information, for example, directly read the posture information through the information carried by the image, and then c And formula g p =P(R c ,K,g 0 ), calculate the projected gravity vector g p , where K is the camera internal parameter, g 0 is the gravity vector in the world coordinate system, P is the projection function; further, according to the formula in, and They represent the angles between the two midline vectors of the quadrilateral and the projected gravity vector respectively, and the minimum value of the two angles is taken as the eigenvalue of the second vertical angle feature.
[0101] Furthermore, according to the calculated x 5 , through the corresponding Gaussian function Get the credibility g corresponding to the second vertical angle feature 5 , where α 5 , μ 5 and σ 5 The preset parameter corresponding to the second vertical angle feature is also called the fifth preset parameter, α 5 Used to characterize the credibility of the vector angle feature corresponding to g 5 The weight, μ 5 is the average value used to characterize the expected value of the middle position, σ 5 is the variance.
[0102] Figure 7 A flowchart of an image processing method 700 provided in an embodiment of the present application.
[0103] The shooting scene of the image can reflect the type and characteristics of the target object to a certain extent. Therefore, setting different parameters for the Gaussian function for different shooting scenes can improve the accuracy of quadrilateral recognition.
[0104] For example, in an art gallery scene, users view paintings at a relatively close distance, so the target object captured has a larger area ratio in the image, and paintings are generally in a hanging state, so the target object captured generally appears as a trapezoid in the image, and accordingly, the preset parameters corresponding to the area ratio feature and the preset parameters corresponding to the first vertical angle feature are set.
[0105] Optional shooting scenes include but are not limited to: art galleries, office buildings, and schools.
[0106] Exemplarily, first, the positioning information when the image was captured is obtained, and the shooting scene of the image is determined based on the positioning information. For example, the shooting scene is predicted based on the positioning information and preset map information, and then the parameter group corresponding to the predicted shooting scene is determined based on the correspondence between the determined shooting scene and the pre-set shooting scene and the parameter group. The parameter group includes preset parameters corresponding to one or more features in any of the above embodiments, and then the credibility of the quadrilateral is calculated based on the determined parameter group.
[0107] Exemplarily, during the image acquisition process, the positioning information of the electronic device is obtained through any positioning technology, for example, the positioning information of the electronic device is obtained through the Global Positioning System (GPS).
[0108] In addition to determining the shooting scene based on the positioning information, the embodiments of the present application can determine the shooting scene based on the user's intention, for example, receiving the shooting scene selected by the user through the human-computer interaction interface, and then determining the corresponding parameter group based on the shooting scene.
[0109] Optionally, a setting instruction input by a user may be received, the setting instruction including a parameter group identifier, and a corresponding parameter group may be determined according to the parameter group identifier indicated by the setting instruction.
[0110] Combination of the above Figures 1 to 7 , describes in detail the method embodiment of the present application, and the following is combined with Figures 8 to 10 , the device embodiments of the present application are described in detail. It should be understood that the device embodiments and the method embodiments correspond to each other, and similar descriptions can refer to the method embodiments.
[0111] Figure 8 FIG. 1 is a schematic block diagram of an image processing device according to an embodiment of the present application. Figure 8 As shown, the image processing device 10 includes:
[0112] An image acquisition unit 11 is used to acquire an image to be identified, where the image includes a target object;
[0113] An image recognition unit 12, used to recognize at least one quadrilateral in the image and obtain coordinate information of each quadrilateral;
[0114] The image processing unit 13 is used to calculate the credibility of each quadrilateral according to the coordinate information of the quadrilateral through Gaussian operation, and the credibility is used to characterize the evaluation result of the quadrilateral as a target area, and the target area is used to characterize the shape of the target object in the image;
[0115] The image processing unit 13 is further configured to determine at least one target quadrilateral according to the credibility of each quadrilateral.
[0116] The image processing device 10 in the embodiment of the present application includes an image acquisition unit 11 and an image recognition unit 12. By recognizing at least one quadrilateral in the image, the coordinate information of each quadrilateral is obtained to achieve preliminary screening of the quadrilaterals; and based on the coordinate information of each quadrilateral, the credibility of the quadrilateral as a target area is obtained through Gaussian operation. By setting the preset parameters in the Gaussian function, the credibility of the quadrilateral close to the target quadrilateral can be made higher; and then one or more quadrilaterals with the highest credibility are selected as the target quadrilateral. The embodiment of the present application can accurately identify the target quadrilateral that can characterize the morphology of the target object, which provides a basis for the subsequent correction of the morphology of the target object to obtain a better presentation effect.
[0117] Optionally, the image processing unit 13 is specifically used for:
[0118] For each feature of the quadrilateral, determining the credibility of the feature according to coordinate information of the quadrilateral and a Gaussian function corresponding to the feature, the quadrilateral including at least one feature;
[0119] The credibility of at least one feature is summed to obtain the credibility of the quadrilateral.
[0120] Optionally, the image processing unit 13 is specifically used for:
[0121] For each feature of the quadrilateral, determine the feature value of the feature according to the coordinate information of the quadrilateral;
[0122] The credibility of the feature is obtained according to the feature value and the Gaussian function, and the Gaussian function includes preset parameters corresponding to the feature.
[0123] Optionally, the image processing unit 13 is specifically used for:
[0124] Determine the area of the quadrilateral according to the coordinate information of the quadrilateral;
[0125] The ratio of the area of the quadrilateral to the area of the image is used as the feature value of the feature.
[0126] Optionally, the image processing unit 13 is specifically used for:
[0127] According to the coordinate information of the quadrilateral, the vectors of the two midlines of the quadrilateral are obtained;
[0128] Determine the angle between the vector of each midline and the vertical line, which is perpendicular to the horizontal edge of the image;
[0129] The smallest of the two angles is taken as the eigenvalue of the feature.
[0130] Optionally, the image processing unit 13 is specifically used for:
[0131] According to the coordinate information of the quadrilateral, determine the four vertex angles of the quadrilateral;
[0132] Perform a difference operation on every two adjacent vertex angles of the quadrilateral to obtain four adjacent angle differences;
[0133] The sum of the four adjacent angle differences is taken as the eigenvalue of the feature.
[0134] Optionally, the image processing unit 13 is specifically used for:
[0135] According to the coordinate information of the quadrilateral, the coordinates of the middle position of the quadrilateral are determined, and the coordinates of the middle position are used as the characteristic value of the feature; the coordinates of the middle position are the average value of the coordinates of the four vertices of the quadrilateral, and the coordinate information includes the coordinates of the four vertices.
[0136] Optionally, preset parameters include weight, mean and variance.
[0137] Optionally, the image processing unit 13 is specifically used for:
[0138] Acquire position and posture information, where the position and posture information is used to represent the position and / or posture of the electronic device when capturing an image;
[0139] The feature value of the feature is determined based on the pose information and the coordinate information of the quadrilateral.
[0140] Optionally, the image processing unit 13 is specifically used for:
[0141] According to the posture information and the projection function, a projection gravity vector is determined, where the projection gravity vector is used to represent the projection of the gravity vector in the world coordinate system in the coordinate system where the image is located;
[0142] According to the coordinate information of the quadrilateral, the vectors of the two midlines of the quadrilateral are obtained;
[0143] Determine the angle between the vector of each midline of the quadrilateral and the projected gravity vector;
[0144] The smallest of the two angles is taken as the eigenvalue of the feature.
[0145] The data processing device provided in the above embodiment can execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0146] exist Figure 8 Based on the embodiment shown, Fig. 9 FIG. 1 is a schematic block diagram of an image processing device according to an embodiment of the present application. Fig. 9 As shown, the image processing device 10 further includes:
[0147] A position acquisition unit 14, used to acquire positioning information, where the positioning information is used to represent the position of the electronic device when capturing an image;
[0148] A parameter determination unit 15, used to determine the shooting scene of the image according to the positioning information;
[0149] The parameter determination unit 15 is further used to determine a corresponding parameter group according to the shooting scene, where the parameter group includes a preset parameter corresponding to at least one feature.
[0150] The data processing device provided in the above embodiment can execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0151] exist Figure 8 or Fig. 9 Based on the embodiment shown, Fig.10 FIG. 1 is a schematic block diagram of an image processing device according to an embodiment of the present application. Fig.10 As shown, the image processing device 10 further includes: a receiving unit 16;
[0152] The receiving unit 16 is used to receive a shooting scene input by a user;
[0153] The parameter determination unit 15 is further used to determine a corresponding parameter group according to the shooting scene, where the parameter group includes at least one preset parameter corresponding to a feature.
[0154] Optionally, the image recognition unit 12 is specifically used for:
[0155] Identify and obtain at least one candidate figure in the image, where the candidate figure consists of four line segments;
[0156] For each candidate figure, when the candidate figure meets the first preset condition, the candidate figure is determined to be a quadrilateral, and coordinate information of the quadrilateral is obtained.
[0157] Optionally, the first preset condition includes at least one of the following:
[0158] The two first angles of the candidate graph are both smaller than the first preset value, and the first angle is the angle between any two non-adjacent line segments of the four line segments;
[0159] or,
[0160] The four second angles of the candidate graph are all greater than the second preset value, and the second angle is the angle between any two adjacent line segments of the four line segments;
[0161] or,
[0162] The area ratio of the candidate graphic is greater than a third preset value, and the area ratio is a ratio of the area of the candidate graphic to the area of the image.
[0163] Optionally, the image recognition unit 12 is specifically used for:
[0164] Recognize multiple line segments in the image;
[0165] Every four line segments of the plurality of line segments are combined to obtain at least one candidate graph.
[0166] Optionally, the image recognition unit 12 is specifically used for:
[0167] Determine at least two line segments that are on the same straight line among the plurality of line segments;
[0168] For each two line segments of the at least two line segments, when the two line segments meet a second preset condition, the two line segments are merged.
[0169] Optionally, the second preset condition includes:
[0170] There is some overlap between the two line segments;
[0171] Or, either of the two line segments is covered by the other line segment;
[0172] Alternatively, the distance between the two line segments is smaller than a fourth preset value.
[0173] Optionally, the image processing unit 13 is specifically used for:
[0174] For at least one quadrilateral, sort the quadrilaterals according to their credibility, and use the n quadrilaterals with the highest credibility as at least one target quadrilateral, where n≥1;
[0175] or,
[0176] For each quadrilateral of the at least one quadrilateral, when the credibility of the quadrilateral is greater than a preset threshold, the quadrilateral is determined to be a target quadrilateral.
[0177] The data processing device provided in the above embodiment can execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.
[0178] Fig.11 It is a schematic structural diagram of an electronic device provided in an embodiment of the present application. Fig.11 The electronic device shown includes a processor 1210, which can call and run a computer program from a memory to implement the method in the embodiment of the present application.
[0179] Alternatively, if Fig.11 As shown, the electronic device 1200 may further include a memory 1220. The processor 1210 may call and run a computer program from the memory 1220 to implement the method in the embodiment of the present application.
[0180] The memory 1220 may be a separate device independent of the processor 1210 , or may be integrated into the processor 1210 .
[0181] Alternatively, if Fig.11 As shown, the electronic device 1200 may further include a transceiver 1230, and the processor 1210 may control the transceiver 1230 to communicate with other devices, specifically, may send information or data to other devices, or receive information or data sent by other devices.
[0182] The transceiver 1230 may include a transmitter and a receiver. The transceiver 1230 may further include an antenna, and the number of the antennas may be one or more.
[0183] Optionally, the electronic device 1200 can implement the corresponding processes in the various methods of the embodiments of the present application, which will not be described here for the sake of brevity.
[0184] It should be understood that the processor of the embodiment of the present application may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and performed. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, and other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.
[0185] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0186] It should be understood that the above-mentioned memory is exemplary but not restrictive. For example, the memory in the embodiments of the present application may also be static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synch link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM), etc. That is to say, the memory in the embodiments of the present application is intended to include but not limited to these and any other suitable types of memory.
[0187] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.
[0188] Optionally, the computer-readable storage medium can be applied to the electronic device in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the electronic device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.
[0189] An embodiment of the present application also provides a computer program product, including computer program instructions.
[0190] Optionally, the computer program product can be applied to the electronic device in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes implemented by the electronic device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.
[0191] The embodiment of the present application also provides a computer program.
[0192] Optionally, the computer program can be applied to the electronic device in the embodiments of the present application. When the computer program runs on a computer, the computer executes the corresponding processes implemented by the electronic device in the various methods of the embodiments of the present application. For the sake of brevity, they are not described here.
[0193] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0194] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0195] In the several embodiments provided in the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0196] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0197] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0198] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. In view of such an understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.
[0199] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A method for processing an image, It is characterized in that include: Acquire an image to be identified, wherein the image includes a target object; Identify at least one quadrilateral in the image to obtain coordinate information of each quadrilateral; For each quadrilateral, according to the coordinate information of the quadrilateral, the credibility of the quadrilateral is calculated by Gaussian operation, and the credibility is used to characterize the evaluation result of the quadrilateral as a target area, and the target area is used to characterize the shape of the target object in the image; Determine at least one target quadrilateral according to the credibility of each quadrilateral; The process of calculating the credibility of each quadrilateral according to the coordinate information of the quadrilateral by Gaussian operation includes: For each feature of the quadrilateral, determining the credibility of the feature according to coordinate information of the quadrilateral and a Gaussian function corresponding to the feature, the quadrilateral including at least one feature; The credibility of the at least one feature is summed to obtain the credibility of the quadrilateral, wherein the features include an area ratio feature, a first vertical angle feature, an adjacent angle feature, and an intermediate position feature of the quadrilateral.
2. The method according to claim 1, It is characterized in that The determining, for each feature of the quadrilateral, the credibility of the feature according to the coordinate information of the quadrilateral and a Gaussian function corresponding to the feature, includes: Determining a feature value of the feature according to the coordinate information of the quadrilateral; The credibility of the feature is obtained according to the feature value and the Gaussian function, wherein the Gaussian function includes preset parameters corresponding to the feature.
3. The method according to claim 2, It is characterized in that If the feature includes an area ratio feature, determining a feature value of the feature according to the coordinate information of the quadrilateral includes: Determine the area of the quadrilateral according to the coordinate information of the quadrilateral; The ratio of the area of the quadrilateral to the area of the image is used as the feature value of the feature.
4. The method according to claim 2, It is characterized in that If the feature includes a first vertical angle feature, determining a feature value of the feature according to the coordinate information of the quadrilateral includes: According to the coordinate information of the quadrilateral, vectors of two midlines of the quadrilateral are obtained; Determine the angle between the vector of each center line and a vertical line, respectively, wherein the vertical line is perpendicular to the horizontal side line of the image; The smallest of the two angles is taken as the characteristic value of the feature.
5. The method according to claim 2, It is characterized in that If the feature includes an adjacent angle feature, determining a feature value of the feature according to the coordinate information of the quadrilateral includes: Determine four vertex angles of the quadrilateral according to the coordinate information of the quadrilateral; Performing a difference operation on every two adjacent vertex angles of the quadrilateral to obtain four adjacent angle differences; The sum of four adjacent angle differences is taken as the feature value of the feature.
6. The method according to claim 2, It is characterized in that If the feature includes an intermediate position feature, determining a feature value of the feature according to the coordinate information of the quadrilateral includes: According to the coordinate information of the quadrilateral, the coordinates of the middle position of the quadrilateral are determined, and the coordinates of the middle position are used as the feature value of the feature; the coordinates of the middle position are the average value of the coordinates of the four vertices of the quadrilateral, and the coordinate information includes the coordinates of the four vertices.
7. The method according to any one of claims 2 to 6, It is characterized in that The preset parameters include weight, mean value and variance.
8. The method according to claim 2, It is characterized in that If the feature includes a second vertical angle feature, determining a feature value of the feature according to the coordinate information of the quadrilateral includes: Acquire position and posture information, where the position and posture information is used to represent the position and / or posture of the electronic device when capturing the image; A feature value of the feature is determined according to the pose information and the coordinate information of the quadrilateral.
9. The method according to claim 8, It is characterized in that The step of determining the feature value of the feature according to the pose information and the coordinate information of the quadrilateral includes: Determine a projected gravity vector according to the posture information and the projection function, wherein the projected gravity vector is used to represent the projection of the gravity vector in the world coordinate system in the coordinate system where the image is located; According to the coordinate information of the quadrilateral, vectors of two midlines of the quadrilateral are obtained; respectively determining the angle between the vector of each midline of the quadrilateral and the projected gravity vector; The smallest of the two angles is taken as the characteristic value of the feature.
10. The method according to any one of claims 2 to 6, It is characterized in that The method further comprises: Acquiring positioning information, where the positioning information is used to represent the position of the electronic device when the image is captured; Determining a shooting scene of the image according to the positioning information; According to the shooting scene, a corresponding parameter group is determined, where the parameter group includes a preset parameter corresponding to at least one feature.
11. The method according to any one of claims 2 to 6, It is characterized in that The method further comprises: Receive a shooting scene input by a user; According to the shooting scene, a corresponding parameter group is determined, where the parameter group includes a preset parameter corresponding to at least one feature.
12. The method according to any one of claims 1 to 6, It is characterized in that The step of identifying at least one quadrilateral in the image to obtain coordinate information of each quadrilateral includes: Identify and obtain at least one candidate figure in the image, wherein the candidate figure consists of four line segments; For each candidate figure, when the candidate figure satisfies a first preset condition, the candidate figure is determined to be the quadrilateral, and coordinate information of the quadrilateral is obtained.
13. The method according to claim 12, It is characterized in that The first preset condition includes at least one of the following: Two first angles of the candidate graph are both smaller than a first preset value, and the first angle is the angle between any two non-adjacent line segments of the four line segments; or, The four second angles of the candidate graph are all greater than a second preset value, and the second angle is the angle between any two adjacent line segments of the four line segments; or, The area ratio of the candidate graphic is greater than a third preset value, and the area ratio is a ratio of the area of the candidate graphic to the area of the image.
14. The method according to claim 12, It is characterized in that The identifying and obtaining at least one candidate graphic in the image includes: Identify and obtain multiple line segments in the image; Every four line segments of the plurality of line segments are combined to obtain the at least one candidate graph.
15. The method according to claim 14, It is characterized in that Before combining every four line segments of the plurality of line segments to obtain the at least one candidate graph, the method further includes: Determine at least two line segments in the plurality of line segments that are on the same straight line; For each two line segments of the at least two line segments, when the two line segments meet a second preset condition, the two line segments are merged.
16. The method according to claim 15, It is characterized in that The second preset condition includes: The two line segments partially overlap; Alternatively, any one of the two line segments is covered by the other line segment; Alternatively, the distance between the two line segments is smaller than a fourth preset value.
17. The method according to any one of claims 1 to 6, It is characterized in that The step of determining at least one target quadrilateral according to the credibility of each quadrilateral comprises: The at least one quadrilateral is sorted according to the credibility of each quadrilateral, and the n quadrilaterals with the highest credibility are used as the at least one target quadrilateral, where n≥1; or, For each quadrilateral of the at least one quadrilateral, when the credibility of the quadrilateral is greater than a preset threshold, the quadrilateral is determined to be a target quadrilateral.
18. An image processing device, It is characterized in that include: An image acquisition unit, used for acquiring an image to be identified, wherein the image includes a target object; An image recognition unit, used to recognize at least one quadrilateral in the image and obtain coordinate information of each quadrilateral; An image processing unit, configured to calculate, for each quadrilateral, the credibility of the quadrilateral according to the coordinate information of the quadrilateral through Gaussian operation, wherein the credibility is used to characterize the evaluation result of the quadrilateral as a target area, and the target area is used to characterize the shape of the target object in the image; The image processing unit is further used to determine at least one target quadrilateral according to the credibility of each quadrilateral; The image processing unit is specifically used for: For each feature of the quadrilateral, determining the credibility of the feature according to coordinate information of the quadrilateral and a Gaussian function corresponding to the feature, the quadrilateral including at least one feature; The credibility of the at least one feature is summed to obtain the credibility of the quadrilateral, wherein the features include an area ratio feature, a first vertical angle feature, an adjacent angle feature, and an intermediate position feature of the quadrilateral.
19. The device according to claim 18, It is characterized in that The image processing unit is specifically used for: Determining a feature value of the feature according to the coordinate information of the quadrilateral; The credibility of the feature is obtained according to the feature value and the Gaussian function, wherein the Gaussian function includes preset parameters corresponding to the feature.
20. The device according to claim 19, It is characterized in that The image processing unit is specifically used for: Determine the area of the quadrilateral according to the coordinate information of the quadrilateral; The ratio of the area of the quadrilateral to the area of the image is used as the feature value of the feature.
21. The device according to claim 19, It is characterized in that The image processing unit is specifically used for: According to the coordinate information of the quadrilateral, vectors of two midlines of the quadrilateral are obtained; Determine the angle between the vector of each center line and a vertical line, respectively, wherein the vertical line is perpendicular to the horizontal side line of the image; The smallest of the two angles is taken as the characteristic value of the feature.
22. The device according to claim 19, It is characterized in that The image processing unit is specifically used for: Determine four vertex angles of the quadrilateral according to the coordinate information of the quadrilateral; Performing a difference operation on every two adjacent vertex angles of the quadrilateral to obtain four adjacent angle differences; The sum of four adjacent angle differences is taken as the feature value of the feature.
23. The device according to claim 19, It is characterized in that The image processing unit is specifically used for: According to the coordinate information of the quadrilateral, the coordinates of the middle position of the quadrilateral are determined, and the coordinates of the middle position are used as the feature value of the feature; the coordinates of the middle position are the average value of the coordinates of the four vertices of the quadrilateral, and the coordinate information includes the coordinates of the four vertices.
24. The device according to any one of claims 19 to 23, It is characterized in that The preset parameters include weight, mean value and variance.
25. The device according to claim 19, It is characterized in that The image processing unit is specifically used for: Acquire position and posture information, where the position and posture information is used to represent the position and / or posture of the electronic device when capturing the image; A feature value of the feature is determined according to the pose information and the coordinate information of the quadrilateral.
26. The device according to claim 25, It is characterized in that The image processing unit is specifically used for: Determine a projected gravity vector according to the posture information and the projection function, wherein the projected gravity vector is used to represent the projection of the gravity vector in the world coordinate system in the coordinate system where the image is located; According to the coordinate information of the quadrilateral, vectors of two midlines of the quadrilateral are obtained; respectively determining the angle between the vector of each midline of the quadrilateral and the projected gravity vector; The smallest of the two angles is taken as the characteristic value of the feature.
27. The device according to any one of claims 19 to 23, It is characterized in that The device also includes: A position acquisition unit, used to acquire positioning information, where the positioning information is used to represent the position of the electronic device when the image is captured; a parameter determination unit, configured to determine a shooting scene of the image according to the positioning information; The parameter determination unit is further used to determine a corresponding parameter group according to the shooting scene, and the parameter group includes a preset parameter corresponding to at least one feature.
28. The device according to any one of claims 19 to 23, It is characterized in that The device further comprises: a receiving unit; The receiving unit is used to receive a shooting scene input by a user; The parameter determination unit is further used to determine a corresponding parameter group according to the shooting scene, and the parameter group includes a preset parameter corresponding to at least one feature.
29. The device according to any one of claims 18 to 23, It is characterized in that The image recognition unit is specifically used for: Identify and obtain at least one candidate figure in the image, wherein the candidate figure consists of four line segments; For each candidate figure, when the candidate figure satisfies a first preset condition, the candidate figure is determined to be the quadrilateral, and coordinate information of the quadrilateral is obtained.
30. The device according to claim 29, It is characterized in that The first preset condition includes at least one of the following: Two first angles of the candidate graph are both smaller than a first preset value, and the first angle is the angle between any two non-adjacent line segments of the four line segments; or, The four second angles of the candidate graph are all greater than a second preset value, and the second angle is the angle between any two adjacent line segments of the four line segments; or, The area ratio of the candidate graphic is greater than a third preset value, and the area ratio is a ratio of the area of the candidate graphic to the area of the image.
31. The device according to claim 29, It is characterized in that The image recognition unit is specifically used for: Identify and obtain multiple line segments in the image; Every four line segments of the plurality of line segments are combined to obtain the at least one candidate graph.
32. The device according to claim 31, It is characterized in that The image recognition unit is specifically used for: Determine at least two line segments in the plurality of line segments that are on the same straight line; For each two line segments of the at least two line segments, when the two line segments meet a second preset condition, the two line segments are merged.
33. The device according to claim 32, It is characterized in that The second preset condition includes: The two line segments partially overlap; Alternatively, any one of the two line segments is covered by the other line segment; Alternatively, the distance between the two line segments is smaller than a fourth preset value.
34. The device according to any one of claims 18 to 23, It is characterized in that The image processing unit is specifically used for: The at least one quadrilateral is sorted according to the credibility of each quadrilateral, and the n quadrilaterals with the highest credibility are used as the at least one target quadrilateral, where n≥1; or, For each quadrilateral of the at least one quadrilateral, when the credibility of the quadrilateral is greater than a preset threshold, the quadrilateral is determined to be a target quadrilateral.
35. An electronic device, It is characterized in that include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 17.
36. A computer readable storage medium, It is characterized in that Used to store a computer program, the computer program causing a computer to execute the method according to any one of claims 1 to 17.
37. A computer program product, It is characterized in that The method comprises computer program instructions which cause a computer to execute the method as claimed in any one of claims 1 to 17.
Citation Information
Patent Citations
Image processing method and image processor, digital camera equipment, and recording medium with image processing program stored thereon
CN101248454A
Image processing methods and systems
US20050169531A1
Method and apparatus for identifying the rotation angle and bounding rectangle of a digitized form
US20080018941A1
Image processing apparatus, image capturing apparatus, image processing method, and storage medium
US20170372481A1
Document image capturing and processing
US8897538B1
