Camera rectangular image scanning method and device

Abstract

The invention belongs to the technical field of image processing, and particularly discloses a camera rectangular image scanning method and device, and the method comprises the steps: obtaining an original image obtained through the shooting of a target rectangle by a camera; detecting the contour of a quadrilateral region in the original image, and orderly extracting first coordinates of four vertexes of the quadrilateral region; inputting the first coordinates into a pre-constructed geometric constraint equation set for solving, and determining extension multiples required for mapping four vertexes on an imaging plane to corresponding points on a plane where the target rectangle is located from a main point of the camera; determining an aspect ratio parameter of the target rectangle based on the extension multiple and the first coordinate; detecting the pixel length of the longest side of the quadrilateral region in the original image, and determining second coordinates of four vertexes of the target rectangle in the output image by combining the aspect ratio parameter; solving a perspective transformation matrix in combination with the first coordinates and the second coordinates; and restoring the quadrilateral region by using the solved perspective transformation matrix to generate an output image containing the target rectangle.

Description

Technical Field

[0001] This application belongs to the technical field of image scanning, and more specifically, relates to a method and device for rectangular image scanning by a camera. Background Art

[0002] The electronic storage and transmission of rectangular images are important means to improve work efficiency and save resources. In the traditional electronic process of rectangular images, a scanner can convert a rectangular image into a high-quality electronic image by virtue of the fixing and flattening functions of its physical fixture. With the development of intelligent mobile terminals, the high-resolution cameras they come with provide a new way for the electronicization of rectangular images, and users increasingly choose to achieve the electronic storage and transmission of rectangular images by taking pictures with mobile phones. This method has high convenience and real-time performance and can complete scanning anytime and anywhere.

[0003] However, due to the limitations of the mobile phone imaging principle and handheld shooting, it is often difficult to directly face the rectangular image during shooting, resulting in easy perspective distortion of the captured electronic image. In related technologies, methods based on Euclidean distance estimation or artificial intelligence are used to achieve the scanning and restoration of rectangular images. The Euclidean distance estimation method can estimate the proportion of the original rectangular image depending on the Euclidean distances of the four corners of the rectangular image in the pixel space, but this method is only applicable to the case where the image distortion is not serious. When the image distortion is serious, it is easy to restore a rectangular image to a square image. The artificial intelligence restoration method has high requirements for the computing power at the edge side, the image detection results are unstable, it is difficult to deploy across platforms, and it has a high development cost. Summary of the Invention

[0004] In view of the above-mentioned defects existing in the prior art, this application provides a method and device for rectangular image scanning by a camera, aiming to solve the problem of scanning and restoration of rectangular images captured by a handheld camera.

[0005] In a first aspect, this application provides a method for rectangular image scanning by a camera, including: Obtaining an original image captured by a camera for a target rectangle; Detecting the contour of the quadrilateral region in the original image and obtaining the first coordinates of the four vertices of the ordered quadrilateral region; Inputting the first coordinates into a pre-constructed geometric constraint equation set, solving the geometric constraint equation set, and determining the extension multiples required to map the four vertices on the imaging plane to the corresponding points on the plane where the target rectangle is located starting from the camera principal point; Based on the corresponding extension multiples and the first coordinates, determining the aspect ratio parameter of the target rectangle; Detecting the pixel length of the longest side of the quadrilateral region in the original image, and combining the pixel length and the aspect ratio parameter to determine the second coordinates of the four vertices of the target rectangle in the output image; Solve a perspective transformation matrix by combining the first coordinates of the four vertices of the quadrilateral region and the second coordinates of the four vertices of the target rectangle; Restore the quadrilateral region in the original image by using the solved perspective transformation matrix to generate an output image containing the target rectangle as the scanning result of the target rectangle; Wherein, the geometric constraint equation set is used to represent the following constraint conditions: the opposite sides of the target rectangle are parallel and equal, the adjacent sides of the target rectangle are perpendicular, and mapping the intersection point of the diagonals of the rectangular image from the camera principal point to the intersection point of the diagonals of the quadrilateral region satisfies a preset ratio.

[0006] In a second aspect, the present application further provides a camera rectangular image scanning device, including: An image acquisition module, configured to acquire an original image obtained by the camera shooting the target rectangle; A first detection module, configured to detect the contour of the quadrilateral region in the original image and orderly extract the first coordinates of the four vertices of the quadrilateral region; A first determination module, configured to input the first coordinates into a pre-constructed geometric constraint equation set, solve the geometric constraint equation set, and determine the extension multiple required to map the four vertices on the imaging plane to the corresponding points on the plane where the target rectangle is located starting from the camera principal point; A second determination module, configured to determine the aspect ratio parameter of the target rectangle based on the corresponding extension multiple and the first coordinates; A second detection module, configured to detect the pixel length of the longest side of the quadrilateral region in the original image, and determine the second coordinates of the four vertices of the target rectangle in the output image by combining the pixel length and the aspect ratio parameter; A matrix solving module, configured to solve a perspective transformation matrix by combining the first coordinates and the second coordinates; An image output module, configured to restore the quadrilateral region in the original image by using the solved perspective transformation matrix to generate an output image containing the target rectangle as the scanning result of the target rectangle; Wherein, the geometric constraint equation set is used to represent the following constraint conditions: the opposite sides of the target rectangle are parallel and equal, the adjacent sides of the target rectangle are perpendicular, and mapping the intersection point of the diagonals of the rectangular image from the camera principal point to the intersection point of the diagonals of the quadrilateral region satisfies a preset ratio.

[0007] In a third aspect, the present application further provides an electronic device, including: at least one memory for storing a program; at least one processor for executing the program stored in the memory, and when the program stored in the memory is executed, the processor is configured to execute the method described in the first aspect or any one of the possible implementation manners of the first aspect.

[0008] Fourthly, the present application also provides a computer-readable storage medium storing a computer program, which, when running on a processor, causes the processor to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0009] Fifthly, the present application also provides a computer program product, which, when running on a processor, causes the processor to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0010] For the camera rectangular image scanning method and device provided by the present application, for any target rectangle with unknown aspect ratio parameters, after shooting the target rectangle to obtain the original image, the first coordinates of the four vertices in the imaging plane quadrilateral region are detected, and the geometric constraint equations pre-constructed are solved using the first coordinates to obtain the extension multiples required to map the four vertices on the imaging plane to the corresponding points on the plane where the target rectangle is located starting from the camera principal point. The geometric constraint equations are used to describe the geometric constraints between the target rectangle and the imaging plane and the plane where the target rectangle is located. The aspect ratio parameters of the target rectangle are determined using the first coordinates of the four vertices in the solved imaging plane quadrilateral region and the solved corresponding extension multiples, and the rectangular image is restored using the determined aspect ratio parameters, and the scanning result of the target rectangle is output. The present application realizes the solution of the aspect ratio parameters when the aspect ratio parameters of the target rectangle are unknown, and can accurately restore the actual ratio of the rectangular image. Description of the Drawings

[0011] In order to more clearly illustrate the technical solutions in the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0012] Figure 1 It is a schematic diagram of perspective deformation of a mobile phone captured image provided by an embodiment of the present application; Figure 2 It is a schematic flowchart of the camera rectangular image scanning method provided by an embodiment of the present application; Figure 3 It is a schematic diagram of the principle of the pinhole imaging model provided by an embodiment of the present application; Figure 4 It is one of the schematic diagrams of the effect of camera rectangular image scanning and restoration provided by an embodiment of the present application; Figure 5 It is the second schematic diagram of the effect of camera rectangular image scanning and restoration provided by an embodiment of the present application; Figure 6It is a schematic structural diagram of a camera rectangular image scanning device provided by an embodiment of the present application; Figure 7 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0013] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0014] Figure 1 It is a perspective deformation schematic diagram of a mobile phone captured image provided by an embodiment of the present application. As Figure 1 shown, due to the limitations of hand-held shooting, it is often difficult to directly face the rectangular image during shooting, resulting in perspective deformation of the captured rectangular image. This kind of deformation is caused by the change of the viewing angle. At different viewing angles, the rectangular image will be projected into a quadrilateral.

[0015] Perspective transformation is the process of projecting an image onto a new viewing plane, specifically including: converting a two-dimensional coordinate system into a three-dimensional coordinate system, and then projecting the three-dimensional coordinates onto a new two-dimensional coordinate system. This is a non-linear transformation process. Therefore, a rectangular image often can only obtain a non-rectangular image after perspective transformation.

[0016] Taking the four vertices of the original rectangular image as source points, the source point coordinates are denoted as ; taking the four vertices corresponding to the quadrilateral obtained after perspective transformation as target points, the target point coordinates are denoted as . With the help of homogeneous coordinates for perspective transformation, the transformation process is as follows: Step a, convert the source point coordinates into homogeneous coordinates .

[0017] Step b, matrix multiplication, satisfying: ; wherein, represents the source point homogeneous coordinates and the homogeneous coordinates corresponding to the target point after transformation.

[0018] Step c, normalization: ; wherein, represents the perspective transformation matrix, which has 8 degrees of freedom (usually fixed as 1 to eliminate scale uncertainty), represents the image linear transformation, is used to generate the image perspective transformation, represents the image translation.

[0019] Consider using a perspective transformation matrix to complete the rectification of the rectangular image. The specific process is as follows: Step a: Determine the source point coordinates in the original image and the corresponding target point coordinates in the rectified image.

[0020] Mark the four vertex coordinates of the quadrilateral where the rectangle to be rectified is located in the original image in sequence, and denote them as: . The corresponding points in the rectified image are: , generally denoted as , where w and h represent the width and height of the rectangle in the rectangular image respectively.

[0021] Step b: Solve the perspective transformation matrix .

[0022] For each pair of source points and target points establish two linear equations (corresponding to x and y coordinates respectively), and the following two equations can be obtained:

[0023] Furthermore, it can be transformed into:

[0024] After expanding and arranging, we get:

[0025] Among them, . 4 pairs of source points and target points generate 8 linear equations in total, just solving 8 unknowns to obtain the perspective transformation matrix .

[0026] Step c: Use the perspective transformation matrix to restore the original image, map the quadrilateral region to the rectified rectangular image region, and obtain the rectified rectangular image. The result of the rectification is a new image, whose content contains the content of the quadrilateral region in the source image and has undergone perspective transformation to adapt to the rectangular shape.

[0027] During the rectification process, generally use the maximum value of the lengths of the sides of the quadrilateral in the original image as the reference length. The reference length determines the proportional benchmark (number of pixels) of the width or height of the rectified rectangular image, and combines the reference length and the pre-given aspect ratio parameter to calculate the width and height of the rectified rectangular image.

[0028] The above transformation process realizes the geometric transformation from an arbitrary quadrilateral region to a rectangular image region, which is widely used in scenarios such as rectangular image scanning and image correction. However, a prerequisite for its application is to know the aspect ratio parameter of the rectangular image after correction. But it is often difficult to obtain this aspect ratio parameter, or it is limited by certain scenarios (such as the deformation correction of A4 paper).

[0029] Based on this, the technical solution provided in the embodiments of this application proposes a method that utilizes the optical characteristics of a camera to calculate the aspect ratio parameter of a rectangular image when the picture has not been cropped and deformed. When the aspect ratio parameter of the rectangular image is known, the perspective transformation matrix can be obtained by solving the perspective transformation of the corners of the rectangular image, and then the restoration of the rectangular image can be realized.

[0030] Figure 2 It is a schematic flowchart of the method for scanning a rectangular image by a camera provided in the embodiments of this application. As Figure 2 shown, the method at least includes the following steps: S201. Obtain the original image captured by the camera for the target rectangle; S202. Detect the contour of the quadrilateral region in the original image, and orderly extract the first coordinates of the four vertices of the quadrilateral region; S203. Input the first coordinates into the pre-constructed geometric constraint equations, solve the geometric constraint equations, and determine the extension multiple required to map the four vertices on the imaging plane to the corresponding points on the plane where the target rectangle is located starting from the camera principal point; S204. Determine the aspect ratio parameter of the target rectangle based on the corresponding extension multiple and the first coordinates; S205. Detect the pixel length of the longest side of the quadrilateral region in the original image, and combine the pixel length and the aspect ratio parameter to determine the second coordinates of the four vertices of the target rectangle in the output image; S206. Solve the perspective transformation matrix by combining the first coordinates and the second coordinates; S207. Use the solved perspective transformation matrix to restore the quadrilateral region in the original image, generate an output image containing the target rectangle, and use it as the scanning result of the target rectangle.

[0031] The camera rectangular image scanning method and device provided by this application, for any target rectangle with unknown aspect ratio parameters, after shooting the target rectangle to obtain the original image, detect the first coordinates of the four vertices within the quadrilateral region of the imaging plane, and use the first coordinates to solve the pre-constructed geometric constraint equations to obtain the extension multiple required to map the four vertices on the imaging plane to the corresponding points on the plane where the target rectangle is located starting from the camera principal point. The geometric constraint equations are used to describe the geometric constraints between the target rectangle and the imaging plane and the plane where the target rectangle is located. Use the first coordinates of the four vertices within the solved quadrilateral region of the imaging plane and the solved corresponding extension multiple to determine the aspect ratio parameters of the target rectangle, and use the determined aspect ratio parameters to restore the rectangular image, and output the scanning result of the target rectangle. This application realizes the solution of the aspect ratio parameters under the condition of unknown aspect ratio parameters of the target rectangle, and can accurately restore the actual ratio of the rectangular image.

[0032] Specifically, the construction of the geometric constraint equations is the key to solving the aspect ratio parameters of the target rectangle. The combined constraint equations are used to describe some geometric constraints of the target rectangle itself, such as opposite sides being parallel and equal, and adjacent sides being perpendicular to each other, and are also used to describe the relationship between the quadrilateral region of the imaging plane and the corresponding points on the plane where the target rectangle is located. It can be imagined that the coordinates of the intersection point of the diagonals of the quadrilateral can be calculated from the coordinates of the four vertices.

[0033] In some embodiments, the geometric constraint equations are pre-constructed through the following steps: Combined with the perspective transformation principle and the pinhole imaging model, taking the camera principal point as a reference, establish the physical model of perspective transformation; Mark the first vectors from the camera principal point to the four vertices and the intersection point of the diagonals of the quadrilateral region on the imaging plane, and define the extension multiple that maps the four vertices and the intersection point of the diagonals of the quadrilateral region on the imaging plane to the corresponding points on the plane where the target rectangle is located starting from the camera principal point; Based on the first vectors and the extension multiple, determine the second vectors from the camera principal point to the four vertices and the intersection point of the diagonals of the target rectangle; Construct the geometric constraint equations based on the second vectors.

[0034] Specifically, Figure 3 is the schematic diagram of the principle of the pinhole imaging model provided by the embodiment of this application. As Figure 3 shown, this pinhole imaging model establishes the physical model of perspective transformation, that is, the model in which light rays start from the surface of an object, pass through the pinhole that receives light by the camera, pass through the imaging plane, form an image, and finally reach the camera principal point.

[0035] First, assume that from the four vertices of the rectangular image and the intersection point of its diagonals Five emitted light beams project points on the imaging plane of the camera , and the first vectors from the principal point E of the camera to these five points on the imaging plane are respectively denoted as .

[0036] Then, the second vectors from the principal point E of the camera to the four vertices of the rectangular image and the intersection point of the diagonals can be expressed as:

[0037] wherein, respectively represent the first vectors from the principal point of the camera to the four vertices and the intersection point of the diagonals of the quadrilateral region on the imaging plane first respectively represent the second vectors from the principal point of the camera to the four vertices and the intersection point of the diagonals on the plane where the target rectangle is located, respectively represent the extension multiples corresponding to mapping each first vector to the second vector, and are used to map the points on the imaging plane to the corresponding points on the plane where the target rectangle is located.

[0038] Based on this, using the constraint conditions that the opposite sides of the rectangle in the rectangular image are parallel and equal, and the intersection point F of the diagonals of the rectangular image is mapped to the intersection point f of the diagonals of the quadrilateral on the imaging plane, the following geometric constraint equations are proposed:

[0039] wherein, the equation represents that the points obtained by extending and from the principal point of the camera, and the points obtained by extending and have a vector difference of 0, and are used to describe the parallel and equal relationship between the opposite sides of the rectangle; the equation represents that the weighted sum of and points to a certain proportional position of , and is used to describe the constraint condition that the intersection point F of the diagonals of the rectangular image is mapped to the intersection point f of the diagonals of the quadrilateral on the imaging plane; the equation represents that and are orthogonal, and are used to describe the perpendicular relationship between the adjacent sides of the rectangular image.

[0040] That is to say, this geometric constraint equation set is used to characterize the following constraint conditions: the opposite sides of the target rectangle are parallel and equal, the adjacent sides of the target rectangle are perpendicular, and mapping the intersection point of the diagonals of the quadrilateral region from the principal point of the camera to the intersection point of the diagonals of the target rectangle satisfies a preset ratio.

[0041] Since it is difficult to know the actual distance of the object from the camera, one of them in can be fixed as a constant much larger than the focal length of the camera. Optionally, since the imaging plane must be between the camera principal point and the actual image, a magnification factor required to map the intersection of the diagonals of the quadrilateral region on the imaging plane to the intersection of the diagonals of the target rectangle starting from the camera principal point is set as a constant much larger than the focal length of the camera. For example, let .

[0042] Since equation (3) is non-linear, that is, this system of equations is non-homogeneous, it cannot be solved according to linear equations. However, by using the constraint relationship, this problem can be transformed into an optimization problem, that is, to find the set of

[0043] where is closest to . .

[0044] Since only a small part of this formula is non-linear and the solution is unique, the common Newton's method (or least squares method, Levenberg-Marquardt method, etc.) for solving non-linear equations is used. After a small number of iterations, a solution with a sufficiently low residual can basically be obtained, thereby obtaining all k. At this time, the lengths of two adjacent sides of the rectangular image can be obtained by: , and further calculate the aspect ratio parameter as:

[0045] where represents the aspect ratio parameter, and represent the width and height of the target rectangle respectively, represent the first vectors from the camera principal point to three consecutive vertices in the quadrilateral region on the imaging plane first , represent the second vectors from the camera principal point to the corresponding points on the plane where the target rectangle is located respectively, represent the magnification factors for mapping the first vectors corresponding to the aforementioned three consecutive vertices to the second vectors respectively.

[0046] It should be noted that the representation of the first vector and the second vector depends on the coordinates of the camera principal point. Although the coordinates of the camera principal point are unknown, the coordinates of the camera principal point can be eliminated during the calculation process, and the unknown principal point coordinates do not affect the calculation of the final aspect ratio parameter.

[0047] Subsequently, use this aspect ratio parameter r , combined with the pixel length of the longest side of the quadrilateral region in the original imagel , calculate the second coordinates of the four vertices of the target rectangle ; then apply the first coordinates of the four vertices of the quadrilateral region in the original image and the second coordinates of the four vertices of the target rectangle to solve the perspective transformation matrix. Through the aforementioned perspective transformation, the original target rectangle can be restored, and an output image containing the target rectangle is generated as the scanning result of the target rectangle.

[0048] Figure 4 is one of the schematic diagrams of the effect of the camera rectangular image scanning and restoration provided by the embodiments of the present application. Figure 5 is the second schematic diagram of the effect of the camera rectangular image scanning and restoration provided by the embodiments of the present application. As Figure 4 and Figure 5 shown, through the technical solution provided by the embodiments of the present application, the actual proportion of the rectangular image can be restored very accurately. The proportion of the restored rectangular image is correct, and the text is clear and distortion-free.

[0049] Next, the camera rectangular image scanning device provided by the present application will be described. The camera rectangular image scanning device described below can be correspondingly referred to the camera rectangular image scanning method described above.

[0050] Figure 6 is the structural schematic diagram of the camera rectangular image scanning device provided by the embodiments of the present application. As Figure 6 shown, the device at least includes: An image acquisition module, configured to acquire the original image obtained by the camera shooting the target rectangle; A first detection module, configured to detect the contour of the quadrilateral region in the original image and orderly extract the first coordinates of the four vertices of the quadrilateral region; A first determination module, configured to input the first coordinates into a pre-constructed geometric constraint equation set, solve the geometric constraint equation set, and determine the extension multiple required to map the four vertices on the imaging plane to the corresponding points on the plane where the target rectangle is located starting from the camera principal point; A second determination module, configured to determine the aspect ratio parameter of the target rectangle based on the corresponding extension multiple and the first coordinates; A second detection module, configured to detect the pixel length of the longest side of the quadrilateral region in the original image, and combine the pixel length and the aspect ratio parameter to determine the second coordinates of the four vertices of the target rectangle in the output image; A matrix solution module, configured to solve the perspective transformation matrix by combining the first coordinates and the second coordinates; An image output module, configured to restore the quadrilateral region in the original image by using the solved perspective transformation matrix, and generate an output image containing the target rectangle as the scanning result of the target rectangle; Among them, the geometric constraint equation set is used to represent the following constraint conditions: the opposite sides of the target rectangle are parallel and equal, the adjacent sides of the target rectangle are perpendicular, and mapping the intersection point of the diagonal of the rectangle image from the camera principal point to the intersection point of the diagonal of the quadrilateral region satisfies a preset ratio.

[0051] In some embodiments, the geometric constraint equation set is pre-constructed through the following steps: Combining the perspective transformation principle and the pinhole imaging model, taking the camera principal point as a reference, a physical model of perspective transformation is established; Mark the first vectors from the camera principal point to the four vertices and the intersection point of the diagonal of the quadrilateral region on the imaging plane, and define the extension multiples for mapping the four vertices and the intersection point of the diagonal of the quadrilateral region on the imaging plane from the camera principal point to the corresponding points on the plane where the target rectangle is located; Based on the first vectors and the extension multiples, determine the second vectors from the camera principal point to the four vertices and the intersection point of the diagonal of the target rectangle; Construct the geometric constraint equation set based on the second vectors.

[0052] In some embodiments, the geometric constraint equation set satisfies:

[0053] Among them, respectively represent the first vectors from the camera principal point to the four vertices and the intersection point of the diagonal in the quadrilateral region on the imaging plane respectively represent the second vectors from the camera principal point to the four vertices and the intersection point of the diagonal on the plane where the target rectangle is located, and respectively represent the extension multiples of each first vector corresponding to the mapping to the second vector.

[0054] In some embodiments, the construction steps of the geometric constraint equation set further include: Preset the extension multiple required for mapping the intersection point of the diagonal of the quadrilateral region on the imaging plane from the camera principal point to the intersection point of the diagonal of the target rectangle to be a constant much larger than the camera focal length.

[0055] In some embodiments, the geometric constraint equation set is solved by the Newton method.

[0056] In some embodiments, determine the aspect ratio parameter of the target rectangle, which satisfies the following calculation formula:

[0057] Among them, represents the aspect ratio parameter, and respectively represent the width and height of the target rectangle, respectively represent three consecutive vertices from the principal point of the camera to the quadrilateral region on the imaging plane First , respectively represent the second vectors from the principal point of the camera to the corresponding points on the plane where the target rectangle is located, respectively represent the extension multiples of the corresponding mapping of the first vectors of the aforementioned three consecutive vertices to the second vectors.

[0058] It can be understood that for the detailed function implementation of each of the above units / modules, reference can be made to the introduction in the foregoing method embodiments, which will not be elaborated herein.

[0059] It should be understood that the above device is used to execute the method in the above embodiments. For the corresponding program modules in the device, their implementation principles and technical effects are similar to those described in the above method. The working process of the device can refer to the corresponding process in the above method, which will not be elaborated herein.

[0060] Based on the method in the above embodiments, an embodiment of the present application provides an electronic device. The device may include: at least one memory for storing programs and at least one processor for executing the programs stored in the memory. Among them, when the program stored in the memory is executed, the processor is used to execute the method described in the above embodiments.

[0061] Figure 7 is a schematic structural diagram of the electronic device provided by an embodiment of the present application. As Figure 7 shown, the electronic device may include: a processor (Processor) 701, a communication interface (Communications Interface) 702, a memory (Memory) 703, and a communication bus 704. Among them, the processor 701, the communication interface 702, and the memory 703 complete mutual communication through the communication bus 704. The processor 701 may call software instructions in the memory 703 to execute the method described in the above embodiments.

[0062] In addition, when the logical instructions in the above memory 703 are implemented in the form of software function units and sold or used as an independent product, they may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application.

[0063] Based on the method in the above embodiments, an embodiment of the present application provides a computer-readable storage medium storing a computer program, and when the computer program runs on a processor, the processor is caused to execute the method in the above embodiments.

[0064] Based on the method in the above embodiments, an embodiment of the present application provides a computer program product, and when the computer program product runs on a processor, the processor is caused to execute the method in the above embodiments.

[0065] It can be understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0066] The method steps in the embodiments of the present application may be implemented in a hardware manner or by a processor executing software instructions. The software instructions may be composed of corresponding software modules, and the software modules may be stored in a random access memory (RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), registers, hard disks, removable hard disks, CD-ROMs or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may be located in an ASIC.

[0067] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0068] It can be understood that the various digital numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0069] Those skilled in the art can easily understand that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for rectangular image scanning of a camera, characterized in that, Including: Obtain the original image captured by the camera for the target rectangle; Detect the contour of the quadrilateral region in the original image, and orderly extract the first coordinates of the four vertices of the quadrilateral region; Input the first coordinates into the pre-constructed geometric constraint equations, solve the geometric constraint equations, and determine the extension multiple required to map the four vertices on the imaging plane to the corresponding points on the plane where the target rectangle is located starting from the camera principal point; Based on the corresponding extension multiple and the first coordinates, determine the aspect ratio parameter of the target rectangle; Detect the pixel length of the longest side of the quadrilateral region in the original image, and combine the pixel length and the aspect ratio parameter to determine the second coordinates of the four vertices of the target rectangle in the output image; Combine the first coordinates and the second coordinates to solve the perspective transformation matrix; Use the solved perspective transformation matrix to restore the quadrilateral region in the original image, generate an output image containing the target rectangle, and use it as the scanning result of the target rectangle; Among them, the geometric constraint equations are used to represent the following constraint conditions: the opposite sides of the target rectangle are parallel and equal, the adjacent sides of the target rectangle are perpendicular, and mapping the intersection point of the diagonals of the quadrilateral region to the intersection point of the diagonals of the target rectangle starting from the camera principal point satisfies a preset ratio.

2. The method for scanning a rectangular image of a camera according to claim 1, wherein, The geometric constraint equations are pre-constructed through the following steps: Combine the perspective transformation principle and the pinhole imaging model, and establish a physical model of perspective transformation with the camera principal point as a reference; Mark the first vectors from the camera principal point to the four vertices and the intersection point of the diagonals of the quadrilateral region on the imaging plane, and define the extension multiple for mapping the four vertices and the intersection point of the diagonals of the quadrilateral region on the imaging plane to the corresponding points on the plane where the target rectangle is located starting from the camera principal point; Based on the first vectors and the extension multiple, determine the second vectors from the camera principal point to the four vertices and the intersection point of the diagonals of the target rectangle; Construct the geometric constraint equations based on the second vectors.

3. The camera rectangular image scanning method according to claim 2, characterized in that, The geometric constraint equations satisfy: Among them, respectively represent the four vertices and the intersection point of the diagonals of the quadrilateral region on the imaging plane starting from the camera principal point First respectively represent the second vectors from the camera principal point to the four vertices and the intersection point of the diagonals on the plane where the target rectangle is located, respectively represent the extension multiples of the corresponding mapping of each first vector to the second vector.

4. The camera rectangular image scanning method according to claim 2, characterized in that, The construction steps of the geometric constraint equations further include: Preset the extension multiple required to map the intersection point of the diagonals of the quadrilateral region on the imaging plane to the intersection point of the diagonals of the target rectangle starting from the camera principal point as a constant much larger than the camera focal length.

5. The method for rectangular image scanning of a camera according to claim 1, wherein The geometric constraint equations are solved by the Newton method.

6. The method for scanning a rectangular image of a camera according to claim 1, characterized in that, Determining the aspect ratio parameter of the target rectangle satisfies the following calculation formula: Among them, represents the aspect ratio parameter, and represent the width and height of the target rectangle respectively, respectively represent the first , which are three consecutive vertices of the quadrilateral region on the imaging plane starting from the camera principal point, respectively represent the extension multiples of the first vectors corresponding to the three consecutive vertices mapped to the second vectors.

7. A rectangular image scanning device for a camera, characterized in that, Including: An image acquisition module for obtaining the original image captured by the camera for the target rectangle; A first detection module for detecting the contour of the quadrilateral region in the original image and orderly obtaining the first coordinates of the four vertices of the quadrilateral region; A first determination module for inputting the first coordinates into the pre-constructed geometric constraint equations, solving the geometric constraint equations, and determining the extension multiple required to map the four vertices on the imaging plane to the corresponding points on the plane where the target rectangle is located starting from the camera principal point; A second determination module, configured to determine an aspect ratio parameter of the target rectangle based on the corresponding stretching multiple and the first coordinates; A second detection module, configured to detect a pixel length of the longest side of the quadrilateral region in the original image, and determine second coordinates of four vertices of the target rectangle in the output image by combining the pixel length and the aspect ratio parameter; A matrix solving module, configured to solve a perspective transformation matrix by combining the first coordinates and the second coordinates; An image output module, configured to restore the quadrilateral region in the original image by using the solved perspective transformation matrix, and generate an output image including the target rectangle as a scanning result of the target rectangle; Wherein, the geometric constraint equation set is used to represent the following constraint conditions: opposite sides of the target rectangle are parallel and equal, adjacent sides of the target rectangle are perpendicular, and mapping the intersection point of the diagonals of the rectangular image from the camera principal point to the intersection point of the diagonals of the quadrilateral region satisfies a preset ratio.

8. An electronic device, characterized in that, Including: At least one memory, configured to store a computer program; At least one processor, configured to execute the program stored in the memory. When the program stored in the memory is executed, the processor is configured to execute the method according to any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program runs on the processor, the processor is caused to execute the method according to any one of claims 1-6.

10. A computer program product, characterized in that, When the computer program product runs on the processor, the processor is caused to execute the method according to any one of claims 1-6.

Images