Image distortion correction processing method and system for book detection

Abstract

The invention provides an image distortion correction processing method and system for book detection, and the method comprises the following steps: positioning a camera image to a bookshelf partition, pre-configuring collection point parameters of a camera, obtaining a real-time image through the camera, and storing the real-time image in the bookshelf partition; wherein the parameters of the acquisition point comprise the zoom multiple, the focal length, the horizontal coordinate and the vertical coordinate of the camera; carrying out identification area labeling on the real-time image in a polygon drawing mode, and obtaining a to-be-processed area needing to be subjected to OCR (Optical Character Recognition); spine text correction processing is conducted on the to-be-processed area, and the spine text correction processing comprises the steps that the spine text is made to be in a vertical state by adjusting a horizontal projection change factor and a vertical projection change factor; and storing the configuration information of the current camera and the spine text correction processing related parameters for subsequent OCR (Optical Character Recognition). The method can ensure that the spine text is in a vertical state before OCR recognition, and greatly improves the accuracy and efficiency of OCR recognition.

Description

Technical Field

[0001] The present application relates to the technical field of image distortion correction, and in particular to an image distortion correction processing method and system for book detection. Background Art

[0002] The daily maintenance of book shelf location information in modern libraries usually involves manual placement of books using call numbers or manual inventory operations using RFID tags and RFID reader / writers to maintain the shelf location information of books for readers to query using the retrieval system. In order to simplify the daily work of library managers, a camera AI inventory system has emerged.

[0003] In the AI ​​inventory system, the images of the book shelves captured by the camera are used as the input images for OCR. The angle and area of ​​the images captured affect the accuracy of OCR recognition. For example, when the hanging camera captures the images of the book shelves at a fixed position, the spine information of the books on the bookshelf that are not facing the camera is not fully captured or distorted due to the complex layout and height of the bookshelves and the limitations of the camera's field of view. This angle deviation will directly affect the OCR system's ability to recognize the text information on the spine.

[0004] In view of this, the present application proposes an image distortion correction processing method, system, device and storage medium for book detection, which can optimize the accuracy of OCR system recognition. Summary of the invention

[0005] In order to solve the problem that the book spine information in the pictures collected by existing cameras is incompletely captured or distorted, which affects the OCR system's ability to recognize the text information on the spine. This application provides an image distortion correction processing method, system, device and storage medium for book detection to solve the above-mentioned technical defects.

[0006] According to the first aspect of the present application, a method for image distortion correction processing for book detection is proposed, the method comprising the following steps:

[0007] S1. Position the camera image to the bookshelf partition, pre-configure the camera's acquisition point parameters, and obtain real-time images through the camera, where the acquisition point parameters include: the camera's zoom factor, focal length, horizontal coordinates, and vertical coordinates;

[0008] S2, marking the recognition area of ​​the real-time image by drawing polygons to obtain the area to be processed that needs to be OCR recognized;

[0009] S3, performing spine text correction processing on the area to be processed, the spine text correction processing comprising: adjusting the horizontal projection change factor and the vertical projection change factor to make the spine text appear vertical;

[0010] S4. Save the current camera configuration information and spine text correction processing related parameters for subsequent OCR recognition.

[0011] In the above technical solution, the present application can ensure that the spine text is in a vertical state before OCR recognition through image distortion correction processing, which greatly improves the accuracy and efficiency of OCR recognition.

[0012] Preferably, the image distortion correction processing method for book detection provided by the present application further includes the following steps:

[0013] S5. Move the camera to the next partition of the bookshelf and perform parameter configuration of the next partition. The parameter configuration includes: reusing or re-marking the coordinates of the recognition area, and performing corresponding correction and saving operations.

[0014] In the above technical solution, the present application supports moving the camera to the next partition for continuous processing, and by reusing or re-marking the recognition area coordinates, it can achieve rapid scanning and processing of the entire bookshelf.

[0015] Preferably, in step S3, the spine text correction process is performed on the area to be processed, including correction in two directions: horizontal shearing and vertical shearing, wherein the formula for horizontal shearing is:

[0016]

[0017] Where Shear(x) is the x-axis coordinate after the change, x is the image x-coordinate of the input area to be processed, y is the image y-coordinate of the input area to be processed, σ x is the horizontal projection change factor, W is the width of the input image of the area to be processed;

[0018] The formula for vertical shear is:

[0019]

[0020] Where Shear(y) is the changed y-axis coordinate, y is the input image y coordinate of the area to be processed, x is the input image x coordinate of the area to be processed, σ y is the vertical projection change factor, H is the height of the input image of the area to be processed;

[0021] By changing the horizontal projection change factor and the vertical projection change factor respectively, an adjusted image is obtained.

[0022] Preferably, in step S1, the camera image is positioned to the bookshelf partition, and the camera collection point parameters are pre-configured, which specifically includes the following sub-steps:

[0023] S11, dividing a shelf into multiple partitions according to the positions of the camera and the bookshelf, and two adjacent partitions partially overlap;

[0024] S12, adjusting the camera position to one of the partitions so that the partition is placed in the center area of ​​the camera screen, and recording the horizontal coordinate and vertical coordinate of the current camera;

[0025] S13, configuring the magnification of the camera so that the camera image only includes the books in the current bookshelf partition;

[0026] S14. Configure the focal length of the camera and adjust it so that the text on the spine of the book is clear in the camera image.

[0027] In the above technical solution, by pre-configuring the acquisition point parameters of the camera, including zoom factor, focal length, horizontal coordinate and vertical coordinate, the method can flexibly adapt to different bookshelf partitions and book arrangements, thereby improving the versatility and adaptability of the system.

[0028] Preferably, in step S2, the real-time image is marked with identification areas by drawing polygons to obtain the area to be processed that needs to be OCR recognized, including: drawing a polygon in the real-time image, and clicking the polygon vertices or dragging the edges with the mouse to draw a closed polygon as the area to be processed.

[0029] In the above technical solution, the area is marked by drawing polygons, which is easy and intuitive to operate and reduces the user's usage threshold.

[0030] Further preferably, the image distortion correction processing method for book detection provided by the present application further includes:

[0031] Keep the marked area of ​​the previous partition of the bookshelf on the camera image layer until the clearing operation is performed;

[0032] If the marked area of ​​the previous partition only covers the books of the current partition, the area coordinate points of the previous partition are directly used for correction and saving operations;

[0033] If the marked area of ​​the previous partition also covers part of the data of the spines of the remaining shelves, the area marking operation is repeated and then the correction and saving operations are performed.

[0034] Through the above technical solution, the camera configuration information and spine text correction processing related parameters can be automatically saved, ensuring the consistency and traceability of the data.

[0035] In a second aspect, the present application provides an image distortion correction processing system for book detection, the system comprising:

[0036] The acquisition module is configured to locate the camera image to the bookshelf partition, pre-configure the acquisition point parameters of the camera, and acquire the real-time image through the camera, wherein the acquisition point parameters include: the zoom factor, focal length, horizontal coordinate and vertical coordinate of the camera;

[0037] The marking module is configured to mark the recognition area of ​​the real-time image by drawing polygons to obtain the area to be processed that needs to be OCR recognized;

[0038] The correction module is configured to perform spine text correction processing on the area to be processed, and the spine text correction processing includes: adjusting the horizontal projection change factor and the vertical projection change factor to make the spine text present a vertical state;

[0039] The storage module is configured to save the current camera configuration information and spine text correction processing related parameters for subsequent OCR recognition.

[0040] In a third aspect, the present application provides a terminal device, including a processor, a memory, and a computer program stored in the memory, wherein the computer program is executed by the processor to implement an image distortion correction processing method for book detection as described in any one of the above.

[0041] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the medium stores a computer program, and when the computer program is executed by a processor, the image distortion correction processing method for book detection as described in any one of the above is implemented.

[0042] Compared with the prior art, the beneficial results of the present invention are:

[0043] (1) Improve OCR recognition accuracy: Through image distortion correction processing, the spine text is ensured to be in a vertical state before OCR recognition, which greatly improves the accuracy and efficiency of OCR recognition.

[0044] (2) Flexibility and adaptability: By pre-configuring the camera’s acquisition point parameters, including zoom factor, focal length, horizontal coordinate, and vertical coordinate, this method can flexibly adapt to different bookshelf partitions and book arrangements, thus improving the system’s versatility and adaptability.

[0045] (3) Efficient Automation: The entire processing flow is highly automated, including steps such as area marking, spine text correction, and parameter saving, which reduces manual intervention and improves processing speed and efficiency.

[0046] (4) Partition processing optimization: Supports moving the camera to the next partition for continuous processing, reducing the number of camera installations and project equipment costs. By reusing or re-marking the recognition area coordinates, rapid scanning and processing of the entire bookshelf is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0048] Figure 1 is a flow chart of an image distortion correction processing method for book detection according to the present application;

[0049] Figure 2a is a schematic diagram of a book spine before text correction processing according to the present application;

[0050] Figure 2b is a schematic diagram of a book spine after text correction processing according to the present application;

[0051] Figure 3 is a structural diagram of an image distortion correction processing system for book detection according to the present application;

[0052] Figure 4 It is a structural diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application. DETAILED DESCRIPTION

[0053] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0054] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0055] Figure 1 The flowchart of the image distortion correction processing method for book detection of the present application is shown as follows: Figure 1 As shown, the method comprises the following steps:

[0056] S1. Position the camera image to the bookshelf partition, pre-configure the camera's acquisition point parameters, and obtain real-time images through the camera, wherein the acquisition point parameters include: the camera's zoom factor, focal length, horizontal coordinates, and vertical coordinates.

[0057] In this embodiment, the camera image is positioned to the bookshelf partition, and the camera collection point parameters are pre-configured, which specifically includes the following sub-steps:

[0058] S11. Log in to the camera, locate the camera image to the relevant bookshelf, and then divide a shelf into multiple partitions according to the position of the camera and the bookshelf, with two adjacent partitions partially overlapping. The shelf is generally divided into 3-5 areas (the specific number of partitions needs to be considered based on the length of the shelf and the angle of the camera, and is not limited to the range of 3-5). Each area needs to partially overlap with the adjacent area to prevent missing books due to later changes in books.

[0059] S12, adjusting the camera position to one of the partitions so that the partition is placed in the center area of ​​the camera screen, and recording the horizontal coordinate and vertical coordinate of the current camera.

[0060] S13, configuring the magnification of the camera, and zooming in until the camera image only includes the books in the current bookshelf partition.

[0061] S14. Configure the focal length of the camera and adjust it so that the text on the spine of the book is clear in the camera image.

[0062] Continue to refer Figure 1 The image distortion correction processing method for book detection provided by the present application also includes:

[0063] S2. Mark the recognition area of ​​the real-time image by drawing a polygon to obtain the area to be processed that needs to be OCR recognized. Specifically, draw a polygon in the real-time image, and click the polygon vertex or drag the edge with the mouse to draw a closed polygon as the area to be processed.

[0064] S3, performing spine text correction processing on the area to be processed, the spine text correction processing includes: adjusting the horizontal projection change factor and the vertical projection change factor to make the spine text appear vertical.

[0065] In this embodiment, the spine text correction processing is performed on the area to be processed, including correction in two directions: horizontal shearing and vertical shearing. The formula for horizontal shearing is:

[0066]

[0067] Where Shear(x) is the x-axis coordinate after the change, x is the image x-coordinate of the input area to be processed, y is the image y-coordinate of the input area to be processed, σ x is the horizontal projection change factor, W is the width of the input image of the area to be processed;

[0068] The formula for vertical shear is:

[0069]

[0070] Where Shear(y) is the changed y-axis coordinate, y is the input image y coordinate of the area to be processed, x is the input image x coordinate of the area to be processed, σ y is the vertical projection change factor, H is the height of the input image of the area to be processed;

[0071] By changing the horizontal projection change factor and the vertical projection change factor respectively, the adjusted image is obtained to observe whether the text conforms to the horizontal and vertical state seen by the human eye. Figure 2a and Figure 2b As shown, Figure 2a shows a schematic diagram before spine text correction processing, Figure 2b A schematic diagram showing the spine text after correction processing.

[0072] The correction process will record the horizontal and vertical σ projection change factor values ​​configured in the previous area of ​​this layer. In different areas on the same shelf, the values ​​of the σ projection change factors are similar. After adjusting one area, the subsequent ones only need to be fine-tuned on the previous value.

[0073] Continue to refer Figure 1 The image distortion correction processing method for book detection provided by the present application also includes:

[0074] S4. Save the current camera configuration information and spine text correction processing related parameters for subsequent OCR recognition. The configuration information includes: camera position, focal length, magnification, and marked area coordinates. The correction processing related parameters include: horizontal projection change factor and vertical projection change factor.

[0075] S5. Move the camera to the next partition of the bookshelf and perform parameter configuration of the next partition. The parameter configuration includes: reusing or re-marking the coordinates of the recognition area, and performing corresponding correction and saving operations.

[0076] In a specific embodiment, the image distortion correction processing method for book detection provided by the present application further includes:

[0077] Keep the marked area of ​​the previous partition of the bookshelf on the camera image layer until the clearing operation is performed;

[0078] If the marked area of ​​the previous partition only covers the books of the current partition, the area coordinate points of the previous partition are directly used for correction and saving operations;

[0079] If the marked area of ​​the previous partition also covers part of the data of the spines of the remaining shelves, the area marking operation is repeated and then the correction and saving operations are performed.

[0080] Further references Figure 3As an implementation of the above method, in the second aspect, the present application provides an embodiment of a structural diagram of an image distortion correction processing system 300 for book detection, which can be specifically applied to various electronic devices. The system 300 includes the following modules:

[0081] The acquisition module 310 is configured to locate the camera image to the bookshelf partition, pre-configure the acquisition point parameters of the camera, and acquire the real-time image through the camera, wherein the acquisition point parameters include: the zoom factor, focal length, horizontal coordinate and vertical coordinate of the camera;

[0082] The marking module 320 is configured to mark the recognition area of ​​the real-time image by drawing polygons to obtain the area to be processed that needs to be OCR recognized;

[0083] The correction module 330 is configured to perform spine text correction processing on the area to be processed, and the spine text correction processing includes: adjusting the horizontal projection change factor and the vertical projection change factor to make the spine text appear vertical;

[0084] The storage module 340 is configured to store the current camera configuration information and spine text correction processing related parameters for subsequent OCR recognition.

[0085] In a third aspect, the present application provides a terminal device, including a processor, a memory, and a computer program stored in the memory, wherein the computer program is executed by the processor to implement an image distortion correction processing method for book detection as described in any one of the above.

[0086] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the medium stores a computer program, and when the computer program is executed by a processor, the image distortion correction processing method for book detection as described in any one of the above is implemented.

[0087] Reference below Figure 4 , which shows a schematic diagram of the structure of a computer system 400 of a terminal device or server suitable for implementing an embodiment of the present application. Figure 4 The terminal device or server shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0088] like Figure 4As shown, the computer system 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage part 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the computer system 400 are also stored. The CPU 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0089] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage section 408 as needed.

[0090] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 409, and / or installed from the removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, the above functions defined in the method of the present application are executed. It should be noted that the computer-readable medium described in the present application can be a computer-readable signal medium or a computer-readable medium or any combination of the above two. The computer-readable medium can be, for example, - but not limited to - a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, an apparatus or a device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, an apparatus or a device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0091] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as C or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0092] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0093] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. An image distortion correction processing method for book detection, characterized in that: The method comprises the following steps: S1, positioning the camera image to the bookshelf partition, pre-configuring the acquisition point parameters of the camera, and acquiring a real-time image through the camera, wherein the acquisition point parameters include: the zoom factor, focal length, horizontal coordinate and vertical coordinate of the camera; S2, marking the recognition area of ​​the real-time image by drawing polygons to obtain the area to be processed that needs to be OCR recognized; S3, performing spine text correction processing on the area to be processed, wherein the spine text correction processing includes: adjusting a horizontal projection change factor and a vertical projection change factor to make the spine text appear vertical; S4. Save the current camera configuration information and spine text correction processing related parameters for subsequent OCR recognition.

2. The image distortion correction processing method for book detection according to claim 1, characterized in that: The following steps are also included: S5. Move the camera to the next partition of the bookshelf and perform parameter configuration of the next partition. The parameter configuration includes: reusing or re-marking the coordinates of the recognition area, and performing corresponding correction and saving operations.

3. The image distortion correction processing method for book detection according to claim 1, characterized in that: In step S3, the spine text correction process is performed on the area to be processed, including correction in two directions: horizontal shearing and vertical shearing, wherein the formula for the horizontal shearing is: Where Shear(x) is the x-axis coordinate after the change, x is the image x-coordinate of the input area to be processed, y is the image y-coordinate of the input area to be processed, σ x is the horizontal projection change factor, W is the width of the input image of the area to be processed; The formula for vertical shear is: Where Shear(y) is the changed y-axis coordinate, y is the input image y coordinate of the area to be processed, x is the input image x coordinate of the area to be processed, σ y is the vertical projection change factor, H is the height of the input image of the area to be processed; By changing the horizontal projection change factor and the vertical projection change factor respectively, an adjusted image is obtained.

4. The image distortion correction processing method for book detection according to claim 1, characterized in that: In step S1, the camera image is positioned to the bookshelf partition, and the acquisition point parameters of the camera are pre-configured, which specifically includes the following sub-steps: S11, dividing a shelf into a plurality of partitions according to the positions of the camera and the bookshelf, with two adjacent partitions partially overlapping; S12, adjusting the camera position to one of the partitions so that the partition is placed in the center area of ​​the camera screen, and recording the current horizontal coordinate and vertical coordinate of the camera; S13, configuring the magnification of the camera so that the camera image only includes the books in the current bookshelf partition; S14, configuring the focal length of the camera to adjust the focal length so that the text on the spine of the book is clear in the camera image.

5. The image distortion correction processing method for book detection according to claim 1, characterized in that: In step S2, the real-time image is marked with an identification area by drawing a polygon to obtain an area to be processed that needs to be OCR recognized, including: drawing a polygon in the real-time image, and clicking the polygon vertices or dragging the edges with a mouse to draw a closed polygon as the area to be processed.

6. The image distortion correction processing method for book detection according to claim 2, characterized in that: Also includes: Keeping the marked area of ​​the previous partition of the bookshelf on the camera image layer until a clearing operation is performed; If the marked area of ​​the previous partition only covers the books of the current partition, the area coordinate points of the previous partition are directly used for correction and saving operations; If the marked area of ​​the previous partition also covers part of the data of the spines of the remaining shelves, the area marking operation is repeated and then the correction and saving operations are performed.

7. An image distortion correction processing system for book detection, characterized in that: The system comprises: An acquisition module is configured to locate the camera image to the bookshelf partition, pre-configure the acquisition point parameters of the camera, and acquire a real-time image through the camera, wherein the acquisition point parameters include: a zoom factor, a focal length, a horizontal coordinate, and a vertical coordinate of the camera; A marking module is configured to mark the recognition area of ​​the real-time image by drawing polygons to obtain the area to be processed that needs to be OCR recognized; A correction module is configured to perform spine text correction processing on the area to be processed, wherein the spine text correction processing includes: adjusting a horizontal projection change factor and a vertical projection change factor to make the spine text present a vertical state; The storage module is configured to store the current configuration information of the camera and the parameters related to the spine text correction processing for subsequent OCR recognition.

8. A terminal device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory, wherein the computer program is executed by the processor to implement the image distortion correction processing method for book detection as claimed in any one of claims 1 to 6.

9. A computer-readable storage medium, wherein a computer program is stored in the medium, and when the computer program is executed by a processor, the image distortion correction processing method for book detection as claimed in any one of claims 1 to 6 is implemented.

Images