A certificate reading all-in-one machine and a certificate reading method based on double cameras

Abstract

The specification discloses a certificate reading integrated machine and a certificate reading method based on double cameras, and the scheme comprises the following steps: in an outdoor mode, a main camera picture is collected in real time, brightness detection is performed, and the brightness detection is compared with a threshold value to adjust exposure parameters and signal gain, wide dynamic optimization is enabled, stability calculation is performed, four-point corner detection is performed after a suspected certificate area is detected and stabilized, and photographing collection is triggered after certificate standard placement is matched and determined; meanwhile, the main and auxiliary cameras are controlled to be synchronously collected and independently perform barcode detection, the auxiliary camera monitors a moving object to start long-distance barcode scanning, and the main camera synchronously detects a barcode in a window area before photographing is performed; a user's preset merging, independent or mutually exclusive mode is acquired, and barcode information and certificate OCR information are output according to the mode. The application realizes stable certificate imaging under complex outdoor illumination, far and near barcode non-blind area identification and flexible information output mode, and can improve the adaptability and efficiency of certificate reading.

Description

Technical Field

[0001] This invention relates to the field of document recognition technology, and more specifically, to an all-in-one document recognition machine and its document recognition method based on dual cameras. Background Technology

[0002] In the field of document recognition technology, document recognition equipment is widely used in government service halls, financial institutions, airports, train stations, hotels, and guesthouses to register and verify customer identity information. Existing document recognition equipment mainly relies on Optical Character Recognition (OCR) and Radio Frequency Identification (RFID) technologies to collect document information. Its hardware solutions often adopt a modular assembly model, with core computing modules and peripherals (cameras, card readers) mostly being outsourced and integrated. In terms of environmental adaptability, existing equipment is primarily designed for indoor use, specifically for standardized indoor lighting environments. Most devices lack automatic exposure adjustment and wide dynamic range optimization functions, making it difficult to adapt to complex outdoor lighting conditions such as strong light, weak light, and backlight. This results in overexposed images or loss of detail in outdoor scenes, significantly reducing recognition accuracy and limiting its application scenarios. Regarding barcode recognition, most existing devices are equipped with only a single camera, which cannot simultaneously handle depth of field and field of view. Either they can only recognize barcodes close to the device window, failing to recognize distant barcodes; or they have a large depth of field but a narrow field of view, making it difficult to completely recognize large areas or numerous barcodes, resulting in blind spots. In addition, existing devices have a fixed output mode for barcode information and document OCR information, with only two modes: single independent output or combined output. They lack a flexible switching mechanism, which cannot meet the information merging processing requirements when the document has its own barcode, nor can they adapt to scenarios where two types of information need to be identified separately and their mutual interference should be avoided. Their functional flexibility is insufficient. Summary of the Invention

[0003] This specification provides an integrated document reader and a document reading method based on dual cameras to overcome at least one technical problem existing in related technologies.

[0004] According to a first aspect of the embodiments of this specification, a document reading all-in-one machine is provided, comprising: case; The main camera, mounted on the housing, is used to capture images of identification documents and barcodes that are affixed to the window area of ​​the device; A secondary camera, mounted on the housing, is used to capture barcode images at a distance greater than the distance of the area attached to the device window center. A controller is electrically connected to the main camera and the secondary camera respectively. The controller includes at least one processor and a memory storing a computer program. When the computer program is executed by the processor, it implements the following modules: The environment adaptation module is used to respond to the device being in outdoor mode, to obtain the brightness value of the output image of the main camera in real time, compare the brightness value with a preset threshold, and adjust the exposure parameters and signal gain of the main camera in real time according to the comparison result, and enable wide dynamic range technology to optimize the image. The photo-taking trigger module is used to perform real-time stability calculations on each frame of the main camera after it has been adjusted by the environment adaptation module. When a target area that appears to be an ID card is detected in the frame and the stability reaches the set standard, the four corner points of the ID card are detected in the target area. Based on the corner point matching results, it is determined whether the target is a properly placed ID card. If so, the photo-taking and recognition operation is triggered. The barcode recognition module is used to control the main camera and the secondary camera to synchronously capture images in real time and to independently perform barcode detection operations. The secondary camera is used to start frame-by-frame barcode scanning when a moving object is detected in the image. The main camera is used to synchronously perform barcode detection on the area of ​​the device window before executing the document photo recognition and judgment process. The information output module is used to output the barcode information recognized by the barcode recognition module and the document optical character recognition information triggered by the photo triggering module according to a preset merging mode, independent mode or mutually exclusive mode.

[0005] In some optional implementations, the information output module includes: The merging unit is used to combine barcode information and document optical character recognition information into a data packet and output it when barcode information and document optical character recognition information are detected simultaneously within a set time. Independent unit, used to output the corresponding data separately when barcode information or optical character recognition information of certificate is detected separately, and to make the transmission and processing of the two types of data independent of each other; The mutual exclusion unit is used to pause the recognition and output of the other type of information for a set time period after detecting either barcode information or optical character recognition information of a certificate, and to resume normal recognition after the time period ends.

[0006] In some alternative implementations, the depth of field of the secondary camera is greater than that of the primary camera.

[0007] In some alternative implementations, the environment adaptation module is also configured to not perform image brightness value comparison, exposure parameter and signal gain adjustment operations as in outdoor mode in response to the device being in indoor mode.

[0008] According to a second aspect of the embodiments of this specification, a document recognition method based on dual cameras is provided, applied to the document recognition all-in-one machine described in any of the preceding descriptions, comprising the following steps: In response to the device being in outdoor mode, the output image of the main camera is captured in real time. The brightness of each captured frame is detected sequentially to obtain the image brightness value. The image brightness value is compared with the preset image expected brightness threshold, and the exposure parameters and signal gain of the main camera are adjusted in real time according to the comparison result. Wide dynamic range technology is enabled to optimize the image. Then, the stability of each optimized frame is calculated in real time. When a target area that resembles an ID card is detected in the image and the image stability reaches the set standard, the four corner points of the ID card are detected in the target area. The corner point matching is used to determine whether the target is a properly placed ID card. If it is determined to be an ID card, the photo recognition operation is triggered to complete the ID card image acquisition. While the main camera performs the document photo recognition and judgment process, the main camera and the secondary camera are controlled to synchronously capture images in real time and independently carry out barcode detection. The secondary camera continuously monitors the dynamics of the image. When a moving object is detected in the image, frame-by-frame barcode scanning and detection is initiated to identify barcodes of targets in front of the device window that are at a distance. The main camera synchronously performs barcode detection on the area attached to the device window before performing the document photo recognition and judgment process. The system acquires the user-preset merge mode, independent mode, or mutually exclusive mode as the information output mode, and outputs the recognized barcode information and document optical character recognition information according to the acquired information output mode.

[0009] In some optional implementations, the step of outputting the identified barcode information and document optical character recognition information according to the acquired information output pattern includes: When the information output mode is the merging mode, if barcode information and document optical character recognition information are detected simultaneously within a set time, the two types of data will be integrated into a data packet and output. When the information output mode is independent mode, if barcode information or optical character recognition information of certificate is detected separately, the corresponding data will be output separately, and the transmission and processing of the two types of data will be independent of each other. When the information output mode is a mutual exclusion mode, if either barcode information or optical character recognition information of a certificate is detected, the recognition and output of the other type of information will be suspended for a set time period, and the normal recognition state will be restored after the time period ends.

[0010] In some alternative implementations, after the step of performing real-time stability calculations on each optimized frame, the following step is also included: If the image stability does not meet the set standard, the image recognition operation will not be triggered, and the stability calculation will continue for the next frame.

[0011] In some optional implementations, the step of determining whether the target is a properly placed document through corner matching includes: The four detected corner points are matched with the preset document corner point standards, and the current target is determined to be a properly placed document based on the matching results.

[0012] In some alternative implementations, the following steps are also included: In response to the device being in indoor mode, the steps of performing brightness detection on each frame and comparing the brightness value of the frame with the preset expected brightness threshold are not performed; instead, the document image is directly captured.

[0013] In some optional implementations, the method further includes dynamically predicting the optimal time to take a picture of the document based on the sequence of corner position changes in multiple consecutive frames captured by the main camera before triggering the photo recognition operation. Specifically, this includes: Extract the coordinate sequence of the four corner points of the document in the continuous multi-frame images, and calculate the coordinates of the center point of the document and the plane rotation angle in each frame; The motion trajectory of the document in the image plane is fitted based on the center point coordinate sequence. A Kalman filter is used to smooth and predict the motion trajectory, and the predicted position of the center point of the document in the next frame is estimated. The angular velocity of the document is calculated based on the planar rotation angle sequence. When the angular velocity is lower than the preset angular velocity threshold and the deviation between the predicted position and the center point of the current frame is less than the preset displacement threshold, it is determined that the document is about to enter the optimal static state. When the next frame, which is determined to be about to enter the optimal static state, arrives, the photo recognition operation is triggered in advance, so that the delay between the photo capture time and the actual static time of the document is less than one image acquisition cycle.

[0014] In some optional implementations, the continuous monitoring of the scene dynamics via the secondary camera, and the initiation of frame-by-frame barcode scanning detection when a moving object is detected in the scene, further includes adaptively adjusting the parameters of the barcode decoding engine based on the direction and speed of the moving object's movement, specifically including: When the secondary camera detects a moving object in the frame, it extracts the motion vector of the moving object and calculates its approach speed and approach angle relative to the center point of the device window. If the approach speed exceeds the preset high-speed threshold, it is determined to be a fast scanning barcode recognition scenario. The barcode decoding engine is automatically switched from global search mode to dynamic tracking mode, and the decoding timeout is shortened to the first preset duration. If the approach speed is lower than the preset low speed threshold and the angle between the approach angle and the normal direction of the device window is less than the preset angle, it is determined to be a stable barcode recognition scenario. The barcode decoding engine is automatically switched to high-precision global decoding mode, and the decoding timeout is extended to a second preset duration, which is greater than the first preset duration. At the same time, the positive or negative value of the approach speed determines whether the moving object is approaching or moving away. If it is determined to be moving away, the current barcode scanning detection is stopped and the decoding cache is cleared to release computing resources.

[0015] In some optional implementations, the method further includes time-synchronized fusion of the pre-triggered image recognition operation with the parameters of the adaptively adjusted barcode decoding engine to form a multimodal collaborative acquisition decision, specifically including: When the pre-capture trigger signal based on corner motion prediction is generated, the mode and timeout parameters of the barcode decoding engine after adaptive adjustment are simultaneously acquired. Calculate the time difference between the generation time of the pre-capture trigger signal and the time when the secondary camera detects a change in the direction of movement of a moving object. If the time difference is less than a preset synchronization threshold, it is determined that the ID card and the barcode belong to the same user interaction behavior. After determining that they are the same interactive behavior, the exposure time of the main camera is aligned with the barcode decoding and scanning frame of the secondary camera, so that the difference in timestamp between the document image captured by the main camera and the barcode information recognized by the secondary camera is less than half an image acquisition cycle. The aligned document image and barcode information are merged and output to the information output module, and a synchronization confidence field is added to the merged data packet. The value of the synchronization confidence field is negatively correlated with the magnitude of the time difference.

[0016] The beneficial effects of the embodiments in this specification are as follows: In this technical solution, by equipping the document reader with dual cameras (primary and secondary), and based on automatic exposure and wide dynamic range optimization, real-time stability calculation, and four-corner detection in outdoor mode, fully automatic quality control of document imaging can be achieved. Furthermore, instead of the crude approach of using a single camera which cannot adequately handle depth of field, distinguish between indoor and outdoor modes, or use a fixed output mode, this solution utilizes simultaneous acquisition and independent detection by dual cameras. The secondary camera continuously monitors the dynamics of the image to identify moving barcodes at a distance, while the primary camera simultaneously detects the barcode attached to the window before taking the document photo. Simultaneously, based on user-preset information output modes—merging, independent, or mutually exclusive—it flexibly adapts to different scenario requirements, such as merging the barcodes on the document itself, performing separate statistics, or using mutual exclusion to prevent interference. By incorporating interference factors such as sudden changes in outdoor brightness, hand shake, and document tilt into frame-by-frame adaptive adjustment and stability judgment, and by skipping the brightness adjustment step directly in indoor mode, the entire reading process can automatically optimize imaging parameters and accurately trigger photography under complex lighting conditions. It can also achieve blind-spot-free collaborative recognition of barcodes at both near and far distances. The output mode can be dynamically switched according to different business logics. Thus, this solution can achieve intelligent, adaptive, and efficient reading of document information and barcode information without manual intervention or increased operational burden. It can effectively solve the problems of poor outdoor imaging quality, limited barcode recognition range, and rigid output modes of existing equipment, while improving the reusability of multiple scenarios and reducing information processing costs. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments or related technologies of this specification, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic diagram of the main structure of an integrated document reader provided for the technical solution of this application; Figure 2 The technical solution in this application is based on Figure 1 The flowchart illustrates a document reading method based on dual cameras, derived from the document reading all-in-one machine provided in the document reader.

[0019] In this context, 1 represents the main camera and 2 represents the secondary camera. Detailed Implementation

[0020] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] To address the shortcomings of existing document reading devices, such as poor image quality under complex outdoor lighting conditions, the inability of a single camera to simultaneously recognize barcodes at different distances, and rigid information output modes, this application proposes a document reading method and device that integrates environment-adaptive imaging, dual-camera barcode collaborative recognition, and multi-mode information output. Specifically, this application first establishes dual indoor and outdoor working modes. In outdoor mode, the main camera image is captured in real time, and exposure parameters and signal gain are adjusted based on a closed-loop brightness detection system. Wide dynamic range technology is used to optimize details in both bright and dark areas. Simultaneously, image stability calculation and four-point corner detection of the document are introduced to ensure clear image capture of standard documents even under adverse conditions such as shaking and backlighting. In indoor mode, brightness adjustment is skipped, and image capture is performed directly to reduce latency. Secondly, this application configures a main and secondary dual-camera system with complementary depth of field and field of view. The secondary camera continuously monitors the dynamics of the image, initiating long-distance barcode scanning when a moving object appears. The main camera simultaneously detects the barcode in the window area before document capture, thereby eliminating blind spots in barcode recognition. Finally, this application designs three switchable information output modes: merge, independent, and mutually exclusive, enabling the device to flexibly merge document information and barcode information, output the two types of information independently, or mutually exclude them to prevent interference, depending on the business scenario.

[0022] Based on the above technical concept, the following section provides a detailed description of the document reading all-in-one machine provided in this application, combining specific hardware structure and software modules, such as... Figure 1 As shown, Figure 1 A schematic diagram of the main structure of an integrated document reader provided for the technical solution of this application, the integrated device may include: Casing (not labeled with numbers in the diagram); Main camera 1, mounted on the housing, is primarily used to capture images of documents and barcodes that are attached to the device window area (i.e., objects pressed against the glass panel). Its recognition range is within... Figure 1 The text is marked with a set of lines. Secondary camera 2, mounted on the housing, has a greater depth of field than the main camera 1, such as... Figure 1 As shown, the secondary camera 2 can be positioned to the side of the main camera 1, with a certain physical distance between them, allowing their fields of view to cover different distance ranges respectively, and their recognition range is within... Figure 1Another set of lines marks the location of the barcode. The secondary camera 2 is used to capture barcode images at a distance greater than the distance between the center of the device window and the actual window area. Figure 1 The symbol "A" indicates the distance between the target to be identified, which can be clearly imaged by the secondary camera 2, and the center of the device window (e.g., a few centimeters to tens of centimeters). A controller is electrically connected to the main camera and the secondary camera respectively. The controller includes at least one processor and a memory storing a computer program. When the computer program is executed by the processor, it implements the following modules: The environment adaptation module is used to respond to the device being in outdoor mode, to obtain the brightness value of the output image of the main camera in real time, compare the brightness value with a preset threshold, and adjust the exposure parameters and signal gain of the main camera in real time according to the comparison result, and enable wide dynamic range technology to optimize the image. The photo-taking trigger module is used to perform real-time stability calculations on each frame of the main camera after it has been adjusted by the environment adaptation module. When a target area that appears to be an ID card is detected in the frame and the stability reaches the set standard, the four corner points of the ID card are detected in the target area. Based on the corner point matching results, it is determined whether the target is a properly placed ID card. If so, the photo-taking and recognition operation is triggered. The barcode recognition module is used to control the main camera and the secondary camera to synchronously capture images in real time and to independently perform barcode detection operations. The secondary camera is used to start frame-by-frame barcode scanning when a moving object is detected in the image. The main camera is used to synchronously perform barcode detection on the area of ​​the device window before executing the document photo recognition and judgment process. The information output module is used to output the barcode information recognized by the barcode recognition module and the document optical character recognition information triggered by the photo triggering module according to a preset merging mode, independent mode or mutually exclusive mode.

[0023] In this embodiment, the housing serves as the outer shell of the entire device and can be made of metal or high-strength engineering plastic, housing components such as the camera, controller, and circuit board. A camera window is provided on the housing, with the main camera and secondary camera mounted in different positions. The main camera can be positioned slightly below the center of the housing, while the secondary camera is positioned above or to the side of the main camera, with a certain physical distance between them, allowing their fields of view to cover different distance ranges. The main camera's field of view is designed to completely capture the area adjacent to the device window (i.e., objects pressed against the glass panel), while the secondary camera's field of view covers a more distant area. The housing can also integrate auxiliary components such as supplementary lighting (LEDs or infrared lights), dedicated lighting for the optical character recognition module, a buzzer, and indicator lights to enhance the user experience.

[0024] The controller is the core computing unit of the device. The processor can be an embedded processor based on the ARM architecture, such as the Rockchip RK3588. This chip has powerful image processing capabilities (supporting multiple camera inputs and hardware encoding / decoding) and a neural network acceleration unit (NPU), capable of processing data streams from two cameras simultaneously without significant latency. The memory can be eMMC flash memory (for storing the operating system, applications, and configuration data) or LPDDR memory (for runtime data caching). The stored computer program contains the implementation code for each of the aforementioned modules. When the device powers on, the processor loads the program from the memory and executes it, instantiating the environment adaptation module, the photo triggering module, the barcode recognition module, and the information output module. Furthermore, the controller can integrate other communication interfaces, such as USB, Ethernet, serial ports (RS232 / RS485), Wi-Fi, and Bluetooth, for data exchange with a host computer or the cloud.

[0025] The core function of the environment adaptation module is to monitor ambient brightness in real time and dynamically adjust imaging parameters. It obtains brightness values ​​by reading the main camera's automatic exposure statistics (e.g., average brightness in 3A statistics) or directly analyzing image pixel values ​​(e.g., calculating the main peak position of the image histogram). These values ​​are then compared with internally stored preset thresholds to calculate the required exposure time, analog gain, and digital gain adjustment. This information is then written to the camera register via the camera driver interface (e.g., the s_ctrl command in the Linux V4L2 framework). Simultaneously, it can control the camera's wide dynamic range mode (e.g., setting the WDR level to high, medium, low, or automatic). The image capture triggering module is responsible for running stability calculation and corner detection algorithms. These algorithms can be implemented using open-source computer vision libraries such as OpenCV, or lightweight algorithm libraries optimized for embedded platforms. The barcode recognition module integrates mainstream barcode decoding libraries (such as ZXing or ZBar), capable of recognizing various common barcode types such as QR codes, PDF417, Code128, EAN-13, and UPC-A, and supports both 1D and 2D barcodes. The information output module sends the recognition results (JSON format or custom protocol) to the host computer or cloud server via serial port, USB or network interface, and supports dynamically changing the output content according to preset mode.

[0026] In an optional embodiment, the information output module may include: The merging unit is used to combine barcode information and document optical character recognition information into a data packet and output it when barcode information and document optical character recognition information are detected simultaneously within a set time. Independent unit, used to output the corresponding data separately when barcode information or optical character recognition information of certificate is detected separately, and to make the transmission and processing of the two types of data independent of each other; The mutual exclusion unit is used to pause the recognition and output of the other type of information for a set time period after detecting either barcode information or optical character recognition information of a certificate, and to resume normal recognition after the time period ends.

[0027] This embodiment corresponds to the information output mode implementation in the aforementioned method, and is further explained from the perspective of the device. The merging unit, independent unit, and mutual exclusion unit can be three independently running software threads or state machines in the controller, sharing the data queues of barcode recognition results and document optical character recognition results. The merging unit maintains a short-timer and a data buffer. When either the barcode recognition module or the document optical character recognition module detects valid information, the merging unit starts the timer and stores the information in the buffer, while simultaneously recording the arrival timestamp of the information. If another module also detects valid information before the timer expires, the merging unit merges the two into a single data packet and sends it through the output interface. The independent unit does not perform any cross-type data association; it immediately outputs what it detects through its respective channel. The two channels can be connected to different downstream processing systems (e.g., the barcode channel connects to the gate control system, and the document channel connects to the public security database interface). The mutual exclusion unit internally maintains a mutual exclusion flag and a timer. When a type of information is detected, it checks the mutual exclusion flag; if it is 0, it outputs the information, sets the flag to 1, and starts the timer; if it is 1, it discards the current information. The flag is reset after the timer expires. These three units can be selectively activated according to the output mode set by the user through configuration files or serial port commands.

[0028] In an optional embodiment, the depth of field of the secondary camera is greater than that of the primary camera.

[0029] This embodiment further defines the optical characteristics of the dual cameras. The difference in depth of field can be achieved by selecting camera modules of different specifications. For example, the main camera can use a module with a fixed focal length of 4.0 mm and an aperture of F2.0. Its hyperfocal distance is relatively large, and the depth of field ranges from approximately 0.5 meters to infinity. However, when imaging at close range (0-5 cm), because the object distance is much smaller than the hyperfocal distance, the depth of field can still cover the area close to the window. The secondary camera can use a module with a fixed focal length of 2.8 mm and an aperture of F2.4. Its depth of field range is even wider, maintaining sharpness from a few centimeters to tens of centimeters. Alternatively, an autofocus camera can be used, and different depth-of-field effects can be achieved by controlling the position of the focusing motor; that is, the main camera focuses at close range (e.g., 3 cm), and the secondary camera focuses at distant range (e.g., 10 cm). Regardless of the specific implementation method, as long as the depth of field range of the secondary camera is greater than that of the main camera, and the furthest distance at which the secondary camera can achieve sharp imaging is greater than that of the main camera, the scope of protection of this application is satisfied. The specific value of the depth of field depends on the specific optical design, and this application does not limit it.

[0030] In an optional embodiment, the environment adaptation module is further configured to not perform image brightness value comparison, exposure parameter and signal gain adjustment operations in outdoor mode when the device is in indoor mode.

[0031] This embodiment clarifies the behavior of the environment adaptation module in indoor mode. In indoor mode, the environment adaptation module no longer calls the brightness detection and exposure parameter adjustment functions, but instead sets the exposure parameters of the main camera to fixed default values. These default values ​​are calibrated before the device leaves the factory and can adapt to most indoor lighting conditions (typical illuminance 300-500 lux). If the user finds the imaging effect unsatisfactory in certain special indoor scenarios (such as a particularly dimly lit basement or a particularly brightly lit exhibition hall), these default parameters can be manually adjusted through the device configuration tool (e.g., increasing exposure compensation or forcibly turning on the fill light). Through this design, this application balances the adaptability in complex outdoor environments with high efficiency and stability in indoor environments. In addition, the wide dynamic range function can be selectively turned off in indoor mode, because indoor lighting is usually uniform and wide dynamic range synthesis is not required, thereby further reducing processing latency.

[0032] The preceding text has provided a detailed description of the device claims in this application, combining hardware structure and software modules, revealing the composition architecture of the integrated document reader and the functions of each module. The following will further elaborate on the specific process of the document reading method executed based on this device. This method fully utilizes the dual-camera configuration (main and secondary), environment adaptation and photo triggering mechanism, barcode collaborative recognition strategy, and information output mode in the aforementioned device, achieving a complete reading process from image acquisition to information output through a series of steps. Each step is described in detail below with reference to the accompanying drawings.

[0033] See appendix Figure 2 This application provides a document recognition method based on dual cameras, which is applied to the aforementioned document recognition all-in-one machine. The method includes the following steps.

[0034] Step 202: In response to the device being in outdoor mode, the output image of the main camera is captured in real time. The brightness of each captured frame is detected sequentially to obtain the image brightness value. The image brightness value is compared with the preset expected image brightness threshold. Based on the comparison result, the exposure parameters and signal gain of the main camera are adjusted in real time. Wide dynamic range technology is enabled to optimize the image. Then, the stability of each optimized frame is calculated in real time. When a target area resembling an ID document is detected in the image and the image stability reaches the set standard, the four corner points of the ID document are detected in the target area. The corner point matching is used to determine whether the target is a properly placed ID document. If it is determined to be an ID document, the photo recognition operation is triggered to complete the ID document image acquisition.

[0035] In the embodiments of this specification, outdoor mode refers to the working state of the device in an outdoor environment. Compared with the standardized indoor lighting environment, the outdoor environment has a variety of complex lighting conditions such as direct sunlight, backlight, shadows, and low light. Traditional fixed exposure parameters cannot adapt to such drastic changes. Therefore, this application first captures the output image of the main camera in real time in outdoor mode. The main camera is the main camera used to take ID photos and recognize barcodes attached to the window area, and each frame of its output image is sent to the subsequent processing flow.

[0036] Brightness detection refers to the quantitative calculation of the overall or local brightness of each frame to obtain a brightness value, which can be compared with a preset expected brightness threshold. The expected brightness threshold is a reasonable brightness range boundary value obtained statistically from a large number of outdoor normal lighting samples. When the actual brightness value is lower than this threshold, it indicates that the ambient light is too dark, requiring increased exposure parameters and signal gain. When the actual brightness value is higher than this threshold, it indicates that the ambient light is too strong, requiring decreased exposure parameters and signal gain. Through this closed-loop feedback, the camera can automatically adjust the exposure parameters within tens of milliseconds, ensuring that subsequent images are not overexposed or underexposed. The exposure parameters specifically include exposure time (shutter speed) and analog gain (ISO), while signal gain refers to digital gain; all three together determine the final image brightness level.

[0037] Meanwhile, this application also employs wide dynamic range (WDR) technology, an image processing technique that can simultaneously preserve details in both bright and dark areas within the same frame. In outdoor backlit scenes, the surface of an ID card may be half illuminated by strong light and half in shadow; ordinary imaging would result in bright areas being completely white and dark areas being completely black. With WDR technology enabled, the camera captures and synthesizes images multiple times at different exposure times, thereby balancing the display effect of bright and dark areas in the image, ensuring that the text, photo, and barcode on the ID card are all clearly visible. WDR technology can employ a multi-exposure synthesis method or a single-frame digital WDR algorithm; this application does not limit the choice between the two.

[0038] After basic imaging optimization, the device algorithm performs real-time stability calculations on each frame. The purpose of stability calculation is to determine whether the current image is static, avoiding blurry photos caused by the user holding the document shakily or the device itself shaking. Stability can be comprehensively evaluated using parameters such as pixel differences between adjacent frames and feature point displacement. For example, the root mean square error of grayscale values ​​between two consecutive frames can be calculated; when this error is less than a preset threshold, the image is considered stable. The algorithm only proceeds to the next step, corner detection, when a target area resembling an document (such as a rectangular area, specific reflective features, etc.) appears in the image and the image stability reaches the preset stability threshold. This design ensures that the image is in an optimal static state at the moment of taking the photo.

[0039] Identification documents typically have four prominent corners. Detecting these corners helps determine the document's precise position and orientation within the image. This application employs a four-point detection algorithm to search for pixels with corner features in the target area, forming a candidate corner set. These corners can be Harris corners, Shi-Tomasi corners, or corner detection results based on deep learning. Then, corner matching is used to determine if these corners can form a standard rectangular or approximately rectangular document outline. Matching can refer to the document's standard aspect ratio (e.g., an ID card is 85.6mm × 54mm, with an aspect ratio of approximately 1.585) and geometric constraints of perspective projection (e.g., parallel opposite sides or intersection at a hidden point). Only when corner matching is successful, indicating that the document is correctly positioned in front of the camera, will the image recognition operation be triggered.

[0040] Once the photo recognition operation is triggered, the device immediately captures the current frame as the document image and hands it over to the subsequent optical character recognition module for text extraction and analysis, ultimately completing the document image acquisition and preliminary processing. This series of processes ensures that the device can automatically adapt to the environment, stabilize imaging, and accurately capture documents even under complex outdoor lighting conditions, avoiding manual intervention and repeated shooting.

[0041] Step 204: While the main camera is performing the document photo recognition and judgment process, the main camera and the secondary camera are controlled to synchronously capture images in real time and independently carry out barcode detection operations. The secondary camera continuously monitors the dynamics of the image. When a moving object is detected in the image, frame-by-frame barcode scanning and detection is initiated to identify barcodes of targets in front of the device window that are within a long distance. The main camera synchronously performs barcode detection on the area attached to the device window before performing the document photo recognition and judgment process.

[0042] In the embodiments described in this specification, the barcode recognition function is performed collaboratively by a main camera and a secondary camera. The main camera is mounted on the housing and typically has a large field of view, capable of completely capturing barcodes that are attached to the device window area (e.g., ID card barcodes or paper barcodes attached to the device's glass panel). The secondary camera, on the other hand, has a large depth of field, enabling it to clearly image targets that are far from the center of the device window (e.g., barcodes within a range of several centimeters to tens of centimeters). The two cameras have clearly defined roles and complement each other.

[0043] In practice, the main camera and the secondary camera simultaneously capture images in real time, each running its barcode detection algorithm independently. Simultaneous capture means both cameras expose at the same frame rate (e.g., 30 frames per second), ensuring they capture the scene at the same moment, facilitating subsequent data alignment. The secondary camera's core task is to continuously monitor the dynamics of the image. It determines whether a moving object is present by comparing differences between consecutive frames. For example, when a user brings a barcode-printed ticket or baggage tag close to the device window but not fully against it, the secondary camera can capture the object's trajectory as it gradually approaches from a distance, thus initiating frame-by-frame barcode scanning. This motion-triggered mechanism avoids the secondary camera constantly performing high-intensity full-frame scanning, reducing the computational overhead of the central processing unit. The secondary camera performs barcode recognition on targets within a certain range in front of the device window. This range is larger than the main camera's close-range recognition range, covering common distances at which users hold barcodes (e.g., 5-10 centimeters).

[0044] Simultaneously, before executing the document photo recognition and judgment process (i.e., corner detection and photo triggering in step 202), the main camera also performs barcode recognition. However, the barcode recognition range of the main camera is limited to the area attached to the device window. This is because although the main camera has a wide field of view, its depth of field is relatively shallow, making it more suitable for detailed imaging of close-up objects. When the user places their document or barcode directly on the device window, the main camera can quickly and accurately read the barcode information. The main camera's barcode detection before executing the document photo recognition and judgment process means that barcode detection and corner detection are performed in parallel or alternately, without blocking each other.

[0045] By employing this dual-camera parallel design, this application can recognize barcodes on moving objects at a distance (such as a user holding a ticket close to the device) as well as barcodes attached to a window (such as barcodes on the back of an ID card), thus solving the problem that a single camera cannot simultaneously handle depth of field and field of view.

[0046] Step 206: Obtain the user-preset merge mode, independent mode or mutually exclusive mode as the information output mode, and output the recognized barcode information and document optical character recognition information according to the obtained information output mode.

[0047] In the embodiments of this specification, the barcode information recognized by the barcode recognition module and the document information recognized by the document optical character recognition module need to be output according to the user's actual needs. Different application scenarios have different requirements for the processing methods of these two types of information. Therefore, this application designs three switchable information output modes.

[0048] The merge mode is suitable for scenarios where the document itself contains barcode information. For example, some ID cards, passports, or driver's licenses have machine-readable barcodes printed on them. These barcodes corroborate the text information on the document, and merging them simplifies the processing logic of downstream systems. In merge mode, if the device detects both barcode information and document optical character recognition information simultaneously within a set time period (e.g., 1 second), it integrates these two types of data into a single data packet before outputting it. The data packet can use JSON, XML, or a custom binary format and includes fields such as information type identifier, barcode content, document name, and document number. After receiving this data packet, the downstream system can directly compare and store the information without having to manually associate two separate records. If only one type of information is detected within the set time period, the system can choose to output the single piece of information or ignore it, depending on the configuration.

[0049] Independent mode is suitable for scenarios where barcode information and ID card information need to be counted and processed separately. For example, in a library's self-service borrowing and returning machine, the optical character recognition information of the reader's card and the book's barcode information are independent and should be output separately for entry into the reader system and the book borrowing system, respectively. In independent mode, when the device detects barcode information or ID card optical character recognition information separately, it directly outputs the corresponding data separately. The transmission and processing of the two types of data are independent and do not interfere with each other. In specific implementation, different communication ports, different data frame headers, or different output queues can be assigned to the two types of information.

[0050] The mutual exclusion mode is suitable for scenarios such as airport security checks and boarding gates. Passengers need to recognize both passports (optical character recognition) and boarding pass barcodes. However, if these two types of information are recognized simultaneously, it may cause information cross-interference, such as mistaking a passport barcode for a boarding pass barcode, or mistaking text on a boarding pass for passport information. In mutual exclusion mode, after the device detects either a barcode or a document, it pauses the recognition and output of the other type of information for a set time period (e.g., 5 seconds). For example, if the device recognizes a passport first, it will not respond to any barcode recognition results for the next 5 seconds. If the device recognizes a boarding pass barcode first, it will not perform document optical character recognition for the next 5 seconds. After the time period ends, normal recognition resumes. This mode can effectively avoid duplicate information recognition and interference, improving the user experience.

[0051] In this technical solution, users can preset the required information output mode according to the actual application scenario through the device's configuration interface, serial port commands, network API, or configuration files. The device reads this configuration at startup and it remains in effect during operation.

[0052] It's important to note that the three output modes—merge, independent, and mutually exclusive—are not automatically recognized and switched based on the specific type of user's identification document (such as ID card, passport, driver's license, etc.). Instead, they are preset by the administrator or user before the device is used, based on the business scenario of the device deployment. For example, in an airport boarding gate scenario, where passport OCR and boarding pass barcodes need to be processed simultaneously and interference should be avoided, the preset mode should be mutually exclusive. In a library self-service borrowing and returning scenario, where reader card OCR and book barcodes belong to different objects, the preset mode should be independent. In scenarios where barcodes are read on the document itself (such as some electronic passports or ID cards with barcodes), the preset mode should be merged. The device always operates according to this preset mode during operation and will not automatically change based on the type of identification document presented by the user. Therefore, even if the type of identification document presented by the user is unknown in advance, there is no need for real-time determination, as the device has fixed the output rules according to the on-site business logic. Regardless of the preset output mode, the main camera and the secondary camera work normally as described above and are not affected by mode switching. Specifically: the main camera is always responsible for document imaging (including environmental adaptation in outdoor mode, stability calculation, corner detection, and triggering photo capture) and barcode detection within the device window area; the secondary camera is always responsible for continuously monitoring the dynamics of the scene and initiating long-distance barcode scanning when a moving object appears. The three output modes only determine the combination logic of barcode information and document OCR information during output (merging into a group of data packets, independent branch output, or mutually exclusive suspension of another type of output), without shutting down or changing the image acquisition and recognition process of either camera. In other words, the hardware collaboration of the two cameras is the foundation, and the information output mode is the upper-level strategy; the two are decoupled.

[0053] Taking airport boarding gate scenarios as an example, traditional solutions typically use a single camera, which struggles to balance depth of field and field of view. For instance, to read the text information on a passport, the passport needs to be held close to the window; if the boarding pass barcode is at a distance, it won't be clearly imaged. Adjusting the camera parameters to accommodate distant barcodes reduces the image resolution of the passport area. More commonly, passengers need to scan the boarding pass barcode on a separate device before scanning their passport on this device. This step-by-step process is not only inefficient but also prone to information mismatch due to the incorrect order of operations. Even integrating both recognition functions into a single device, simultaneous operation can easily lead to logical interference, such as the machine-readable code on the passport being misinterpreted as the boarding pass barcode, or the passenger's name on the boarding pass being mistaken for document OCR information, or even cross-contamination between data from different passengers. This technical solution resolves the spatial conflict at its source by physically dividing the tasks of the main and secondary cameras. The main camera is dedicated to passport imaging and OCR recognition within the window area, with its optical path and depth of field optimized for close-range documents. The secondary camera is responsible for dynamic monitoring at a greater distance (e.g., up to 10cm from the center of the window), continuously capturing the user's movement as they approach with their boarding pass and performing barcode scanning. Both cameras simultaneously capture images in real time, detecting independently without interfering with each other. Furthermore, the device internally maintains a mutual exclusion flag and a timer to forcibly isolate the two types of recognition tasks along the timeline. Specifically, when the main camera detects the passport first and triggers OCR recognition, the mutual exclusion flag is immediately set, and then the secondary camera's barcode recognition output is forcibly paused for a set time period (e.g., 5 seconds). Even if the secondary camera captures the boarding pass barcode during this time, the device will not respond, ensuring complete and uninterrupted output of passport information. Conversely, if the secondary camera detects the boarding pass barcode first, the mutual exclusion flag is also set, and the main camera's document OCR recognition is paused for the set time period, focusing solely on barcode decoding. After the timer expires, the mutex flag automatically resets, and the device resumes normal response to both types of information. This mechanism ensures that within a very short time window, the device focuses on processing only one type of information, eliminating logical confusion and data crosstalk caused by simultaneous recognition. Furthermore, this solution utilizes multimodal collaborative acquisition decision-making to associate the two types of information with the same user. Specifically, it calculates the time difference between the generation time of the pre-capture trigger signal (based on corner motion prediction) and the time when the secondary camera detects a change in the direction of movement of a moving object. If this time difference is less than a preset synchronization threshold (e.g., 200 milliseconds), it determines that the passport placement action and the boarding pass presentation action belong to the same continuous interaction (details will be elaborated later). At this point, the device merges the passport OCR information captured by the main camera and the boarding pass barcode information recognized by the secondary camera, and appends a synchronization confidence field to the output data packet. The value of this field is negatively correlated with the size of the time difference; the smaller the time difference, the higher the confidence. Downstream security equipment can automatically complete the comparison of the person, ID, and ticket based on this confidence level, eliminating the need for manual verification.Therefore, this solution is not simply a combination of two recognition functions. Instead, it achieves rapid "one-stop" recognition in the complex scenario of airport boarding gates through multiple means, such as spatial division of labor between dual cameras, temporal isolation of mutually exclusive modes, and data alignment at the decision-making level. This avoids mutual interference of information and is superior to existing solutions that rely on separate recognition or step-by-step operations.

[0054] In this technical solution, a dual-camera system for document recognition is equipped with a primary and secondary camera. Based on automatic exposure and wide dynamic range optimization in outdoor mode, real-time stability calculation, and four-corner detection, fully automatic quality control of document imaging is achieved. Furthermore, instead of the crude approach of using a single camera that cannot adequately handle depth of field, distinguish between indoor and outdoor modes, or employ a fixed output mode, this solution utilizes simultaneous acquisition and independent detection by the dual cameras. The secondary camera continuously monitors the dynamics of the image to identify moving barcodes at a distance, while the primary camera simultaneously detects the barcode attached to the window before taking the document photo. Simultaneously, based on user-preset information output modes—merging, independent, or mutually exclusive—it flexibly adapts to different scenario requirements, such as merging the barcodes on the document itself, performing separate statistics, or using mutual exclusion to prevent interference. By incorporating interference factors such as sudden changes in outdoor brightness, hand shake, and document tilt into frame-by-frame adaptive adjustment and stability judgment, and by skipping the brightness adjustment step directly in indoor mode, the entire reading process can automatically optimize imaging parameters and accurately trigger photography under complex lighting conditions. It can also achieve blind-spot-free collaborative recognition of barcodes at both near and far distances. The output mode can be dynamically switched according to different business logics. Thus, this solution can achieve intelligent, adaptive, and efficient reading of document information and barcode information without manual intervention or increased operational burden. It effectively solves the problems of poor outdoor imaging quality, limited barcode recognition range, and rigid output modes of existing equipment, improves the reusability of multiple scenarios, and reduces information processing costs.

[0055] Based on the technical solutions described above, this specification also provides some specific implementation schemes, which are described below.

[0056] In an optional embodiment, the step of outputting the identified barcode information and document optical character recognition information according to the acquired information output mode may include: When the information output mode is the merging mode, if barcode information and document optical character recognition information are detected simultaneously within a set time, the two types of data will be integrated into a data packet and output. When the information output mode is independent mode, if barcode information or optical character recognition information of certificate is detected separately, the corresponding data will be output separately, and the transmission and processing of the two types of data will be independent of each other. When the information output mode is a mutual exclusion mode, if either barcode information or optical character recognition information of a certificate is detected, the recognition and output of the other type of information will be suspended for a set time period, and the normal recognition state will be restored after the time period ends.

[0057] This embodiment further defines the specific implementation of the information output mode in step 206. In actual engineering implementation, the merging mode requires a short-timer and a data buffer. When either the barcode recognition module or the document optical character recognition module detects valid information, the timer is started and the information is stored in the buffer, while the arrival timestamp of the information is recorded. If the other module also detects valid information before the timer expires, the two are merged into a single data packet and output, while the buffer is cleared and the timer is reset. The format of the data packet can be flexibly designed, for example, using a TLV (Type-Length-Value) structure, outputting the document information first, then the barcode information, and marking the data packet type as "Merging Mode" in the header. If the timer expires without receiving the other type of information, the configuration determines whether to output the cached information (considered as a single information output), discard it, or output a special data packet containing only one type of information. This design takes into account both the merging requirements of the document's built-in barcode and the processing scenario of single information.

[0058] The implementation of the independent mode is relatively simple. The barcode recognition module and the optical character recognition module each have their own independent output channel and buffer. When the barcode recognition module detects barcode information, it immediately encapsulates the information into an independent data packet and sends it through the barcode output channel. When the optical character recognition module detects document information, it similarly encapsulates it immediately and sends it through the document output channel. There are no locking, waiting, or priority mechanisms between the two channels; they are processed completely in parallel. In independent mode, the device can even output both types of information simultaneously, as long as they are detected close to each other. This mode is particularly suitable for sending two types of information to different business processing systems or database tables, such as sending barcode information to an inventory system and document information to a membership system.

[0059] The mutual exclusion mode implements a mutex lock mechanism. Internally, the device maintains a mutex flag (MutexFlag) and a timer. The initial value of the mutex flag is 0, indicating that recognition of any type of information is allowed. When either the barcode recognition module or the document optical character recognition module detects valid information, it first checks the mutex flag: if the flag is 0, the information is output normally, then the flag is set to 1 and a timer is started (the timer duration is the set time period, such as 3 seconds, 5 seconds, or 10 seconds). If the flag is 1, the currently detected information is discarded without any output. During the timer's operation, even if another module detects information, it will be discarded. After the timer expires, the mutex flag is reset to 0, restoring full-type recognition. This mechanism effectively prevents interference between two types of information in the same business scenario, and is especially suitable for security inspection scenarios that require strict differentiation between document scanning and barcode scanning. Users can adjust the time period of the mutual exclusion mode through the configuration interface to adapt to the operational rhythm of different businesses.

[0060] In an optional embodiment, in the step of continuously monitoring the dynamics of the screen through the secondary camera and initiating frame-by-frame barcode scanning detection to identify barcodes for targets in front of the device window at a distance when a moving object is detected in the screen, the depth of field of the secondary camera is greater than that of the main camera.

[0061] This embodiment further clarifies the depth-of-field relationship between the main camera and the secondary camera, which is the physical basis for achieving blind-spot-free barcode recognition at both near and far distances. Depth of field refers to the range of object distances within which a camera can clearly image, and it is affected by factors such as lens focal length, aperture size, sensor size, and focusing distance. A larger depth of field means that the image maintains acceptable sharpness over a wide range of object distance variations; a smaller depth of field means that the image is only clear within a very narrow range of object distances.

[0062] The main camera typically uses a short focal length, large aperture lens design to obtain a wider field of view (making it easier to capture the entire area close to the window), but its depth of field is relatively shallow. The optimal imaging distance is designed to be close to the area of ​​the device window (approximately 0 to 2 centimeters). The advantage of this design is that when the document is close to the window, the main camera can obtain the highest resolution details, ensuring the accuracy of optical character recognition and barcode recognition. If the user places the barcode at a greater distance, the image captured by the main camera will become blurry and undecipherable.

[0063] The secondary camera can employ a relatively long focal length or a smaller aperture lens design, sacrificing some field of view for a greater depth of field. Its optimal imaging distance range can cover several centimeters or even tens of centimeters extending outward from the device window. This design allows the secondary camera to clearly capture barcodes held by the user (such as boarding passes, baggage tags, express delivery slips, membership cards, etc.), even if the user does not hold the barcode completely against the window. The greater depth of field of the secondary camera also brings another benefit: even if the barcode is slightly shaken in the user's hand, as long as it remains within the depth of field, the image remains clear, thus reducing the probability of motion blur.

[0064] It should be noted that the specific depth-of-field values ​​of the main camera and the secondary camera can be adjusted according to the selection of the camera module and the optical design. This application does not impose absolute numerical limitations. As long as the depth of field of the secondary camera is greater than that of the main camera, the two can complement each other and jointly form a blind-spot-free barcode recognition capability from the window to a certain distance. For example, the optimal clear distance of the main camera is 0-3 cm, and the optimal clear distance of the secondary camera is 4-15 cm. The two may overlap or smoothly transition at 3-4 cm, thereby ensuring that at least one camera can clearly image within the entire recognition range.

[0065] In an optional embodiment, after the step of performing real-time stability calculation on each optimized frame, the following step may also be included: If the image stability does not meet the set standard, the image recognition operation will not be triggered, and the stability calculation will continue for the next frame.

[0066] This embodiment supplements the photo-taking trigger conditions, ensuring that the document image is only captured when the scene is still, thus avoiding optical character recognition failures caused by motion blur. In real-world scenarios, users inevitably experience hand tremors or the document sliding on the window when placing it. If the device triggers the photo-taking while the scene is still unstable, the resulting image is likely to have motion blur, causing the text and barcodes on the document to be unable to be correctly recognized, thereby reducing the user experience and increasing the number of retries.

[0067] Various algorithms can be used to calculate stability. A common and computationally efficient method is to calculate the root mean square error (RMSE) or mean absolute error (MAE) between two adjacent frames. Specifically, the current frame and the previous frame are converted to grayscale images, the average of the sum of squares of the differences in grayscale values ​​of corresponding pixels is calculated, and then the square root is taken to obtain the RMSE. When the RMSE of multiple consecutive frames (e.g., 3 frames) is less than a preset threshold (e.g., 5 grayscale levels), the image is considered stable. Another method is to detect the positional offset of the four corner points of the document in consecutive frames, which can be done using optical flow or by directly calculating the Euclidean distance of the corner point coordinates. When the displacement of all corner points is less than a preset pixel value (e.g., 2 pixels), the image is considered stable. This application does not limit the specific stability calculation method, as long as it can output a quantized value characterizing the degree of image stability. To improve robustness, the inter-frame structural similarity index (SSIM) can also be used for evaluation.

[0068] When the calculated stability value fails to meet the preset standard (e.g., RMSE exceeds the threshold, or corner displacement exceeds 2 pixels), the device will not trigger a photo capture. Instead, it will discard the current frame or skip the photo capture action, continue receiving the next frame, and recalculate the stability. This cycle will continue until the image stabilizes or the user moves the document away. To avoid getting stuck in an infinite loop due to prolonged instability, a timeout mechanism is usually set. If stability is not achieved within a preset timeout period (e.g., 10 seconds), the device will proactively abandon the recognition attempt and prompt the user to reposition the document. This mechanism ensures that an image is only captured when the document is completely still, thus significantly improving the success rate of photo capture and the accuracy of subsequent optical character recognition. Simultaneously, by avoiding triggering a photo capture during shaking, the storage and processing overhead of invalid image data can be reduced.

[0069] In an optional embodiment, the step of determining whether the target is a properly placed document through corner matching may include: The four detected corner points are matched with the preset document corner point standards, and the current target is determined to be a properly placed document based on the matching results.

[0070] This embodiment details the specific implementation of corner matching. The document corner standard is a mathematical model pre-established based on the document's physical dimensions and the geometric relationship of image perspective. Taking the common second-generation resident ID card as an example, its standard size is 85.6 mm × 54 mm, with an aspect ratio of approximately 1.585. Under ideal, face-to-face placement conditions (i.e., the document plane is perpendicular to the camera's optical axis), the four corner points after imaging should form a rectangle with a ratio of the longer side to the shorter side close to 1.585, equal diagonal lengths, and adjacent sides perpendicular to each other. When the document is tilted or rotated, the quadrilateral after imaging will undergo perspective distortion, but the geometric constraints between its corner points (such as cross-ratio invariance, collinearity of the intersection points of opposite side extensions, etc.) still satisfy the inherent laws of perspective projection.

[0071] The four detected corner points form a quadrilateral. The geometric parameters of this quadrilateral, such as side length, diagonal, angles, and corner point coordinates, are substituted into the standard model of document corner points for matching calculations. Matching algorithms can employ Iterative Closest Point (ICP), Random Sample Consensus (RANSAC), or a scoring function based on geometric constraints. A practical method is to first calculate the homography matrix of the document plane based on the four corner points, correct the imaged quadrilateral to a front view, and then check whether the aspect ratio of the document in the corrected image is within 1.585 ± tolerance, and whether the four corners are right angles (error < 5 degrees). Another method is to directly calculate the deviation of the actual quadrilateral's aspect ratio from the standard aspect ratio, the deviation of the included angle of adjacent sides from 90 degrees, and the percentage difference in diagonal lengths, etc., and then sum these deviations with weights to obtain a matching score. The higher the matching score, the closer the geometric features of the current quadrilateral are to the projected features of the standard document.

[0072] When the matching accuracy exceeds a preset threshold (e.g., 95%), the current target is considered a properly placed document. When the matching accuracy is below the threshold, it is determined to be other rectangular objects (such as bank cards, membership cards, mobile phones, business cards, papers, etc.) or an improperly placed document (e.g., excessively tilted, flipped, too far away, or partially obscured corners). In this way, this application can effectively filter out interference from non-document objects, ensuring that photography is only triggered when the document is correctly placed, avoiding invalid operations due to misidentification. In addition, the corner matching results can also be used to further determine the orientation of the document (front and back, top and bottom), thereby assisting in the subsequent optical character recognition area positioning.

[0073] In optional embodiments, the technical solution may further include the following steps: In response to the device being in indoor mode, the steps of performing brightness detection on each frame and comparing the brightness value of the frame with the preset expected brightness threshold are not performed; instead, the document image is directly captured.

[0074] This embodiment further clarifies the processing flow in indoor mode, contrasting sharply with outdoor mode. Indoor and outdoor modes represent two switchable operating states for the device. When the device is installed in a standardized indoor lighting environment, such as a government service window, bank counter, hotel reception, or airport check-in counter, the ambient lighting is relatively stable, without extreme conditions such as direct sunlight, backlighting, or large shadows. In this case, if the complex frame-by-frame brightness detection, dynamic adjustment of exposure parameters, and wide dynamic range optimization of outdoor mode were still performed, it would not only consume unnecessary computing resources (increased processor load and longer response latency), but may also introduce unnecessary fluctuations in image quality due to over-adjustment (e.g., repeatedly adjusting exposure when there are slight fluctuations in light, causing screen flicker).

[0075] Therefore, in indoor mode, the device skips the brightness detection and threshold comparison steps, and no longer adjusts exposure parameters and signal gain in real time. Instead, the device uses a set of fixed exposure parameters pre-calibrated for indoor lighting conditions for image acquisition. These fixed parameters are determined during factory calibration or on-site installation and commissioning by analyzing the optimal imaging effect under typical indoor lighting environments (e.g., illuminance of 300-500 lux). Fixed parameters may include exposure time (e.g., 1 / 60 second), analog gain (e.g., ISO 100), and digital gain (e.g., 0 dB). Directly using these fixed parameters ensures image clarity and contrast while significantly reducing processing latency and power consumption. For self-service terminals that need to operate continuously for extended periods (e.g., self-service document processing machines, self-service ticket machines, self-service turnstiles, etc.), omitting unnecessary calculation steps in indoor mode can significantly improve the device's response speed and energy efficiency. After the user places their document, the device can immediately acquire and recognize the image without waiting for the exposure parameters to converge, resulting in a smoother user experience. Meanwhile, the device retains core functions such as corner detection and stability assessment to ensure that recognition accuracy is not affected.

[0076] Indoor and outdoor modes can be switched via a physical switch on the device, software configuration, or an automatic light sensor. For example, the device automatically switches modes when it detects that the ambient light intensity is consistently below or above a certain threshold. This dual-mode design allows the device to flexibly adapt to different scenarios, ensuring image quality for outdoor use while avoiding unnecessary overhead in indoor mode.

[0077] Through the above description of specific embodiments, those skilled in the art should understand that the document reading all-in-one machine and its document reading method based on dual cameras proposed in this application can effectively solve the problems of poor imaging quality, limited barcode recognition range, and rigid output modes of existing document reading devices in complex lighting environments through automatic exposure and wide dynamic range optimization in outdoor mode, stability calculation and four-point corner detection, barcode collaborative recognition by main and secondary dual cameras, and three flexible information output modes.

[0078] In optional embodiments, the technical solution may further include dynamically predicting the optimal time to take a picture of the document based on the sequence of corner point position changes in multiple consecutive frames captured by the main camera before triggering the photo recognition operation. Specifically, this may include: Extract the coordinate sequence of the four corner points of the document in the continuous multi-frame images, and calculate the coordinates of the center point of the document and the plane rotation angle in each frame; The motion trajectory of the document in the image plane is fitted based on the center point coordinate sequence. A Kalman filter is used to smooth and predict the motion trajectory, and the predicted position of the center point of the document in the next frame is estimated. The angular velocity of the document is calculated based on the planar rotation angle sequence. When the angular velocity is lower than the preset angular velocity threshold and the deviation between the predicted position and the center point of the current frame is less than the preset displacement threshold, it is determined that the document is about to enter the optimal static state. When the next frame, which is determined to be about to enter the optimal static state, arrives, the photo recognition operation is triggered in advance, so that the delay between the photo capture time and the actual static time of the document is less than one image acquisition cycle.

[0079] This embodiment further optimizes the photo-taking triggering mechanism, aiming to eliminate the inherent delay of one image acquisition cycle in the traditional stability judgment process of "waiting for stability → triggering photo taking". In conventional implementations, the device needs to detect that multiple consecutive frames meet stability conditions (e.g., corner displacement is less than a threshold) before triggering photo taking. This means that the actual photo taking time lags behind the moment the document is truly stationary by at least several image acquisition cycles. In scenarios where users quickly place their documents, this delay may cause the document to have already slightly shifted by the time the photo is taken, thus introducing motion blur.

[0080] To address this issue, this embodiment introduces a predictive imaging mechanism based on corner motion sequences. Specifically, in the continuously captured multi-frame images, the device first extracts the coordinates of the four corner points of the document from each frame. Based on these four corner points, the coordinates of the document's center point (the arithmetic mean of the four corner point coordinates) and the document's planar rotation angle (e.g., the angle between the document's long side and the horizontal direction) can be calculated. The center point coordinates of each frame are arranged in chronological order to form a discrete motion trajectory sequence. Due to sensor noise and minor hand tremors, this sequence may exhibit some random fluctuations. Therefore, this application employs a Kalman filter to smooth and predict the trajectory. The Kalman filter is a recursive estimation algorithm that dynamically adjusts model parameters based on the prediction error of the previous frame and the observation value of the current frame, thereby suppressing noise while accurately estimating the predicted position of the document's center point in the next frame.

[0081] Simultaneously, the angular velocity of the document is calculated based on the planar rotation angle sequence, which is the change in rotation angle between adjacent frames divided by the time interval. When the angular velocity decreases below a preset angular velocity threshold (e.g., 5 degrees per second), it indicates that the document's movement in the rotational direction has essentially stopped; and when the Euclidean distance between the center point position of the next frame predicted by the Kalman filter and the center point position of the current frame is less than a preset displacement threshold (e.g., 2 pixels), it indicates that the document is also about to stop in the translational direction. When both conditions are met simultaneously, the algorithm determines that the document will enter the optimal static state when the next frame arrives. At this point, the device no longer waits for the stability calculation result of the next frame, but immediately triggers the photo recognition operation at the moment the next frame's exposure begins. Thus, the delay between the photo capture moment and the actual static moment of the document is compressed to less than one image acquisition cycle (e.g., less than 33 milliseconds at a frame rate of 30 frames per second), significantly improving the timing accuracy of the capture.

[0082] In an optional embodiment, the step of continuously monitoring the dynamics of the image through the secondary camera and initiating frame-by-frame barcode scanning detection when a moving object is detected in the image further includes adaptively adjusting the parameters of the barcode decoding engine based on the direction and speed of the moving object's movement. Specifically, this may include: When the secondary camera detects a moving object in the frame, it extracts the motion vector of the moving object and calculates its approach speed and approach angle relative to the center point of the device window. If the approach speed exceeds the preset high-speed threshold, it is determined to be a fast scanning barcode recognition scenario. The barcode decoding engine is automatically switched from global search mode to dynamic tracking mode, and the decoding timeout is shortened to the first preset duration. If the approach speed is lower than the preset low speed threshold and the angle between the approach angle and the normal direction of the device window is less than the preset angle, it is determined to be a stable barcode recognition scenario. The barcode decoding engine is automatically switched to high-precision global decoding mode, and the decoding timeout is extended to a second preset duration, which is greater than the first preset duration. At the same time, the positive or negative value of the approach speed determines whether the moving object is approaching or moving away. If it is determined to be moving away, the current barcode scanning detection is stopped and the decoding cache is cleared to release computing resources.

[0083] This embodiment aims to address the problem that existing barcode decoders, with their fixed parameters, cannot adapt to different user habits. Users exhibit two typical behavior patterns when presenting barcodes: rapid scanning (e.g., quickly waving a phone in front of a turnstile) and stable placement (e.g., flatly placing a paper barcode on a window). Traditional decoders typically cannot distinguish between these two patterns, leading to failures due to excessive decoding timeouts during rapid scanning and missed reads due to overly aggressive decoding strategies during stable placement.

[0084] This application utilizes motion information provided by a secondary camera to dynamically optimize the decoding engine. When the secondary camera detects a moving object in the frame, it first extracts the object's motion vector using optical flow or background subtraction. Then, it calculates the object's approach velocity (i.e., the projection of the motion vector in the direction pointing to the window center) and approach angle (i.e., the angle between the motion direction and the window normal) relative to the center point of the device window. When the approach velocity exceeds a preset high-speed threshold (e.g., 30 cm / s), it is determined that the user is performing a rapid scan. At this point, the decoding engine switches from the default global search mode (traversing the entire image to search for the barcode area) to dynamic tracking mode. Dynamic tracking mode uses the current position of the moving object as the center and performs barcode searches only within a small region of interest, while shortening the decoding timeout from the default value (e.g., 200 milliseconds) to a first preset duration (e.g., 50 milliseconds). This both matches the short dwell time of rapid scans and reduces the computational load.

[0085] Conversely, when the approach speed is lower than a preset low-speed threshold (e.g., 5 cm per second) and the angle between the approach angle and the window normal is less than a preset angle (e.g., 30 degrees), it is determined that the user is stably placing the barcode in front of the window. At this time, the decoding engine switches to high-precision global decoding mode. High-precision global decoding mode enables more complex image preprocessing (such as adaptive binarization and distortion correction), finer edge detection, and stronger error correction algorithms (such as Reed-Solomon error correction), and extends the decoding timeout to a second preset duration (e.g., 500 milliseconds) to ensure successful decoding of damaged, low-quality, or distant barcodes. In addition, this embodiment also determines the movement trend of the moving object based on the positive or negative value of the approach speed: when the speed is positive, it indicates that the object is approaching the window, and when the speed is negative, it indicates that the object is moving away from the window. Once moving away is detected (speed < 0), the device immediately stops the current barcode scanning detection and clears the transient data in the decoding cache, thereby freeing up valuable computing resources and avoiding invalid scanning.

[0086] In optional embodiments, the technical solution further includes time-synchronized fusion of the pre-triggered image recognition operation with the parameters of the adaptively adjusted barcode decoding engine to form a multimodal collaborative acquisition decision, which may specifically include: When the pre-capture trigger signal based on corner motion prediction is generated, the mode and timeout parameters of the barcode decoding engine after adaptive adjustment are simultaneously acquired. Calculate the time difference between the generation time of the pre-capture trigger signal and the time when the secondary camera detects a change in the direction of movement of a moving object. If the time difference is less than a preset synchronization threshold, it is determined that the ID card and the barcode belong to the same user interaction behavior. After determining that they are the same interactive behavior, the exposure time of the main camera is aligned with the barcode decoding and scanning frame of the secondary camera, so that the difference in timestamp between the document image captured by the main camera and the barcode information recognized by the secondary camera is less than half an image acquisition cycle. The aligned document image and barcode information are merged and output to the information output module, and a synchronization confidence field is added to the merged data packet. The value of the synchronization confidence field is negatively correlated with the magnitude of the time difference.

[0087] This embodiment deeply integrates the aforementioned two optimization strategies (predictive photo-taking timing and adaptive barcode decoding) to solve a long-neglected problem: how to ensure that the two types of information captured by the main camera and the secondary camera are strictly aligned in time when a user uses both identification and barcodes simultaneously (e.g., presenting a passport and boarding pass at airport security, or an ID card and order QR code at hotel check-in), and how to determine whether they belong to the same business operation. In traditional devices, the acquisition processes of the two cameras are completely independent, resulting in a difference of hundreds of milliseconds between the timestamp of the identification image and the timestamp of the barcode information. Downstream systems cannot determine whether the two correspond to the same user, thus necessitating the addition of a manual verification step.

[0088] This application integrates the two at the collaborative decision-making layer. When a pre-capture trigger signal based on corner motion prediction is generated (indicating that the main camera will capture an image of the document within a very short delay), the system immediately reads the current mode (fast scan mode or stable placement mode) and its timeout parameters of the barcode decoding engine. Subsequently, it calculates the time difference between the moment the pre-capture trigger signal is generated and the moment when the secondary camera last detected a significant change in the direction of motion of a moving object (e.g., from approaching to stationary, or from scanning to moving away). If this time difference is less than a preset synchronization threshold (e.g., 200 milliseconds), it is determined that the user's document placement action and barcode display action belong to the same continuous interaction.

[0089] After determining that the interaction is the same, the system performs a fine-grained time alignment operation, that is, by adjusting the exposure sequence of the main camera to align it with the start time of the next frame scan by the secondary camera. Specifically, hardware trigger signal synchronization or software cache alignment can be used to ensure that the timestamp of the document image captured by the main camera and the recording time of the barcode information recognized by the secondary camera differ by less than half an image acquisition cycle (e.g., less than 16 milliseconds at 30 frames / second). Finally, the aligned document image (or its optical character recognition result) and barcode information are merged into a single data packet and sent through the information output module. To facilitate downstream systems in evaluating the reliability of the association, a synchronization confidence field can be added to the merged data packet. The value of this field is negatively correlated with the magnitude of the time difference; that is, the smaller the time difference, the higher the confidence (e.g., a value of 0-100, where the confidence is 100 when the time difference is less than 50 milliseconds, and the confidence decreases by 0.5 for every 1 millisecond increase in the time difference). The downstream system can decide whether to automatically complete the information comparison or require manual intervention based on this confidence level. This multimodal collaborative acquisition mechanism can improve the accuracy and automation of data association in complex business scenarios.

[0090] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A document reading all-in-one machine, characterized in that, include: case; The main camera, mounted on the housing, is used to capture images of identification documents and barcodes that are affixed to the window area of ​​the device; A secondary camera, mounted on the housing, is used to capture barcode images at a distance greater than the distance of the area attached to the device window center. A controller is electrically connected to the main camera and the secondary camera respectively. The controller includes at least one processor and a memory storing a computer program. When the computer program is executed by the processor, it implements the following modules: The environment adaptation module is used to respond to the device being in outdoor mode, to obtain the brightness value of the output image of the main camera in real time, compare the brightness value with a preset threshold, and adjust the exposure parameters and signal gain of the main camera in real time according to the comparison result, and enable wide dynamic range technology to optimize the image. The photo-taking trigger module is used to perform real-time stability calculations on each frame of the main camera after it has been adjusted by the environment adaptation module. When a target area that appears to be a document is detected in the frame and the stability reaches the set standard, the four corner points of the document are detected in the target area. Based on the corner point matching results, it is determined whether the target is a properly placed document. If so, the photo-taking and recognition operation is triggered. The barcode recognition module is used to control the main camera and the secondary camera to synchronously capture images in real time and to independently perform barcode detection operations. The secondary camera is used to start frame-by-frame barcode scanning when a moving object is detected in the image. The main camera is used to synchronously perform barcode detection on the area of ​​the device window before executing the document photo recognition and judgment process. The information output module is used to output the barcode information recognized by the barcode recognition module and the document optical character recognition information triggered by the photo triggering module according to a preset merging mode, independent mode or mutually exclusive mode.

2. The document reading all-in-one machine according to claim 1, characterized in that, The information output module includes: The merging unit is used to combine barcode information and document optical character recognition information into a data packet and output it when barcode information and document optical character recognition information are detected simultaneously within a set time. Independent unit, used to output the corresponding data separately when barcode information or optical character recognition information of certificate is detected separately, and to make the transmission and processing of the two types of data independent of each other; The mutual exclusion unit is used to pause the recognition and output of the other type of information for a set time period after detecting either barcode information or optical character recognition information of a certificate, and to resume normal recognition after the time period ends.

3. The document reading all-in-one machine according to claim 1, characterized in that, The depth of field of the secondary camera is greater than that of the primary camera.

4. The document reading all-in-one machine according to claim 1, characterized in that, The environment adaptation module is also used to respond to the device being in indoor mode by not performing the image brightness value comparison, exposure parameter and signal gain adjustment operations in outdoor mode.

5. A document recognition method based on dual cameras, applied to the document recognition all-in-one machine according to any one of claims 1 to 4, characterized in that, Includes the following steps: In response to the device being in outdoor mode, the output image of the main camera is captured in real time. The brightness of each captured frame is detected sequentially to obtain the image brightness value. The image brightness value is compared with the preset image expected brightness threshold, and the exposure parameters and signal gain of the main camera are adjusted in real time according to the comparison result. Wide dynamic range technology is enabled to optimize the image. Then, the stability of each optimized frame is calculated in real time. When a target area that resembles an ID card is detected in the image and the image stability reaches the set standard, the four corner points of the ID card are detected in the target area. The corner point matching is used to determine whether the target is a properly placed ID card. If it is determined to be an ID card, the photo recognition operation is triggered to complete the ID card image acquisition. While the main camera performs the document photo recognition and judgment process, the main camera and the secondary camera are controlled to synchronously capture images in real time and independently carry out barcode detection. The secondary camera continuously monitors the dynamics of the image. When a moving object is detected in the image, frame-by-frame barcode scanning and detection is initiated to identify barcodes of targets in front of the device window that are at a distance. The main camera synchronously performs barcode detection on the area attached to the device window before performing the document photo recognition and judgment process. The system acquires the user-preset merge mode, independent mode, or mutually exclusive mode as the information output mode, and outputs the recognized barcode information and document optical character recognition information according to the acquired information output mode.

6. The document recognition method based on dual cameras according to claim 5, characterized in that, The step of outputting the identified barcode information and document optical character recognition information according to the acquired information output pattern includes: When the information output mode is the merging mode, if barcode information and document optical character recognition information are detected simultaneously within a set time, the two types of data will be integrated into a data packet and output. When the information output mode is independent mode, if barcode information or optical character recognition information of certificate is detected separately, the corresponding data will be output separately, and the transmission and processing of the two types of data will be independent of each other. When the information output mode is a mutual exclusion mode, if either barcode information or optical character recognition information of a certificate is detected, the recognition and output of the other type of information will be suspended for a set time period, and the normal recognition state will be restored after the time period ends.

7. The document recognition method based on dual cameras according to claim 5, characterized in that, Following the step of performing real-time stability calculations on each optimized frame, the following step is also included: If the image stability does not meet the set standard, the image recognition operation will not be triggered, and the stability calculation will continue for the next frame.

8. The document recognition method based on dual cameras according to claim 5, characterized in that, The step of determining whether the target is a properly placed document through corner matching includes: The four detected corner points are matched with the preset document corner point standards, and the current target is determined to be a properly placed document based on the matching results.

9. The document recognition method based on dual cameras according to claim 5, characterized in that, It also includes the following steps: In response to the device being in indoor mode, the steps of performing brightness detection on each frame and comparing the brightness value of the frame with the preset expected brightness threshold are not performed; instead, the document image is directly captured.

10. The document recognition method based on dual cameras according to claim 5, characterized in that, It also includes dynamically predicting the optimal time to take a picture of the document based on the sequence of corner position changes in multiple consecutive frames captured by the main camera before triggering the photo recognition operation. Specifically, this includes: Extract the coordinate sequence of the four corner points of the document in the continuous multi-frame images, and calculate the coordinates of the center point of the document and the plane rotation angle in each frame; The motion trajectory of the document in the image plane is fitted based on the center point coordinate sequence. A Kalman filter is used to smooth and predict the motion trajectory, and the predicted position of the center point of the document in the next frame is estimated. The angular velocity of the document is calculated based on the planar rotation angle sequence. When the angular velocity is lower than the preset angular velocity threshold and the deviation between the predicted position and the center point of the current frame is less than the preset displacement threshold, it is determined that the document is about to enter the optimal static state. When the next frame, which is determined to be about to enter the optimal static state, arrives, the photo recognition operation is triggered in advance, so that the delay between the photo capture time and the actual static time of the document is less than one image acquisition cycle.

Images