Intelligent document capture method and system fusing ambient light perception and edge detection

Abstract

The application provides a kind of intelligent document capture method and system that fuses ambient light perception and edge detection, comprising: obtaining image frame by parsing video stream, determining candidate target area and calculating texture density by gradient and local binary pattern joint analysis, obtaining local sharpness score, generating sharpness determination threshold combined with adjustment factor proportional to texture density, completing sharpness condition discrimination;Extract the measured coordinates of the four corners of the document outline in the candidate target area, and generate the predicted coordinates of the four corners according to the historical vertex coordinates and synchronous motion data, generate the distance tolerance that changes reversely with the sharpness margin, and determine the position stability according to the Euclidean distance of the vertex coordinates;Select background reference area, and judge whether the light is constant according to the Bhattacharyya distance of multi-frame brightness channel histogram;When the three conditions are met simultaneously, start the waiting period, the period length is proportional to the Euclidean distance of the vertex, and the shooting instruction is triggered if the conditions continue to meet during the period.

Description

Technical Field

[0001] This application belongs to the field of document technology, and in particular relates to an intelligent document capture method and system that integrates ambient light perception and edge detection. Background Technology

[0002] By analyzing video stream images, a shooting command is executed when specific conditions are met, avoiding device shake issues caused by manual operation. In conventional applications, the sharpness of the current image is calculated in real time, document edges are detected, and the image is judged to be static, serving as the basis for triggering the shooting. The lighting environment in the shooting scene is variable; flickering external light sources, shadow occlusion, or instantaneous changes in ambient light can all interfere with image brightness and target feature extraction. Conventional methods use relatively single indicators when judging lighting conditions and lack the ability to shield the impact of background lighting disturbances on the stability of the target area. In addition, in the execution phase after confirming that various shooting conditions are met, a fixed waiting period is often used to trigger the command. This fixed delay strategy is difficult to adjust based on the deviation between the actual and expected positions of the document, and is prone to triggering shooting when the document is not fully aligned or has not reached its optimal stable state, resulting in the acquired image quality not meeting expectations. Therefore, this field seeks a more reasonable automatic document shooting triggering scheme that can adjust the judgment criteria according to the actual texture features of the image, positional deviation, and ambient lighting disturbances, identify appropriate shooting opportunities in variable shooting environments, and acquire high-quality document images. Summary of the Invention

[0003] According to one aspect of the present invention, a smart document capture method integrating ambient light perception and edge detection is proposed to solve the problem that existing methods tend to trigger shooting when the document is not fully aligned or has not reached its optimal stable state, resulting in the acquired image quality not meeting expectations. The method includes the following steps: The system acquires real-time video streams collected by image sensors and parses image frames. It obtains synchronous motion data corresponding to the timestamps of the image frames, performs gradient and local binary mode joint analysis on the current image frame, determines candidate target regions and their texture density values, calculates local sharpness scores within the candidate target regions, and generates sharpness judgment thresholds based on the benchmark value and the adjustment factor determined by the texture density values. When the local sharpness score is higher than the threshold, the sharpness condition is considered to be met. The coordinates of the four corner vertices of the document outline are extracted within the candidate target area as measured vertex coordinates. The predicted coordinates of the four corner vertices are obtained based on the historical vertex coordinates and synchronous motion data. The distance tolerance is determined in reverse based on the difference between the local sharpness score and the sharpness judgment threshold, and it is not lower than the preset minimum distance tolerance. When the vertex deviation distance between the measured vertex coordinates and the predicted coordinates is less than the distance tolerance, the document position stability condition is considered to be met. A background reference area is selected outside the document outline. Based on the brightness channel histogram of multiple consecutive frames of the background reference area within the time window, the Bartl distance between two adjacent frames is calculated. When the Bartl distance between each adjacent frame is less than the brightness perturbation threshold, the illumination constant condition is considered to be met. When the clarity condition, document position stability condition, and constant illumination condition are simultaneously met for the first time, the waiting period is determined. The duration of the period is determined based on the vertex deviation distance between the measured vertex coordinates and the predicted coordinates. If the above conditions are continuously met within the waiting period, a shooting command is triggered.

[0004] According to another aspect of the present invention, an intelligent document capture system integrating ambient light perception and edge detection is proposed, comprising the following modules: The acquisition module is used to acquire real-time video streams collected by image sensors and parse image frames, obtain synchronous motion data corresponding to the timestamps of image frames, perform gradient and local binary mode joint analysis on the current image frame, determine candidate target regions and their texture density values, calculate local sharpness scores within candidate target regions, generate sharpness judgment thresholds based on benchmark values ​​and adjustment factors determined by texture density values, and determine that the sharpness condition is met when the local sharpness score is higher than the threshold. The generation module is used to extract the coordinates of the four corner vertices of the document outline within the candidate target area as measured vertex coordinates, and obtain the predicted coordinates of the four corner vertices based on historical vertex coordinates and synchronous motion data. The distance tolerance is determined in reverse based on the difference between the local sharpness score and the sharpness judgment threshold, and is not lower than the preset minimum distance tolerance. When the vertex deviation distance between the measured vertex coordinates and the predicted coordinates is less than the distance tolerance, the document position stability condition is considered to be met. A background reference area is selected outside the document outline, and the Bartlett distance between adjacent frames is calculated based on the brightness channel histogram of multiple consecutive frames of the background reference area within the time window. When the Bartlett distance between each adjacent frame is less than the brightness perturbation threshold, the illumination constant condition is considered to be met. The determination module is used to determine the waiting period when the sharpness condition, document position stability condition, and constant illumination condition are met simultaneously for the first time. The duration of the period is determined based on the vertex deviation distance between the measured vertex coordinates and the predicted coordinates. If the above conditions are met continuously during the waiting period, a shooting command is triggered.

[0005] This invention utilizes a combined gradient and local binary mode analysis, along with an adjustment factor proportional to texture density, to generate a sharpness threshold. This enables sharpness determination for documents with different texture features, reducing false triggers or missed triggers caused by a single fixed threshold. The sharpness determination result is correlated with document position stability conditions. The distance tolerance is determined inversely based on the difference between the sharpness exceeding the threshold. A preset minimum distance tolerance prevents the distance tolerance from being abnormally small, while a preset maximum distance tolerance prevents abnormally large distance tolerances when the sharpness margin is close to zero. This improves the reliability of image sharpness constraints and achieves more refined position alignment and image stabilization control. Furthermore, a constant illumination detection based on the Bach distance of the brightness histogram of the background reference area across multiple frames is introduced, reducing interference from ambient light flicker or shadow changes. The waiting period is adjusted by comparing the actual and predicted vertex Euclidean distances. When alignment errors are large, the confirmation time is extended to ensure stability; when alignment accuracy is high, the waiting time is shortened for rapid triggering. This cascaded decision-making mechanism achieves a balance between shooting response speed and image quality, improving the accuracy and stability of automatic document shooting. Attached Figure Description

[0006] Figure 1 A flowchart of the first embodiment; Figure 2 A schematic diagram showing the comparison between Kalman predicted vertices and measured document corner points; Figure 3 A schematic diagram is generated for the annular background reference area; Figure 4 This is a histogram of brightness levels for adjacent frames. Detailed Implementation

[0007] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0008] In the first embodiment, the present invention proposes an intelligent document capture method that integrates ambient light perception and edge detection, such as... Figure 1 It includes the following steps: S1. Acquire the real-time video stream collected by the image sensor and parse the image frames. Obtain the synchronous motion data corresponding to the timestamp of the image frame. Perform gradient and local binary mode joint analysis on the current image frame to determine the candidate target region and its texture density value. Calculate the local sharpness score in the candidate target region. Generate a sharpness judgment threshold based on the benchmark value and the adjustment factor determined by the texture density value. When the local sharpness score is higher than the threshold, the sharpness condition is considered to be met.

[0009] The image sensor is activated, a video stream channel is established, and the video stream resolution and frame rate are set. Continuous image frames are extracted from the video stream and converted into single-channel grayscale images. The Sobel operator is used to calculate the gradient magnitudes in the horizontal and vertical directions of the grayscale image to obtain the initial gradient magnitude of each pixel. The current image frame is divided into multiple independent non-overlapping grid blocks. The initial gradient magnitude is calculated within each grid block, and the distribution of local binary pattern encoded values ​​is also calculated. The initial gradient magnitudes of all pixels within each grid block are summed to obtain the gradient statistical score for that grid block. Based on the distribution of local binary pattern encoded values, the number of transitions between 0 and 1 is counted for each octet in a circular adjacent order. When the number of transitions is greater than 2, the local binary pattern encoded value is determined to be a non-uniform pattern. The number of pixels corresponding to the non-uniform pattern within the grid block is counted, and this number is multiplied by a preset texture weight. The coefficient is used to obtain the texture distribution score of the grid block; the gradient statistical score is added to the texture distribution score to obtain the joint texture score of the grid block; four spatially continuous but not isolated grid blocks are used as a sliding window for traversal, and the four grid blocks with the largest sum of joint texture scores are selected. The initial position of the candidate target region is determined based on the four grid blocks, and connected edge regions that are continuous with the document edge are searched around the initial position. The connected edge regions form a candidate target region containing the outline of the main body of the document. The arithmetic mean of the joint texture scores of the four grid blocks is used as the texture density value representing the candidate target region.

[0010] The non-uniform pattern frequency in the local binary pattern encoded value distribution is used to participate in the joint texture scoring of generated mesh blocks. The Laplacian operator is applied to calculate the second derivative of each pixel within the candidate target region. The squares of the second derivative response values ​​of all pixels are calculated, and the arithmetic mean is obtained to obtain the mean square value of the detail response. This mean square value of the detail response is used as the local sharpness score. A sharpness benchmark value and a positive scaling factor are preset. The positive scaling factor is multiplied by the texture density value to obtain an adjustment factor. The sharpness benchmark value and the adjustment factor are added to generate a sharpness judgment threshold. The calculated local sharpness score is compared with the sharpness judgment threshold. If the former is greater than the latter, a Boolean truth value is written to determine that the sharpness condition is met.

[0011] In an optional embodiment, the step of performing joint gradient and local binary pattern analysis on the current image frame to determine candidate target regions and their texture density values ​​includes: The Sobel operator is used to extract the horizontal and vertical gradient data of each pixel in the current image frame. The absolute values ​​of the horizontal and vertical gradient data are added together to obtain the initial gradient magnitude of the current pixel. Using the current pixel as the center point, a nine-grid neighborhood centered on the center point is selected. An eight-bit binary string is generated by comparing the grayscale values ​​of the eight surrounding pixels in the neighborhood with those of the center pixel. The binary string is then converted into a decimal integer as the local binary mode encoding value of the center pixel. The current image frame is evenly divided into multiple independent non-overlapping grid blocks with a resolution of 16×16 pixels. The initial gradient magnitude of all pixels is accumulated in each grid block, and the distribution result of the local binary mode coding value is statistically analyzed. Based on the statistical result of the initial gradient magnitude and the distribution result of the local binary mode coding value, a joint texture score for each grid block is generated. Four spatially continuous but non-isolated grid blocks are traversed as a sliding window to filter out the four grid blocks with the largest sum of joint texture scores. The initial position of the candidate target region is determined based on the four grid blocks, and the candidate target region is enclosed by the connected edge regions around the initial position. The arithmetic mean of the joint texture scores of the four grid blocks is used as the texture density value representing the region.

[0012] Taking an image frame with a resolution of 1920×1080 pixels as an example, since 1080 is not divisible by 16, the edge region in the height direction of the image is first filled with 8 rows of zeros or edge copy fills to form an image to be divided into 1920×1088 pixels. The fill pixels are only used to complete the grid division. The Sobel operator is used to extract the horizontal and vertical gradient data of the current pixel. Let the horizontal gradient data be 120 and the vertical gradient data be -50. The absolute values ​​of the two sets of data are added together to obtain the initial gradient magnitude of the pixel as 170. Using the current pixel as the center point, a 3x3 neighborhood is selected centered on this point. The grayscale value of the center pixel is set to 100. The grayscale values ​​of the eight surrounding pixels in the neighborhood are compared with those of the center pixel to generate an 8-bit binary string. For example, if the grayscale values ​​of the surrounding pixels are 110, 95, 120, 105, 80, 90, 102, and 99 respectively, and are greater than or equal to the grayscale value of the center pixel, they are recorded as 1; otherwise, they are recorded as 0. The resulting binary string is 10110010. This binary string is then converted to a decimal integer 178 as the local binary pattern encoding value for the center pixel. When the width or height of the current image frame is not divisible by 16, the edge regions of the current image frame are zero-padded or edge-copying padded to ensure that the width and height of the padded image are both integer multiples of 16.

[0013] The current image frame is uniformly divided into multiple independent non-overlapping grid blocks with a resolution of 16×16 pixels. Within each grid block, the initial gradient magnitude of all pixels is accumulated, with a total of 35800. This total is used as the gradient statistical score for that grid block. The distribution of local binary mode encoded values ​​within the grid block is then statistically analyzed. Assuming the number of pixels belonging to the non-uniform mode within the grid block is 170, and a preset texture weight coefficient of 250, 170 multiplied by 250 yields a texture distribution score of 42500. Finally, the gradient statistical score 35800 is added to the texture distribution score 42500 to obtain the joint texture score 78300 for the grid block. Four spatially continuous but not isolated grid blocks are traversed as a sliding window to filter out the four grid blocks with the largest sum of joint texture scores. Let the scores of these four grid blocks be 78300, 80200, 79500 and 81000, respectively. The initial position of the candidate target region is determined based on these four grid blocks, and connected edge regions that are continuous with the document edge are searched around the initial position. The connected edge regions enclose the candidate target region containing the outline of the main body of the document, and the arithmetic mean of the joint texture scores of the four grid blocks, 79750, is used as the texture density value representing the region.

[0014] In an optional embodiment, calculating the local sharpness score within the candidate target region includes: Extract the red, green, and blue color channel data of each pixel in the candidate target area, and use a color conversion matrix to weight and sum the data of the three channels according to a preset brightness weight to obtain a single-channel grayscale region matrix. A Laplacian second derivative spatial filter template with a scale of 3×3 pixels is selected. The edge pixels of the grayscale region matrix are processed by copy filling or zero filling. The filter template and the processed grayscale region matrix are subjected to two-dimensional matrix convolution traversal pixel by pixel to obtain a response result matrix that represents the degree of drastic change at the edge of each pixel and has the same size as the grayscale region matrix. The mean square value of the detailed response to the candidate target region is obtained by squaring each output pixel value in the response result matrix and taking the arithmetic mean of the squared values. The mean square value of the detail response is used as the local sharpness score.

[0015] For the candidate target region, such as a 32×32 pixel region composed of 4 grid blocks, containing a total of 1024 pixels, the data of the red, green and blue color channels of all pixels in the region are extracted. Assuming that a certain pixel has R=210, G=150, B=80, the data of the three channels are weighted and summed according to the preset brightness weight using a color conversion matrix, such as the formula Y=0.299×R+0.587×G+0.114×B, to calculate a grayscale value of approximately 160, and then the dimension is reduced to a single-channel grayscale region matrix. A Laplacian second derivative spatial filter template with a scale of 3×3 pixels is selected, such as a matrix with a center value of 8 and surrounding values ​​of -1. The edge pixels of the grayscale region matrix are processed by copy filling or zero filling. The filter template and the processed grayscale region matrix are subjected to a two-dimensional matrix convolution traversal pixel by pixel to obtain a response result matrix that represents the degree of drastic change at the edge of each pixel and has the same size as the grayscale region matrix. For example, when the center of the filter template covers a pixel with a gray value of 160, if the sum of the gray values ​​of 8 adjacent pixels is 1200, the response result is 160×8-1200=80. The squares of all output pixel values ​​in the response result matrix are iterated and summed. Assuming there are 10 response values ​​of 80, 100 response values ​​of 40, and the rest of 0, the sum of the squares is 224000. This sum is then divided by the total number of pixels in the candidate target region, 1024, to obtain the mean square value of the detail response for the candidate target region, which is 218.75. This mean square value of the detail response, 218.75, is used as the local sharpness score.

[0016] In an optional embodiment, generating a sharpness determination threshold based on a reference value and an adjustment factor determined by the texture density value includes: Obtain the baseline value; Obtain a preset proportional feedback parameter constant in the control buffer, multiply the proportional feedback parameter constant by the obtained texture density value, and use the resulting product as the adjustment factor; The sharpness determination threshold is obtained by adding the benchmark value to the adjustment factor.

[0017] Obtain a reference lower limit constant, set to 150.0, and confirm this constant as the baseline value. Obtain a preset proportional feedback parameter constant in the control buffer, set its value to 0.0005, and multiply this proportional feedback parameter constant 0.0005 by the texture density value obtained in real time, setting it to 79750, to obtain an adjustment factor of 39.875. Add the baseline value 150.0 to the adjustment factor 39.875 to obtain the sharpness judgment threshold of 189.875.

[0018] S2. Extract the coordinates of the four corner vertices of the document outline within the candidate target area as measured vertex coordinates, and obtain the predicted coordinates of the four corner vertices based on historical vertex coordinates and synchronous motion data. Determine the distance tolerance in reverse based on the difference between the local sharpness score and the sharpness judgment threshold, and ensure that it is not lower than the preset minimum distance tolerance. When the vertex deviation distance between the measured vertex coordinates and the predicted coordinates is less than the distance tolerance, the document position stability condition is considered to be met. Select a background reference area outside the document outline, and calculate the Bartlett distance between adjacent frames based on the brightness channel histogram of multiple consecutive frames of the background reference area within the time window. When the Bartlett distance between each adjacent frame is less than the brightness perturbation threshold, the illumination constant condition is considered to be met.

[0019] Canney edge detection is performed on the candidate target region containing the outline of the main body of the document to obtain a binarized edge map. The Hough line transform is used to extract candidate line segments in the edge map. The candidate line segments are divided into two groups of approximately parallel boundary line segments according to the line segment direction. In each group, two boundary line segments located at the outer boundary of the document are selected to obtain four target line segments corresponding to the four sides of the document. The intersection points between adjacent target line segments are calculated. Intersection points located outside the candidate target region or that cannot form a convex quadrilateral are removed. The remaining four intersection points are used as the coordinates of the four corner vertices actually extracted in the current image frame. A Kalman filter model is established. The posterior estimated vertex coordinates of the previous frame are used as the historical vertex coordinates predicted for the current frame. Combined with synchronous motion data and the state transition matrix, the prior predicted coordinates of the four corner vertices of the current frame are derived and used as the predicted coordinates. The vertex deviation distance of the current frame is calculated using the prior predicted coordinates and the actual corner vertex coordinates extracted from the current image frame. The actual corner vertex coordinates extracted from the current image frame are used as measurements only to update the Kalman filter. The posterior estimated vertex coordinates obtained after the measurement update are used as the historical vertex coordinates predicted for the next frame. The positions of the Kalman predicted vertices and the measured document corner points are compared as follows: Figure 2 As shown.

[0020] The difference between the local sharpness score and the sharpness judgment threshold is calculated to obtain the sharpness margin. A base distance constant, a small positive number to prevent the denominator from being zero, a preset minimum distance tolerance, and a preset maximum distance tolerance are set. The sharpness margin is added to the small positive number to obtain the denominator, and the base distance constant is divided by this denominator to obtain the candidate distance tolerance. The candidate distance tolerance is compared with the preset minimum distance tolerance, and the larger value is taken to obtain the distance tolerance after lower limit processing. The distance tolerance after lower limit processing is then compared with the preset maximum distance tolerance, and the smaller value is taken to obtain the final distance tolerance. The square root of the sum of the squares of the differences between the horizontal and vertical coordinates of the four actual extracted vertices and their corresponding predicted vertices is calculated. The maximum value among the four Euclidean distances is determined as the vertex deviation distance. If the vertex deviation distance is less than the distance tolerance, the document position stability condition is considered satisfied. Using the document outline formed by the four extracted vertices as a reference, a fixed pixel distance is extended outward to obtain an outer rectangle. The annular band area obtained by subtracting the inner area of ​​the document outline from the outer rectangle is set as the background reference area. The annular background reference area generated by expanding the document outline is as follows: Figure 3 As shown.

[0021] When the bounding rectangle exceeds the boundary of the current image frame, it is cropped to the effective coordinate range of the current image frame, and only pixels that remain outside the document outline after cropping are retained as background reference areas. When the number of retained pixels is less than a preset minimum area threshold, the fixed pixel distance of the outward expansion is reduced, or a side area that is not covered by the document outline and whose area meets the requirements is selected from the top, bottom, left, and right directions outside the document outline as a background reference area. If a background reference area that meets the preset minimum area threshold cannot be obtained after the above processing, the current image frame does not perform the constant illumination condition judgment and waits for the next image frame to regenerate the background reference area. A sliding time window with a length of five frames is established. The background reference area image of each frame within the time window is converted from the red-green-blue color space to the hue-saturation-luminance color space. The luminance channel is extracted, and the luminance value is divided into 128 intervals. The number of pixels in each interval is counted and normalized to obtain a luminance channel histogram. The luminance statistical distribution of adjacent frames of the background reference area within the time window is as follows. Figure 4 As shown. Calculate the square root of the product of the probabilities of corresponding intervals in the histograms of two adjacent frames, and sum the square roots of all intervals to obtain the histogram similarity coefficient. Limit the histogram similarity coefficient to a preset minimum positive number within the range of 1 to obtain the effective similarity coefficient. Take the negative logarithm of the effective similarity coefficient to obtain the Barthel distance between two adjacent frames. If the Barthel distance of all adjacent frames within the sliding time window is less than a preset brightness perturbation threshold, then the constant illumination condition is considered satisfied.

[0022] In an optional embodiment, the generation of distance tolerance, which determines the distance tolerance in reverse based on the difference between the local sharpness score and the sharpness determination threshold, and is not lower than a preset minimum distance tolerance, includes: Obtain the base distance constant, the preset minimum distance tolerance, the preset maximum distance tolerance, and the preset small positive number used to prevent the denominator from being zero; Subtract the sharpness determination threshold from the local sharpness score to obtain the sharpness margin; The sharpness margin is added to the preset small positive number to obtain the denominator, and the candidate distance tolerance is obtained by dividing the basic distance constant by the denominator. The candidate distance tolerance is compared with the preset minimum distance tolerance, and the maximum value of the two is taken. Then, the result of taking the larger value is compared with the preset maximum distance tolerance, and the minimum value of the two is taken as the distance tolerance.

[0023] Obtain a base distance constant, set to 2000; a preset minimum distance tolerance, set to 10 pixels; a preset maximum distance tolerance, set to 80 pixels; and a preset small positive number, set to 0.001, to prevent the denominator from being zero. Subtract the previously calculated sharpness threshold of 189.875 from the currently calculated local sharpness score of 218.75 to obtain a sharpness margin of 28.875. Add the sharpness margin of 28.875 to the preset small positive number 0.001 to obtain a denominator of 28.876. Divide the base distance constant 2000 by this denominator to obtain a candidate distance tolerance of approximately 69.26 pixels. Compare this candidate distance tolerance of 69.26 pixels with the preset minimum distance tolerance of 10 pixels, and take the larger value of 69.26 pixels. Then compare 69.26 pixels with the preset maximum distance tolerance of 80 pixels, and take the smaller value of 69.26 pixels as the effective distance tolerance.

[0024] In an optional embodiment, calculating the Bach distance between two adjacent frames based on the luminance channel histograms of multiple consecutive frames of the background reference region within the time window includes: Obtain the current recognized image frame and the previous recognized image frame adjacent to the time in the background reference area within the coverage of the time window, and read the brightness level value of each pixel in the brightness channel in these two frames; Two hierarchical histogram arrays, each with 128 data storage scales, are established. The pixel occurrence frequency of the current recognized image frame and the pixel occurrence frequency of the previous recognized image frame are calculated and accumulated separately according to the interval distribution, and then normalized to form a first frequency distribution list and a second frequency distribution list. Using the index codes of the corresponding positions, extract the grid content corresponding to the first frequency distribution list item and the second frequency distribution list item respectively, and perform a group-by-group digital multiplication operation. Take the square root of each product and sum the square root results to obtain the histogram similarity coefficient. The histogram similarity coefficient is restricted to a range from a preset minimum positive number to 1 to obtain an effective similarity coefficient. The negative logarithm of the effective similarity coefficient is then calculated using a logarithmic function and used as the Bach distance between two adjacent frames.

[0025] The current recognized image frame and the previous recognized image frame adjacent to the time frame within the background reference area covered by the time window are obtained. For example, if the area has a total of 40,000 pixels, the brightness level value of each pixel in the brightness channel of these two frames is read. The brightness level value is located in the range of 0 to 127. Two histogram recorder arrays with 128 data storage scales are established. The pixel occurrence frequency of the current recognized image frame and the pixel occurrence frequency of the previous recognized image frame are calculated and accumulated separately according to the interval distribution and normalized. Let the occurrence frequency of a certain brightness level value of 50 in the current frame be 0.05 after normalization, and the frequency of the corresponding scale in the previous frame be 0.048 after normalization, to form a first frequency distribution list and a second frequency distribution list. Using the index codes of the corresponding positions, the corresponding grid segments of the first frequency distribution list item and the second frequency distribution list item are extracted and grouped by digital multiplication. 0.05 and 0.048 are multiplied together to obtain 0.0024. The square root of all the decimal values ​​obtained by multiplication is calculated. The square root of 0.0024 is approximately 0.04899. All square root results are summed to obtain the histogram similarity coefficient. The histogram similarity coefficient is set to 0.985. Since the histogram similarity coefficient is within the range of a preset minimum positive number to 1, 0.985 is taken as the effective similarity coefficient. The natural logarithm of the effective similarity coefficient is calculated using a logarithmic function with the natural constant as the base, which is approximately -0.0151. Its negative value, 0.0151, is taken as the Bach distance between two adjacent frames.

[0026] In an optional embodiment, obtaining the predicted coordinates of the four corner vertices based on historical vertex coordinates and synchronous motion data includes: By combining synchronized gyroscope angular velocity data, and based on inter-frame sampling time and camera intrinsic parameters, the gyroscope angular velocity data is converted into vertex prediction offsets in pixel coordinate system. Kalman filtering is then used to fuse historical vertex coordinate data and the vertex prediction offsets to obtain the predicted coordinates of the four corner vertices of the current frame document.

[0027] Combining synchronously read angular velocity data from a three-axis gyroscope, with angular velocities around the X, Y, and Z axes set to 0.05 rad / s, -0.02 rad / s, and 0.01 rad / s respectively, Kalman filtering is used to fuse historical vertex coordinate data and angular velocity data. In the filtering prediction stage, the angular velocity data is converted into motion velocity components in the corresponding pixel coordinate system using the camera's intrinsic parameter matrix and focal length. Combined with historical vertex coordinate data from the previous frame and the inter-frame time interval (e.g., 33 milliseconds for compensation offset), a priori prediction state is derived. The four corner vertex coordinates from this priori prediction state are then used as the predicted coordinates for the four corner vertex of the current frame document. In the filtering measurement update stage, the extracted actual coordinate observations are combined with the estimated covariance and measurement noise covariance to calculate the Kalman gain, set to 0.57. This actual coordinate is used to correct the priori state and generate a posterior estimate, which is then used to update the historical vertex coordinates required for the next frame prediction.

[0028] S3. When the sharpness condition, document position stability condition, and constant illumination condition are met simultaneously for the first time, the waiting period is determined. The duration of the period is determined based on the vertex deviation distance between the measured vertex coordinates and the predicted coordinates. If the above conditions are met continuously during the waiting period, the shooting command is triggered.

[0029] The status flags in the status register are continuously monitored at a fixed frame rate. When the sharpness condition flag, document position stability condition flag, and constant illumination condition flag simultaneously change from false jump to true for the first time, the current timestamp is recorded as the initial moment. A base waiting time is set, and the maximum value among the four calculated Euclidean distances is extracted as the vertex deviation distance. The preset time coefficient is multiplied by the vertex deviation distance, and then added to the base waiting time to obtain the waiting period. A high-precision asynchronous timer is started to begin timing. During the entire waiting period of the timer, all the aforementioned feature extraction and judgment steps are continuously performed on each newly arrived frame. If any frame fails to simultaneously meet the above three conditions during the timing period, the current timer is immediately destroyed, and all status flags are reset, waiting for the next initial moment. If the timer runs continuously until the waiting period expires without interruption, proving that the above conditions are continuously met within the period, a single-frame shooting command is sent to the image acquisition device to acquire and save the document image data.

[0030] In an optional embodiment, the step of determining the waiting period when the sharpness condition, document position stability condition, and constant illumination condition are simultaneously met for the first time, wherein the period duration is determined based on the vertex deviation distance between the measured vertex coordinates and the predicted coordinates, includes: Read the actual extracted coordinates of the four corner vertices and the predicted coordinates corresponding to the four corner vertices; Calculate the Euclidean distance between each actual extracted vertex coordinate and its corresponding predicted coordinate, and take the maximum value among the four Euclidean distances as the vertex deviation distance; The waiting period used for the current round of judgment is obtained by multiplying the preset time coefficient by the vertex deviation distance, or by first multiplying the preset time coefficient by the vertex deviation distance and then adding the resulting product to the preset basic waiting time.

[0031] The system reads the actual extracted coordinates of the four vertices and their corresponding predicted coordinates, where each vertex coordinate includes both horizontal and vertical components. Based on Euclidean distance measurement theory, the linear spatial deviation lengths between the four corresponding positions are calculated one by one. Let the deviation lengths for the four positions be 7.07, 5.39, 10.44, and 11.18 pixels, respectively. The vertex deviation distance is obtained based on the maximum value of these four Euclidean distances. For example, when the four Euclidean distances are 7.07, 5.39, 10.44, and 11.18 pixels, the vertex deviation distance is 11.18 pixels. A preset time coefficient of 15 milliseconds per pixel is used. Multiplying the preset time coefficient of 15 by the vertex deviation distance of 11.18 yields a waiting period of 167.7 milliseconds, which is then used as the waiting period for the current round's judgment.

[0032] In the second embodiment, the present invention also proposes an intelligent document capture system that integrates ambient light perception and edge detection, comprising the following modules: The acquisition module is used to acquire real-time video streams collected by image sensors and parse image frames, obtain synchronous motion data corresponding to the timestamps of image frames, perform gradient and local binary mode joint analysis on the current image frame, determine candidate target regions and their texture density values, calculate local sharpness scores within candidate target regions, generate sharpness judgment thresholds based on benchmark values ​​and adjustment factors determined by texture density values, and determine that the sharpness condition is met when the local sharpness score is higher than the threshold. The generation module is used to extract the coordinates of the four corner vertices of the document outline within the candidate target area as measured vertex coordinates, and obtain the predicted coordinates of the four corner vertices based on historical vertex coordinates and synchronous motion data. The distance tolerance is determined in reverse based on the difference between the local sharpness score and the sharpness judgment threshold, and is not lower than the preset minimum distance tolerance. When the vertex deviation distance between the measured vertex coordinates and the predicted coordinates is less than the distance tolerance, the document position stability condition is considered to be met. A background reference area is selected outside the document outline, and the Bartlett distance between adjacent frames is calculated based on the brightness channel histogram of multiple consecutive frames of the background reference area within the time window. When the Bartlett distance between each adjacent frame is less than the brightness perturbation threshold, the illumination constant condition is considered to be met. The determination module is used to determine the waiting period when the sharpness condition, document position stability condition, and constant illumination condition are met simultaneously for the first time. The duration of the period is determined based on the vertex deviation distance between the measured vertex coordinates and the predicted coordinates. If the above conditions are met continuously during the waiting period, a shooting command is triggered.

[0033] The above description represents the preferred embodiments of the present invention. It should be noted that, for those skilled in the art, various improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. An intelligent document capture method integrating ambient light perception and edge detection, characterized in that, include: The system acquires real-time video streams collected by image sensors and parses image frames. It obtains synchronous motion data corresponding to the timestamps of the image frames, performs gradient and local binary mode joint analysis on the current image frame, determines candidate target regions and their texture density values, calculates local sharpness scores within the candidate target regions, and generates sharpness judgment thresholds based on the benchmark value and the adjustment factor determined by the texture density values. When the local sharpness score is higher than the threshold, the sharpness condition is considered to be met. The coordinates of the four corner vertices of the document outline are extracted within the candidate target area as measured vertex coordinates. The predicted coordinates of the four corner vertices are obtained based on the historical vertex coordinates and synchronous motion data. The distance tolerance is determined in reverse based on the difference between the local sharpness score and the sharpness judgment threshold, and it is not lower than the preset minimum distance tolerance. When the vertex deviation distance between the measured vertex coordinates and the predicted coordinates is less than the distance tolerance, the document position stability condition is considered to be met. A background reference area is selected outside the document outline. Based on the brightness channel histogram of multiple consecutive frames of the background reference area within the time window, the Bartl distance between two adjacent frames is calculated. When the Bartl distance between each adjacent frame is less than the brightness perturbation threshold, the illumination constant condition is considered to be met. When the clarity condition, document position stability condition, and constant illumination condition are simultaneously met for the first time, the waiting period is determined. The duration of the period is determined based on the vertex deviation distance between the measured vertex coordinates and the predicted coordinates. If the above conditions are continuously met within the waiting period, a shooting command is triggered.

2. The method according to claim 1, characterized in that, The step of performing joint gradient and local binary pattern analysis on the current image frame to determine candidate target regions and their texture density values ​​includes: The Sobel operator is used to extract the horizontal and vertical gradient data of each pixel in the current image frame. The absolute values ​​of the horizontal and vertical gradient data are added together to obtain the initial gradient magnitude of the current pixel. Using the current pixel as the center point, a nine-grid neighborhood centered on the center point is selected. An eight-bit binary string is generated by comparing the grayscale values ​​of the eight surrounding pixels in the neighborhood with those of the center pixel. The binary string is then converted into a decimal integer as the local binary mode encoding value of the center pixel. The current image frame is evenly divided into multiple independent non-overlapping grid blocks with a resolution of 16×16 pixels. The initial gradient magnitude of all pixels is accumulated in each grid block, and the distribution result of the local binary mode coding value is statistically analyzed. Based on the statistical result of the initial gradient magnitude and the distribution result of the local binary mode coding value, a joint texture score for each grid block is generated. Four spatially continuous but non-isolated grid blocks are traversed as a sliding window to filter out the four grid blocks with the largest sum of joint texture scores. The initial position of the candidate target region is determined based on the four grid blocks, and the candidate target region is enclosed by the connected edge regions around the initial position. The arithmetic mean of the joint texture scores of the four grid blocks is used as the texture density value representing the region.

3. The method according to claim 1, characterized in that, The calculation of local sharpness scores within the candidate target region includes: Extract the red, green, and blue color channel data of each pixel in the candidate target area, and use a color conversion matrix to weight and sum the data of the three channels according to a preset brightness weight to obtain a single-channel grayscale region matrix. A Laplacian second derivative spatial filter template with a scale of 3×3 pixels is selected. The edge pixels of the grayscale region matrix are processed by copy filling or zero filling. The filter template and the processed grayscale region matrix are subjected to two-dimensional matrix convolution traversal pixel by pixel to obtain a response result matrix that represents the degree of drastic change at the edge of each pixel and has the same size as the grayscale region matrix. The mean square value of the detailed response to the candidate target region is obtained by squaring each output pixel value in the response result matrix and taking the arithmetic mean of the squared values. The mean square value of the detail response is used as the local sharpness score.

4. The method according to claim 1, characterized in that, The process of generating a sharpness threshold based on a reference value and an adjustment factor determined by the texture density value includes: Obtain the baseline value; Obtain a preset proportional feedback parameter constant in the control buffer, multiply the proportional feedback parameter constant by the obtained texture density value, and use the resulting product as the adjustment factor; The sharpness determination threshold is obtained by adding the benchmark value to the adjustment factor.

5. The method according to claim 2, characterized in that, The generated distance tolerance is determined by reverse calculation based on the difference between the local sharpness score and the sharpness judgment threshold, and is not lower than a preset minimum distance tolerance, including: Obtain the base distance constant, the preset minimum distance tolerance, the preset maximum distance tolerance, and the preset small positive number used to prevent the denominator from being zero; Subtract the sharpness determination threshold from the local sharpness score to obtain the sharpness margin; The sharpness margin is added to the preset small positive number to obtain the denominator, and the candidate distance tolerance is obtained by dividing the basic distance constant by the denominator. The candidate distance tolerance is compared with the preset minimum distance tolerance, and the maximum value of the two is taken. Then, the result of taking the larger value is compared with the preset maximum distance tolerance, and the minimum value of the two is taken as the distance tolerance.

6. The method according to claim 1, characterized in that, The step of calculating the Bartlett distance between two adjacent frames based on the luminance channel histogram of multiple consecutive frames of the background reference area within the time window includes: Obtain the current recognized image frame and the previous recognized image frame adjacent to the time in the background reference area within the coverage of the time window, and read the brightness level value of each pixel in the brightness channel in these two frames; Two hierarchical histogram arrays, each with 128 data storage scales, are established. The pixel occurrence frequency of the current recognized image frame and the pixel occurrence frequency of the previous recognized image frame are calculated and accumulated separately according to the interval distribution, and then normalized to form a first frequency distribution list and a second frequency distribution list. Using the index codes of the corresponding positions, extract the grid content corresponding to the first frequency distribution list item and the second frequency distribution list item respectively, and perform a group-by-group digital multiplication operation. Take the square root of each product and sum the square root results to obtain the histogram similarity coefficient. The histogram similarity coefficient is restricted to a range from a preset minimum positive number to 1 to obtain an effective similarity coefficient. The negative logarithm of the effective similarity coefficient is then calculated using a logarithmic function and used as the Bach distance between two adjacent frames.

7. The method according to claim 1, characterized in that, The waiting period is determined when the clarity condition, document position stability condition, and constant illumination condition are simultaneously met for the first time. The duration of the period is determined based on the vertex deviation distance between the measured vertex coordinates and the predicted coordinates, including: Read the actual extracted coordinates of the four corner vertices and the predicted coordinates corresponding to the four corner vertices; Calculate the Euclidean distance between each actual extracted vertex coordinate and its corresponding predicted coordinate, and take the maximum value among the four Euclidean distances as the vertex deviation distance; The waiting period used for the current round of judgment is obtained by multiplying the preset time coefficient by the vertex deviation distance, or by first multiplying the preset time coefficient by the vertex deviation distance and then adding the resulting product to the preset basic waiting time.

8. The method according to claim 1, characterized in that, The process of obtaining the predicted coordinates of the four corner vertices based on historical vertex coordinates and synchronous motion data includes: By combining synchronized gyroscope angular velocity data, and based on inter-frame sampling time and camera intrinsic parameters, the gyroscope angular velocity data is converted into vertex prediction offsets in pixel coordinate system. Kalman filtering is then used to fuse historical vertex coordinate data and the vertex prediction offsets to obtain the predicted coordinates of the four corner vertices of the current frame document.

9. An intelligent document capture system integrating ambient light perception and edge detection, characterized in that, include: The acquisition module is used to acquire real-time video streams collected by image sensors and parse image frames, obtain synchronous motion data corresponding to the timestamps of image frames, perform gradient and local binary mode joint analysis on the current image frame, determine candidate target regions and their texture density values, calculate local sharpness scores within candidate target regions, generate sharpness judgment thresholds based on benchmark values ​​and adjustment factors determined by texture density values, and determine that the sharpness condition is met when the local sharpness score is higher than the threshold. The generation module is used to extract the coordinates of the four corner vertices of the document outline within the candidate target area as measured vertex coordinates, and obtain the predicted coordinates of the four corner vertices based on historical vertex coordinates and synchronous motion data. The distance tolerance is determined in reverse based on the difference between the local sharpness score and the sharpness judgment threshold, and is not lower than the preset minimum distance tolerance. When the vertex deviation distance between the measured vertex coordinates and the predicted coordinates is less than the distance tolerance, the document position stability condition is considered to be met. A background reference area is selected outside the document outline, and the Bartlett distance between adjacent frames is calculated based on the brightness channel histogram of multiple consecutive frames of the background reference area within the time window. When the Bartlett distance between each adjacent frame is less than the brightness perturbation threshold, the illumination constant condition is considered to be met. The determination module is used to determine the waiting period when the sharpness condition, document position stability condition, and constant illumination condition are met simultaneously for the first time. The duration of the period is determined based on the vertex deviation distance between the measured vertex coordinates and the predicted coordinates. If the above conditions are met continuously during the waiting period, a shooting command is triggered.

10. The system according to claim 9, characterized in that, The step of performing joint gradient and local binary pattern analysis on the current image frame to determine candidate target regions and their texture density values ​​includes: The Sobel operator is used to extract the horizontal and vertical gradient data of each pixel in the current image frame. The absolute values ​​of the horizontal and vertical gradient data are added together to obtain the initial gradient magnitude of the current pixel. Using the current pixel as the center point, a nine-grid neighborhood centered on the center point is selected. An eight-bit binary string is generated by comparing the grayscale values ​​of the eight surrounding pixels in the neighborhood with those of the center pixel. The binary string is then converted into a decimal integer as the local binary mode encoding value of the center pixel. The current image frame is evenly divided into multiple independent non-overlapping grid blocks with a resolution of 16×16 pixels. The initial gradient magnitude of all pixels is accumulated in each grid block, and the distribution result of the local binary mode coding value is statistically analyzed. Based on the statistical result of the initial gradient magnitude and the distribution result of the local binary mode coding value, a joint texture score for each grid block is generated. Four spatially continuous but non-isolated grid blocks are traversed as a sliding window to filter out the four grid blocks with the largest sum of joint texture scores. The initial position of the candidate target region is determined based on the four grid blocks, and the candidate target region is enclosed by the connected edge regions around the initial position. The arithmetic mean of the joint texture scores of the four grid blocks is used as the texture density value representing the region.

Images