An adaptive photographing time interval dynamic adjustment method based on image matching

Abstract

The application relates to the technical field of image processing, in particular to a self-adaptive photographing time interval dynamic adjustment method based on image matching, which comprises the following steps: collecting the images, running speed and vibration frequency of each product on a conveying belt in real time; judging whether optimization is needed when a characteristic point is matched based on the abnormal degree of the running speed and vibration frequency between adjacent frame images; when optimization is needed, the number of groups when the adjacent frame images are divided in scale is optimized, the contrast threshold of each layer of Gaussian difference images after scale division is optimized, and then image matching is carried out; and the photographing time interval after the current frame is adjusted based on the offset distance between the product in the current frame and the product in the previous frame and the offset distance between the product in the current frame and the product in a template image. The photographing time interval is adaptively adjusted, so that the problems of product offset or loss of key details in the collected images are avoided.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and specifically to an adaptive dynamic adjustment method for image-matching time interval. Background Technology

[0002] In scenarios requiring continuous image acquisition, controlling the image capture interval directly impacts the effectiveness of data acquisition. Existing technologies typically rely on hardware devices such as photoelectric sensors and encoders to trigger image capture at fixed intervals. However, during prolonged continuous acquisition, factors such as environmental vibrations and changes in operating speed can cause the product's position in the captured images to gradually shift, leading to a disconnect between the image capture interval and the product's actual movement. Therefore, dynamically adjusting the image capture interval is crucial.

[0003] Currently, when continuously acquiring images in flexible transmission scenarios with mechanical vibration or unstable conveyor belt speed, traditional image acquisition methods typically set fixed photo intervals. These methods do not fully consider the impact of interference factors such as conveyor belt speed changes or equipment vibrations in actual operating scenarios. As a result, errors gradually accumulate during long-term continuous acquisition, leading to product position shifts in later acquired images. Furthermore, existing image acquisition methods lack precise displacement deviation measurement and dynamic correction mechanisms for photo intervals, ultimately resulting in product shifts or loss of key details in the acquired images, affecting the accuracy and reliability of subsequent data processing. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides an adaptive photo-taking time interval dynamic adjustment method based on image matching, thereby resolving the existing issues.

[0005] The adaptive photo interval dynamic adjustment method based on image matching proposed in this application adopts the following technical solution:

[0006] One embodiment of this application provides an adaptive photo-taking time interval dynamic adjustment method based on image matching, the method comprising the following steps:

[0007] Acquire template images and capture images of each product on the conveyor belt in real time; acquire the conveyor belt's running speed and vibration frequency in real time;

[0008] Based on the degree of abnormality and dispersion of running speed and vibration frequency within a time period between adjacent frame images, a dynamic influence difference value is constructed between adjacent frame images to determine whether adjacent frame images need to be optimized when matching feature points; when optimization is required, based on the dynamic influence difference value, the number of groups of adjacent frame images when performing scale division is calculated, and then the adjacent frame images are scaled.

[0009] Based on the pixel distribution differences between adjacent frame images in the same layer of Gaussian difference images after scale division, the preset initial contrast threshold of each layer of Gaussian difference images in adjacent frame images is adjusted, and then image matching is performed on adjacent frame images.

[0010] Based on the offset distance between the feature points matched between the current frame and the previous frame, and the actual movement distance of the product in the current frame from the time of the previous frame to the current time, the time interval for taking pictures of the current frame is adjusted; based on the offset distance between the current frame and the product in the template image, the adjusted time interval for taking pictures of the current frame is corrected, thereby adjusting the time interval for taking pictures of the subsequent frames.

[0011] Preferably, the method for constructing the dynamic influence difference value between adjacent frame images is as follows:

[0012] Obtain the LOF values ​​of all running speeds and all vibration frequencies within a preset historical time period prior to each frame image;

[0013] The percentage of data with an LOF value greater than 1 in all running speeds between each frame and the previous frame, and the percentage of data with an LOF value greater than 1 in all vibration frequencies;

[0014] Calculate the variance of all running speeds and the variance of all vibration frequencies between each frame and the previous frame.

[0015] The difference in dynamic influence between adjacent frames is positively correlated with the two proportions and the two variances mentioned above.

[0016] Preferably, the specific process for determining whether optimization is needed when matching feature points in adjacent frames is as follows:

[0017] If the difference in dynamic influence between adjacent frames is greater than or equal to a preset judgment threshold, then the adjacent frames are determined to need optimization in feature point matching; otherwise, optimization is not required.

[0018] Preferably, the calculation process for the number of groups when dividing adjacent frame images into scales is as follows: In the formula, The number of groups when scaling the t-th frame and its preceding frame; This is the preset initial number of groups when performing scale division based on the SIFT algorithm; It is the normalized value of the difference between the dynamic influence difference between the t-th frame and the previous frame and the preset judgment threshold; This is the rounding function; The maximum number of groups is preset. This is the function for finding the maximum value.

[0019] Preferably, the preset initial contrast threshold of the Gaussian difference images in adjacent frames is adjusted: In the formula, The contrast threshold is the difference between the j-th layer Gaussian image of the t-th frame and the image of the previous frame. The initial contrast threshold is preset. The normalized result of the dynamic feature coefficients of the j-th layer Gaussian difference image in the t-th frame and the previous frame image; This is a preset adjustment factor.

[0020] Preferably, the method for obtaining the dynamic feature coefficients of the Gaussian difference images of each layer in adjacent frame images is as follows:

[0021] Calculate the standard deviation of all gray values ​​in each column of each layer of the Gaussian difference image, and arrange all the standard deviations in order from left to right, which is recorded as the vertical mapping data sequence of each layer of the Gaussian difference image.

[0022] Calculate the standard deviation of all gray values ​​in each row of each layer of the Gaussian difference image, and arrange all the standard deviations in order from top to bottom, which is recorded as the horizontal mapping data sequence of each layer of the Gaussian difference image.

[0023] Calculate the Manhattan distance between the horizontal and vertical mapping data sequences of adjacent frame images in each layer of Gaussian difference images, respectively.

[0024] The dynamic characteristic coefficients are positively correlated with both Manhattan distances mentioned above.

[0025] Preferably, in the process of image matching between adjacent frames, feature points are extracted using the contrast threshold adjusted by the Gaussian difference images of each layer in the adjacent frames.

[0026] Preferably, the specific process of adjusting the time interval for capturing the current frame image is as follows:

[0027] The average difference between the x-coordinates of all successfully matched feature point pairs between the current frame and the previous frame is calculated.

[0028] Based on the ratio between the pre-acquired image size and the actual product size, the mean value is converted into the actual offset distance;

[0029] Calculate the average running speed of all frames between the current frame and the previous frame, and denote it as the average running speed of the current frame.

[0030] Calculate the product between the time interval of the current frame image and the average running speed to obtain the actual distance the product in the current frame image has moved from the time of the previous frame image to the current time.

[0031] Calculate the time interval between the adjusted images in the current frame: In the formula, The adjusted time interval for capturing the current frame image; This indicates the time interval between the capture of the current frame image; This indicates the actual offset distance between the current frame image and its previous frame image; This represents the actual distance the product in the current frame has moved from the time of its previous frame to the current time.

[0032] Preferably, the specific process of correcting the adjusted time interval of the current frame image is as follows:

[0033] Match the current frame image with the template image;

[0034] Calculate the difference in x-coordinates between all feature point pairs between the current frame image and the template image, and convert this difference into the actual offset distance according to the ratio between the pre-obtained image size and the actual product size. This is recorded as the actual template offset distance of the product in the current frame image relative to the product in the template image.

[0035] Calculate the time interval between image captures after correction for the current frame: In the formula, This indicates the time interval between the corrected images of the current frame. The adjusted time interval for capturing the current frame image; This represents the actual template offset distance between the product in the current frame image and the product in the template image. This represents the average running speed of the current frame image. This is a preset constant.

[0036] Preferably, the specific process of adjusting the time interval for taking pictures after the current frame image is as follows: the calculated corrected time interval for taking pictures of the current frame image is used as the time interval for taking pictures of subsequent images.

[0037] This application has at least the following beneficial effects:

[0038] Traditional continuous frame image acquisition typically uses fixed time intervals for capturing images, failing to adequately consider the impact of various interference factors in real-world scenarios, leading to significant product position shifts in later acquired images. To address this issue, this application proposes an adaptive dynamic adjustment method for the capture time interval based on image matching. By analyzing abnormal changes in the running speed and vibration frequency between adjacent frames, a dynamic influence difference value is constructed to determine whether optimization is needed during feature point matching. When optimization is required, the number of scale space groups and the contrast threshold in the SIFT algorithm are adjusted based on the dynamic influence difference value and the pixel distribution differences between adjacent frames, resulting in more accurate subsequent image matching results and a more accurate calculation of the actual product offset distance in the current frame. Based on the actual product offset distance of the current frame relative to the previous frame, the capture time interval of the current frame is adjusted. Furthermore, based on the actual product offset distance of the current frame relative to the template image, cumulative error correction is applied to the adjusted capture time interval, thus adaptively obtaining the capture time interval for subsequent image acquisition. This improves the accuracy of the capture time interval adjustment and avoids product shifts or loss of key details in the acquired images. Attached Figure Description

[0039] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart illustrating the steps of an adaptive photo-taking time interval dynamic adjustment method based on image matching provided in this application;

[0041] Figure 2 This is a flowchart for image matching of adjacent frames provided in this application. Detailed Implementation

[0042] To further illustrate the technical means and effects adopted by this application to achieve the intended inventive objective, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an adaptive photo interval dynamic adjustment method based on image matching proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0044] The following description, in conjunction with the accompanying drawings, details a specific scheme for an adaptive photo-taking time interval dynamic adjustment method based on image matching provided in this application.

[0045] This application provides an embodiment of an adaptive image capture time interval dynamic adjustment method based on image matching. Specifically, it provides the following adaptive image capture time interval dynamic adjustment method based on image matching. Please refer to [link to relevant documentation]. Figure 1 The method includes the following steps:

[0046] Step 1: Obtain the template image and collect images of each product on the conveyor belt in real time; obtain the running speed and vibration frequency of the conveyor belt in real time.

[0047] When collecting image data of products at fixed intervals on a production line conveyor belt, image acquisition equipment is typically installed at a fixed location. Products appearing within the field of view are photographed at fixed time intervals to ensure that each product is in a fixed position within the image. However, when the conveyor belt speed changes or the equipment vibrates, the fixed photographing time interval cannot adapt to these changes. Furthermore, prolonged operation can lead to error accumulation, causing product shifts in the captured images. In severe cases, it can result in incomplete product capture, affecting the acquisition of product images.

[0048] To achieve precise adjustment of the time interval and reduce cumulative impact, it is first necessary to establish a unit template to determine a standard and stable reference benchmark. Specifically, in practical applications, a target product with no defects and standard posture is used as a template sample. A high-definition industrial camera is used to capture the front image of the template sample, thereby obtaining the template image. During the acquisition process, it is ensured that the distance and angle between the camera and the product, as well as the lighting conditions, are under standard working conditions.

[0049] Furthermore, a preset time interval for continuous image acquisition is set. At the start of image acquisition, the image acquisition device acquires images of each product on the conveyor belt at the preset time interval. During acquisition, the acquisition angle is aligned with the acquisition angle of the product template image to obtain a continuous and valid image sequence of the product's movement. Simultaneously, the conveyor belt's running speed and the device's vibration frequency are acquired in real time for auxiliary reference during time interval calibration. The acquisition frequency is fHz, and in this embodiment, f is 50. It should be noted that if an acquisition interruption occurs, acquisition must be immediately restarted, and keyframe images of the interrupted time period must be reacquired to ensure data integrity.

[0050] Furthermore, the acquired images are preprocessed to reduce the impact of noise interference during the acquisition process on image quality. Specifically, for each acquired image frame, a Gaussian filtering algorithm is used to denoise each frame, reducing the impact of random noise interference, and a median filtering algorithm is used to reduce the interference of salt-and-pepper noise in each frame. Based on the above denoising processing, the processed image is converted into a grayscale image. The Gaussian filtering algorithm and the median filtering algorithm are well known to those skilled in the art, and their detailed processes will not be elaborated further.

[0051] Step 2: Based on the degree of abnormality and dispersion of running speed and vibration frequency within the time period between adjacent frame images, construct the dynamic influence difference value between adjacent frame images to determine whether the adjacent frame images need to be optimized when matching feature points; when optimization is required, calculate the number of groups of the adjacent frame images when performing scale division based on the dynamic influence difference value, and then perform scale division on the adjacent frame images.

[0052] Furthermore, after acquiring consecutive frame images of the product, existing methods typically perform image matching on adjacent frames, calculate the offset distance between consecutive frames based on the matching results, and then dynamically adjust the shooting time interval according to the calculation and analysis results. However, in actual processing, during continuous image acquisition, dynamic changes occur due to equipment vibration, speed, and product surface condition at different times, leading to varying degrees of differences in adjacent frame images acquired at different time points, thus affecting the image matching effect between adjacent frames. Therefore, to accurately achieve dynamic adjustment of the shooting time interval, it is necessary to analyze the dynamic changes during continuous image acquisition, thereby improving the accuracy of feature point extraction and matching of consecutive image frames, and ultimately improving the accuracy of subsequent shooting time interval adjustments.

[0053] Specifically, considering the significant differences in dynamic impact characteristics of consecutive frame images at different time points during actual acquisition, the feature point extraction and matching of adjacent frame images at different time points are dynamically optimized and adjusted based on changes in running speed and equipment vibration frequency during image matching analysis. Specifically, for any set of adjacent frame images during continuous acquisition, taking frame t as an example, all running speeds and vibration frequencies within a preset historical time period (set to 5 seconds in this embodiment) preceding frame t are used as inputs. The LOF anomaly detection algorithm is used to obtain the LOF values ​​of all running speeds and all vibration frequencies. The percentages of data with LOF values ​​greater than 1 for all running speeds and all vibration frequencies between frame t and its preceding frame are statistically analyzed. The larger these two percentages are, the greater the difference in dynamic impact between frame t and its preceding frame due to abnormal running speed and vibration frequency changes.

[0054] Furthermore, considering the differences in product motion and equipment operating states at different time stages during continuous data acquisition, the degree of dynamic influence between adjacent frames is analyzed. Specifically, the variances of all running speeds and all vibration frequencies are calculated within the time interval between frame t and its preceding frame. The larger these two variances, the greater the fluctuation in product running speed and equipment vibration frequency within the time interval between frame t and its preceding frame, and thus the greater the difference in dynamic influence between these two frames during data acquisition.

[0055] To eliminate the impact of dimensional inconsistencies on subsequent calculations, the variances of the running speeds and vibration frequencies between the t-th frame and its preceding frame are normalized using the minimum and maximum variances of the variances of all running speeds and vibration frequencies within a historical preset time period. The minimum-maximum normalization method is a well-known technique, and its specific process will not be elaborated upon.

[0056] As a preferred implementation, a dynamic impact difference value is constructed based on the degree of anomaly and dispersion of running speed and vibration frequency within the time period between adjacent frame images. This value is used to characterize the degree of difference in the dynamic impact of equipment vibration and product operating status during the acquisition of adjacent frame images. The method for constructing the dynamic impact difference value between adjacent frame images is as follows: Statistically calculate the proportion of data with an LOF value greater than 1 for all running speeds and the proportion of data with an LOF value greater than 1 for all vibration frequencies between each frame and its preceding frame; Statistically calculate the variance of all running speeds and the variance of all vibration frequencies between each frame and its preceding frame. The dynamic impact difference value between adjacent frame images is positively correlated with both of the above proportions and variances. This positive correlation means that the dependent variable increases (decreases) as the independent variable increases (decreases).

[0057] In this embodiment, the difference in dynamic influence between the t-th frame and its previous frame is denoted as . The specific calculation formula is as follows: In the formula, This represents the difference in dynamic influence between the t-th frame and its preceding frame. This represents the percentage of data with an LOF value greater than 1 across all running speeds during the time interval between frame t and its preceding frame. This represents the percentage of data with an LOF value greater than 1 among all vibration frequencies during the time interval between the t-th frame and its preceding frame. This represents the normalized value of the variance of all running speeds within the time interval between the t-th frame and its previous frame. This represents the normalized value of the variance of all vibration frequencies during the time interval between the t-th frame and its previous frame.

[0058] Calculated The larger the value, the greater the difference between the t-th frame and the previous frame during the acquisition process due to equipment vibration and the dynamic influence of product running speed.

[0059] Furthermore, a preset judgment threshold is set. In this embodiment, the calculation process of the preset judgment threshold is as follows: calculate the average value of the dynamic influence difference between all adjacent frame images within a preset historical time period, and use the obtained average value as the preset judgment threshold to determine whether adjacent frames need to be optimized and adjusted during feature point extraction and matching. If the dynamic influence difference between frame t and its previous frame image is greater than or equal to the preset judgment threshold, it is determined that frame t and its previous frame image need to be optimized during feature point matching; otherwise, it is determined that no optimization is needed.

[0060] Based on the above judgment results, for situations where the feature point matching analysis process of adjacent frame images needs to be optimized and adjusted, firstly, under the conditions of equipment vibration and changes in product running speed, the local feature differences between adjacent frame images will be large due to dynamic influences; therefore, based on the difference value of dynamic influences during the acquisition of adjacent frame images, the number of octaves in the scale division of adjacent frame images based on the SIFT algorithm is adjusted, so that the local detailed features of adjacent frame images are highlighted under the condition of large differences in dynamic influences, thereby improving the accuracy of subsequent feature point extraction and matching analysis.

[0061] Furthermore, based on the difference in dynamic influence between adjacent frames, the number of groups for scale division of adjacent frames is determined first, and the specific calculation formula is as follows:

[0062] In the formula, The number of groups when scaling the t-th frame and its preceding frame; In this embodiment, the initial number of groups is set to 3 to represent the preset number of groups when performing scale division based on the SIFT algorithm. It is the normalized value of the difference between the dynamic influence difference between the t-th frame and the previous frame and the preset judgment threshold. The normalization method is: based on the minimum and maximum values ​​of all the differences within the preset historical time period, the minimum-maximum normalization method is used for normalization. This is a rounding function used to round the calculation result to the nearest integer. The maximum number of groups is preset, and its specific value needs to be set according to the actual image resolution. In this embodiment, it is set to 5. This is the function for finding the maximum value.

[0063] The larger the value, the greater the influence of device vibration and motion speed changes between the t-th frame and the previous frame during the acquisition process. Therefore, a larger number of groups should be set when dividing these two frames into scales, so that the algorithm can detect richer feature points in different scale spaces, capture features of different sizes in the image more precisely, and avoid the problem of large deviations in subsequent image feature matching caused by differences in dynamic influence.

[0064] Furthermore, based on the number of groups in the scale division of adjacent frame images determined above, and the preset number of layers in each group (the preset number of layers is set to 4 in this embodiment), scale division is performed between adjacent frame images, and then a Gaussian difference pyramid is constructed. Finally, two adjacent frame images are divided into several groups, and each group contains several layers of Gaussian difference images.

[0065] Step 3: Based on the pixel distribution differences between the Gaussian difference images of the same layer after scale division of adjacent frame images, the preset initial contrast threshold of each layer of Gaussian difference images in adjacent frame images is adjusted, and then image matching is performed on adjacent frame images.

[0066] For each layer of Gaussian difference image after scale division between adjacent frames, considering the influence of product motion and abnormal equipment vibration during image acquisition, the local features of product dynamic changes reflected in the same layer of Gaussian difference image between adjacent frames may differ. Based on these features, the standard deviation of all gray values ​​in each column of each layer of Gaussian difference image is calculated, and all standard deviations are arranged in order from left to right, denoted as the vertical mapping data sequence of each layer of Gaussian difference image; the standard deviation of all gray values ​​in each row of each layer of Gaussian difference image is calculated, and all standard deviations are arranged in order from top to bottom, denoted as the horizontal mapping data sequence of each layer of Gaussian difference image. The standard deviation of each row or column reflects the pixel distribution characteristics of the corresponding row or column. The greater the difference in pixel distribution between Gaussian layers in adjacent frames due to differences in product movement speed and equipment vibration, the more significant the difference in dynamic features between the two frames in each Gaussian layer. Therefore, it is necessary to highlight the detailed features of each Gaussian layer to improve the accuracy of feature point matching between subsequent adjacent frames, thereby improving the accuracy of offset distance calculation between adjacent frames.

[0067] Specifically, the Manhattan distance between the horizontal mapping data sequences of frame t and its previous frame in each layer of the Gaussian difference image is calculated, as well as the Manhattan distance between the vertical mapping data sequences. The dynamic feature coefficients of frame t and its previous frame in each layer of the Gaussian difference image are positively correlated with both of the above Manhattan distances. The calculation of the Manhattan distance is a well-known technique, and the specific process will not be elaborated further.

[0068] In this embodiment, the sum of the two Manhattan distances is used as the dynamic feature coefficient of the t-th frame and its previous frame in each layer of Gaussian difference image. The larger the obtained dynamic feature coefficient, the more significant the difference in dynamic features between the t-th frame and its previous frame in the corresponding Gaussian layer. Therefore, it is necessary to increase the contrast threshold of the corresponding Gaussian difference image to filter out unstable low-contrast feature points, thereby improving the accuracy of subsequent feature point extraction and matching.

[0069] Furthermore, based on the maximum and minimum values ​​of all dynamic feature coefficients between all adjacent frames within a preset historical time period, the min-max normalization method is used to obtain the normalized results of the dynamic feature coefficients of the Gaussian difference images at each layer between adjacent frames. The purpose is to dynamically adjust the contrast threshold by combining the normalized results of the dynamic influence difference features between the t-th frame and its previous frame at all Gaussian layers, thereby accurately extracting feature points at different scales.

[0070] As a preferred implementation, the preset initial contrast threshold of each layer of Gaussian difference images in adjacent frame images is adjusted based on the pixel distribution difference between each layer of Gaussian difference images after scale division.

[0071] In this embodiment, the contrast threshold of the j-th layer Gaussian difference image between the t-th frame and the previous frame is denoted as . The specific calculation expression is as follows: In the formula, The contrast threshold is the difference between the j-th layer Gaussian image of the t-th frame and the image of the previous frame. To preset the initial contrast threshold, in this embodiment the preset initial contrast threshold is 0.02; The normalized result of the dynamic feature coefficients of the j-th layer Gaussian difference image in the t-th frame and the previous frame image; The preset adjustment factor is used to adjust the range of increase of the contrast threshold to prevent the adjusted contrast threshold from being too large. Its value range is [0.03, 0.08], and the value in this embodiment is 0.05.

[0072] The calculation logic for the above contrast threshold is as follows: the more significant the difference in dynamic influence between the Gaussian difference images of adjacent frames, the higher the contrast threshold of the corresponding Gaussian difference image should be, so as to filter out unstable low-contrast feature points and retain high-contrast features.

[0073] Furthermore, adjacent frame images are used as inputs to the SIFT algorithm. The contrast thresholds of each layer of the Gaussian difference image in the calculated adjacent frame images are used as the contrast thresholds of each layer of the Gaussian difference image. The feature points in these two frame images are output, and feature point matching is performed. The SIFT algorithm and the feature point matching process are well known to those skilled in the art, and the specific process will not be described in detail. After feature point matching, the number of successfully matched feature point pairs between adjacent frame images is counted.

[0074] The flowchart for image matching between adjacent frames is as follows: Figure 2 As shown.

[0075] Step 4: Based on the offset distance between the feature points matched between the current frame and the previous frame, and the actual movement distance of the product in the current frame from the time of the previous frame to the current time, adjust the shooting time interval of the current frame; based on the offset distance between the current frame and the product in the template image, correct the adjusted shooting time interval of the current frame, thereby adjusting the shooting time interval after the current frame.

[0076] Furthermore, in this embodiment, the conveyor belt moves horizontally to the right in the image acquisition field of view, so the horizontal rightward direction in the image coordinate system is set as the positive direction. For each pair of feature points that are successfully matched in the t-th frame and its previous frame, the coordinates of each feature point in the corresponding image are counted. The average difference between the x-coordinates of all feature point pairs in the t-th frame and its previous frame is recorded as the coordinate offset distance between the t-th frame and its previous frame. Then, based on the ratio between the pre-obtained image size and the actual product size, the coordinate offset distance is converted into an actual offset distance. The actual offset distance reflects the actual offset of the product in the t-th frame image relative to the product in the previous frame image.

[0077] It should be noted that the actual offset distance is signed data. When the actual offset distance is positive, it means that the product speed in the t-th frame is faster than the previous frame, so the image acquisition time interval should be reduced. When the actual offset distance is negative, it means that the product speed in the t-th frame is slower than the previous frame, so the image acquisition time interval should be increased. When the actual offset distance is zero, it means that the product speed in the t-th frame is the same as the previous frame, so the image acquisition time interval remains unchanged.

[0078] The actual offset distance calculated after optimizing the feature point matching process can reduce the problem of low offset distance calculation accuracy caused by dynamic differences during continuous acquisition, thereby improving the accuracy of time interval adjustment and making the adjustment result more accurately match the actual movement state of the product.

[0079] Furthermore, based on the actual offset distance between the current frame image and its previous frame image, the time interval for taking pictures at the next sampling moment is calibrated. Specifically, the average of all running speeds between the current frame image and its previous frame image is calculated and denoted as the average running speed of the current frame image; and the product between the time interval for taking pictures of the current frame image and the average running speed is calculated to obtain the actual distance the product in the current frame image has moved from the moment of its previous frame image to the current moment.

[0080] Based on the offset distance between the feature point pairs matched between the current frame and its previous frame, and the actual distance the product in the current frame has moved from the time of its previous frame to the current time, the time interval for capturing the current frame is adjusted. The adjustment formula is as follows: In the formula, The adjusted time interval for capturing the current frame image; This indicates the time interval between the capture of the current frame image; This indicates the actual offset distance between the current frame image and its previous frame image; This represents the actual distance the product in the current frame has moved from the time of its previous frame to the current time.

[0081] Furthermore, in the above processing, the time interval calibration between consecutive frames can only reduce the deviation between the current frame and the previous frame, but cannot reduce the error that gradually accumulates during long-term continuous operation.

[0082] Based on the above analysis, this application uses the template image as a reference benchmark. By matching and comparing the current frame image with the template image, it aims to reduce error accumulation during continuous operation and improve the consistency of image position during continuous acquisition. Specifically, the SIFT algorithm is used to perform image matching between the current frame image and the template image to obtain the feature point matching results between the two images.

[0083] Furthermore, based on the above secondary matching results, the difference in x-coordinates between all feature point pairs between the current frame image and the template image is calculated, and this difference is converted into an actual offset distance according to the ratio between the pre-acquired image size and the actual product size. This is recorded as the actual template offset distance of the product in the current frame image relative to the product in the template image. The obtained actual template offset distance represents the cumulative distance deviation of the product in the current frame image relative to the template image during continuous acquisition.

[0084] Based on the offset distance between the product in the current frame image and the template image, cumulative error correction is performed on the adjusted shooting time interval of the current frame image. The correction formula is as follows: In the formula, This indicates the time interval between the corrected images of the current frame. The adjusted time interval for capturing the current frame image; This represents the actual template offset distance between the product in the current frame image and the product in the template image. This represents the average running speed of the current frame image. This is a preset constant.

[0085] Furthermore, the calculated corrected image capture time interval for the current frame is used as the capture time interval for subsequent image acquisitions. That is, when the actual template offset distance is positive and the value is larger, the capture time interval should be reduced to compensate for the displacement deviation of the product center in the image; when the actual offset distance is negative and the value is smaller, the capture time interval should be increased.

[0086] To prevent excessive computation caused by excessively high frequency of time interval adjustment, the above-mentioned dynamic correction method for photo interval is implemented once every N frames (N is 50 in this embodiment).

[0087] Thus, a method for dynamically adjusting the adaptive photo capture time interval based on image matching is completed.

[0088] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments of this specification have been described above. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0089] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0090] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them; modifications to the technical solutions described in the foregoing embodiments, or equivalent substitutions of some of the technical features, do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for dynamically adjusting the adaptive photo capture time interval based on image matching, characterized in that, The method includes the following steps: Acquire template images and capture images of each product on the conveyor belt in real time; acquire the conveyor belt's running speed and vibration frequency in real time; Based on the degree of abnormality and dispersion of running speed and vibration frequency within a time period between adjacent frame images, a dynamic influence difference value is constructed between adjacent frame images to determine whether adjacent frame images need to be optimized when matching feature points; when optimization is required, based on the dynamic influence difference value, the number of groups of adjacent frame images when performing scale division is calculated, and then the adjacent frame images are scaled. Based on the pixel distribution differences between adjacent frame images in the same layer of Gaussian difference images after scale division, the preset initial contrast threshold of each layer of Gaussian difference images in adjacent frame images is adjusted, and then image matching is performed on adjacent frame images. Based on the offset distance between the feature points matched between the current frame and the previous frame, and the actual movement distance of the product in the current frame from the time of the previous frame to the current time, the time interval for taking pictures of the current frame is adjusted; based on the offset distance between the current frame and the product in the template image, the adjusted time interval for taking pictures of the current frame is corrected, thereby adjusting the time interval for taking pictures of the subsequent frames.

2. The adaptive photo interval dynamic adjustment method based on image matching as described in claim 1, characterized in that, The method for constructing the dynamic influence difference value between adjacent frame images is as follows: Obtain the LOF values ​​of all running speeds and all vibration frequencies within a preset historical time period prior to each frame image; The percentage of data with an LOF value greater than 1 in all running speeds between each frame and the previous frame, and the percentage of data with an LOF value greater than 1 in all vibration frequencies; Calculate the variance of all running speeds and the variance of all vibration frequencies between each frame and the previous frame. The difference in dynamic influence between adjacent frames is positively correlated with the two proportions and the two variances mentioned above.

3. The adaptive photo-taking time interval dynamic adjustment method based on image matching as described in claim 1, characterized in that, The specific process for determining whether optimization is needed when matching feature points in adjacent frames is as follows: If the difference in dynamic influence between adjacent frames is greater than or equal to a preset judgment threshold, then the adjacent frames are determined to need optimization in feature point matching; otherwise, optimization is not required.

4. The adaptive photo-taking time interval dynamic adjustment method based on image matching as described in claim 1, characterized in that, The calculation process for the number of groups in adjacent frame images during scale division is as follows: In the formula, The number of groups when scaling the t-th frame and its preceding frame; This is the preset initial number of groups when performing scale division based on the SIFT algorithm; It is the normalized value of the difference between the dynamic influence difference between the t-th frame and the previous frame and the preset judgment threshold; This is the rounding function; The maximum number of groups is preset; This is the function for finding the maximum value.

5. The adaptive photo interval dynamic adjustment method based on image matching as described in claim 1, characterized in that, The preset initial contrast threshold of each layer of Gaussian difference images in adjacent frame images is adjusted as follows: In the formula, The contrast threshold is the difference between the j-th layer Gaussian image of the t-th frame and the image of the previous frame. The initial contrast threshold is preset. The normalized result of the dynamic feature coefficients of the j-th layer Gaussian difference image in the t-th frame and the previous frame image; This is a preset adjustment factor.

6. The adaptive photo interval dynamic adjustment method based on image matching as described in claim 5, characterized in that, The method for obtaining the dynamic feature coefficients of the Gaussian difference images of each layer in adjacent frames is as follows: Calculate the standard deviation of all gray values ​​in each column of each layer of the Gaussian difference image, and arrange all the standard deviations in order from left to right, which is recorded as the vertical mapping data sequence of each layer of the Gaussian difference image. Calculate the standard deviation of all gray values ​​in each row of each layer of the Gaussian difference image, and arrange all the standard deviations in order from top to bottom, which is recorded as the horizontal mapping data sequence of each layer of the Gaussian difference image. Calculate the Manhattan distance between the horizontal and vertical mapping data sequences of adjacent frame images in each layer of Gaussian difference images, respectively. The dynamic characteristic coefficients are positively correlated with both Manhattan distances mentioned above.

7. The adaptive photo-taking time interval dynamic adjustment method based on image matching as described in claim 1, characterized in that, In the process of image matching between adjacent frames, feature points are extracted using the contrast threshold adjusted by the Gaussian difference images of each layer in the adjacent frames.

8. The adaptive photo interval dynamic adjustment method based on image matching as described in claim 1, characterized in that, The specific process of adjusting the time interval for capturing the current frame image is as follows: The average difference between the x-coordinates of all successfully matched feature point pairs between the current frame and the previous frame is calculated. Based on the ratio between the pre-acquired image size and the actual product size, the mean value is converted into the actual offset distance; Calculate the average running speed of all frames between the current frame and the previous frame, and denote it as the average running speed of the current frame. Calculate the product between the time interval of the current frame image and the average running speed to obtain the actual distance the product in the current frame image has moved from the time of the previous frame image to the current time. Calculate the time interval between the adjusted images in the current frame: In the formula, The adjusted time interval for capturing the current frame image; This indicates the time interval between the capture of the current frame image; This indicates the actual offset distance between the current frame image and its previous frame image; This represents the actual distance the product in the current frame has moved from the time of its previous frame to the current time.

9. The adaptive photo interval dynamic adjustment method based on image matching as described in claim 8, characterized in that, The specific process of correcting the adjusted time interval of the current frame image is as follows: Match the current frame image with the template image; Calculate the difference in x-coordinates between all feature point pairs between the current frame image and the template image, and convert this difference into the actual offset distance according to the ratio between the pre-obtained image size and the actual product size. This is recorded as the actual template offset distance of the product in the current frame image relative to the product in the template image. Calculate the time interval between image captures after correction for the current frame: In the formula, This indicates the time interval between the corrected images of the current frame. The adjusted time interval for capturing the current frame image; This represents the actual template offset distance between the product in the current frame image and the product in the template image. This represents the average running speed of the current frame image. This is a preset constant.

10. The adaptive photo-taking time interval dynamic adjustment method based on image matching as described in claim 1, characterized in that, The specific process of adjusting the time interval for taking pictures after the current frame image is as follows: the calculated corrected time interval for taking pictures of the current frame image is used as the time interval for taking pictures of subsequent images.

Images