A method, system, device and medium for judging zero displacement in photoelectric navigation
By using prediction vectors and correlation calculations in photoelectric navigation devices to determine whether the device is undergoing relative motion, the problem of zero-displacement output error in photoelectric navigation is solved, accurate output is achieved when there is no relative motion, and the stability of the device is improved.
Patent Information
- Application Number
- CN202211322524.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-27
- Publication Date
- 2025-12-23
- Estimated Expiration
- 2042-10-27
AI Technical Summary
In existing photoelectric navigation technologies, motion estimation algorithms are prone to errors when the device is not in relative motion, leading to abnormal motion trajectories and an inability to correctly output zero displacement.
By determining the correlation between the current target frame and the reference frame in the optoelectronic navigation device, the search area is determined using the prediction vector, and the correlation calculation is performed within this area. The optimal correlation value is compared with the baseline correlation value to determine whether the device has undergone relative motion, thereby outputting the correct zero displacement.
When there is no relative motion between the photoelectric navigation device and the device, it can accurately output zero displacement, which improves the accuracy of the motion estimation algorithm, avoids error loops, and ensures the stability of the device.
Smart Images
Figure CN115661191B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of photoelectric navigation technology, in particular to a method, system, device and medium for judging zero displacement in photoelectric navigation. BACKGROUND
[0002] In photoelectric navigation technology, a commonly used motion estimation method is as follows: a matching interval is preset, then correlation calculation is performed on a reference frame and a target frame to find an optimal matching block, and a motion vector is output according to the relative position of the optimal matching block. The "preset matching interval" is determined according to a predicted vector, and the interval should make the reference frame and the target frame have the strongest correlation (theoretically, the image parts have overlap).
[0003] If an error occurs in the process of correlation calculation, the subsequent prediction may be wrong, and the operation enters a vicious error cycle. Even if the device and the target object have no relative motion, the error does not stop. For example, the existing correlation calculation method: Anhua uses a bowl-shaped algorithm for judgment. That is, the matching results S1-S9 are placed in the middle, and the optimal value (denoted as Sopt) is placed in the middle, and the other results are placed around. If Sopt is much smaller (or larger, depending on the algorithm) than the other values, then the bowl is a deep bowl, and this matching is a good matching. In this method, since there is no standard threshold, comparing images with high and low contrast, it can be found that for images with high contrast, even if the matching is wrong, the depth of the bowl is greater than the result obtained in the image sequence with low contrast.
[0004] In the image estimation algorithm of photoelectric navigation, there is a predicted vector and a matching process, and there is a possibility of error in prediction and matching. For a single or small amount of error, the multi-point matching algorithm itself has a correction effect; but for continuous multiple errors, it will lead to the fact that the matching cannot be positioned to the correct interval and the optimal correlation module is found. The result is: the motion trajectory is completely abnormal. For example, in the mouse application, it is manifested as: even if the mouse has been stationary, the cursor is flying everywhere.
[0005] Therefore, there is an urgent need for a method for judging zero displacement in photoelectric navigation, which can correctly output zero displacement when the photoelectric navigation device has no relative motion. SUMMARY
[0006] The purpose of the present application is to provide a method, system, device and medium for judging zero displacement in photoelectric navigation, so as to correctly output zero displacement when the photoelectric navigation device has no relative motion.
[0007] To achieve the above purpose, the present application provides the following solutions:
[0008] A method for judging zero displacement in photoelectric navigation, the judging method comprising:
[0009] taking an electronic image acquired by the target photoelectric navigation device at a current time as a current target frame;
[0010] determining a prediction vector according to the output vectors of the historical time; the prediction vector is an output vector of a previous time or a linear relationship of output vectors of a plurality of continuous historical times before the current time;
[0011] determining a search area between the current reference frame and the current target frame according to the prediction vector; the current reference frame is a previous target frame or an original reference frame; the previous target frame is an electronic image acquired by the target photoelectric navigation device at a previous time; the original reference frame is determined according to an electronic image acquired by the target photoelectric navigation device at a historical time; the search area is an overlapping area between the current reference frame and the current target frame;
[0012] determining a reference block in the search area corresponding to the current reference frame;
[0013] determining a reference target block and a plurality of to-be-matched target blocks in the search area corresponding to the current target frame; the plurality of to-be-matched target blocks include a center target block and a plurality of neighborhood target blocks; the center target block is an area overlapping with the reference block on the current target frame; the neighborhood target block is an area obtained by moving around the center target block; the reference target block is an area on the current target frame having the same position and size as the reference block;
[0014] respectively performing correlation operations on each of the to-be-matched target blocks and the reference block to determine an optimal correlation value, and taking the to-be-matched target block corresponding to the optimal correlation value as an optimal matching block; the optimal correlation value is a minimum value of a sum of absolute errors of pixel points or a maximum value of a similar quantity;
[0015] calculating a current motion vector of the optimal matching block relative to the reference block;
[0016] performing a correlation operation on the reference target block and the reference block to obtain a reference correlation value; when the optimal correlation value is the minimum value of the sum of absolute errors of pixel points, the reference correlation value is the sum of absolute errors of pixel points of the reference target block and the reference block; when the optimal correlation value is the maximum value of the similar quantity, the reference correlation value is the similar quantity of pixel points of the reference target block and the reference block;
[0017] determining whether the target photoelectric navigation device has a relative motion at the current time according to the optimal correlation value and the reference correlation value;
[0018] If relative motion occurs, output the current motion vector; if no relative motion occurs, output zero displacement.
[0019] Optionally, the determining whether the target photoelectric navigation device has relative motion at the current time according to the optimal correlation value and the reference correlation value specifically comprises:
[0020] comparing the optimal correlation value with the reference correlation value;
[0021] When the optimal correlation value is the minimum value of the sum of absolute errors, if the optimal correlation value is greater than or equal to the reference correlation value, it is determined that the target photoelectric navigation device has no relative motion at the current time; otherwise, it is determined that the target photoelectric navigation device has relative motion at the current time.
[0022] When the optimal correlation value is the maximum value of the number of similarities, if the optimal correlation value is less than or equal to the reference correlation value, it is determined that the target photoelectric navigation device has no relative motion at the current time; otherwise, it is determined that the target photoelectric navigation device has relative motion at the current time.
[0023] Optionally, the determining whether the target photoelectric navigation device has relative motion at the current time according to the optimal correlation value and the reference correlation value specifically comprises:
[0024] comparing the optimal correlation value with the reference correlation value;
[0025] When the optimal correlation value is the minimum value of the sum of absolute errors, if the optimal correlation value at all times within a set time interval is greater than or equal to the reference correlation value, it is determined that the target photoelectric navigation device has no relative motion at the current time; otherwise, it is determined that the target photoelectric navigation device has relative motion at the current time.
[0026] When the optimal correlation value is the maximum value of the number of similarities, if the optimal correlation value at all times within a set time interval is less than or equal to the reference correlation value, it is determined that the target photoelectric navigation device has no relative motion at the current time; otherwise, it is determined that the target photoelectric navigation device has relative motion at the current time; the starting time of the set time interval is the next time of the time at which the target photoelectric navigation device has relative motion before the current time.
[0027] Optionally, the determining whether the target photoelectric navigation device has relative motion at the current time according to the optimal correlation value and the reference correlation value specifically comprises:
[0028] comparing the optimal correlation value with the reference correlation value;
[0029] When the optimal correlation value is the minimum value of the absolute error sum, if the number of times that the optimal correlation value is greater than or equal to the reference correlation value within a set time interval is greater than a set value, it is determined that the target photoelectric navigation device does not have relative motion at the current time; otherwise, it is determined that the target photoelectric navigation device has relative motion at the current time.
[0030] When the optimal correlation value is the maximum value of the similar quantity, if the number of times that the optimal correlation value is less than or equal to the reference correlation value within a set time interval is greater than a set value, it is determined that the target photoelectric navigation device does not have relative motion at the current time; otherwise, it is determined that the target photoelectric navigation device has relative motion at the current time; the starting time of the set time interval is the next time of the time when the target photoelectric navigation device has relative motion before the current time.
[0031] Optionally, the method further comprises:
[0032] determining an actual motion vector of the previous target frame relative to the current reference frame;
[0033] determining a predicted motion vector of the current target frame relative to the current reference frame according to the actual motion vector and the prediction vector;
[0034] determining a search area between the current reference frame and the current target frame according to the predicted motion vector.
[0035] Optionally, before the step of determining the search area between the current reference frame and the current target frame according to the prediction vector, the method further comprises:
[0036] judging whether the size of the search area between the previous reference frame and the current target frame is within a set size range;
[0037] if yes, taking the previous target frame as the current reference frame;
[0038] if no, taking the original reference frame as the current reference frame.
[0039] Optionally, the correlation operation is performed by using an SAD algorithm.
[0040] The application further provides a zero displacement judgment system in photoelectric navigation, which is applied to the zero displacement judgment method in photoelectric navigation.
[0041] a current target frame determination module, configured to take an electronic image acquired by a target photoelectric navigation device at a current time as a current target frame;
[0042] a prediction vector determination module configured to determine a prediction vector according to an output vector of a historical moment; the prediction vector is an output vector of a previous moment or a linear relationship of output vectors of a plurality of continuous historical moments before the current moment;
[0043] a search region determination module configured to determine a search region between a current reference frame and the current target frame according to the prediction vector; the current reference frame is a previous target frame or an original reference frame; the previous target frame is an electronic image acquired by the target photoelectric navigation device at a previous moment; the original reference frame is determined according to an electronic image acquired by the target photoelectric navigation device at a historical moment; the search region is an overlapping region between the current reference frame and the current target frame;
[0044] a reference block determination module configured to determine a reference block in the search region corresponding to the current reference frame;
[0045] a target block determination module configured to determine a reference target block and a plurality of to-be-matched target blocks in the search region corresponding to the current target frame; the plurality of to-be-matched target blocks include a center target block and a plurality of neighborhood target blocks; the center target block is an overlapping region of the current target frame and the reference block; the neighborhood target block is a region obtained by moving around the center target block; the reference target block is a region of the current target frame with the same position and size as the reference block;
[0046] a first correlation operation module configured to perform correlation operations on each of the to-be-matched target blocks and the reference block respectively, to determine an optimal correlation value, and to determine the to-be-matched target block corresponding to the optimal correlation value as an optimal matching block; the optimal correlation value is a minimum value of a sum of absolute errors of pixel points or a maximum value of a number of similarities;
[0047] a current motion vector calculation module configured to calculate a current motion vector of the optimal matching block relative to the reference block;
[0048] a second correlation operation module configured to perform a correlation operation on the reference target block and the reference block to obtain a reference correlation value; when the optimal correlation value is the minimum value of the sum of absolute errors of pixel points, the reference correlation value is a sum of absolute errors of pixel points of the reference target block and the reference block; when the optimal correlation value is the maximum value of the number of similarities, the reference correlation value is a number of similarities of pixel points of the reference target block and the reference block;
[0049] a relative motion determination module configured to determine whether the target photoelectric navigation device has a relative motion at the current moment according to the optimal correlation value and the reference correlation value;
[0050] A displacement output module is configured to output the current motion vector if relative motion occurs, and output zero displacement if no relative motion occurs.
[0051] The application further provides an electronic device comprising a memory for storing a computer program and a processor for running the computer program to enable the electronic device to perform the method for judging zero displacement in photoelectric navigation.
[0052] The application further provides a computer readable storage medium storing a computer program, which, when executed by a processor, implements the method for judging zero displacement in photoelectric navigation.
[0053] According to the embodiments of the application, the following technical effects are achieved.
[0054] The method for judging zero displacement provided by the application adds a judgment process for whether relative motion occurs in a target photoelectric navigation device on the basis of a conventional motion estimation algorithm, i.e., determining a reference target block by determining a region on a current target frame having the same position and size as a reference block, performing correlation operations on the reference target block and each target block to be matched and the reference block respectively to determine a reference correlation value of the reference target block and optimal correlation values of each target block to be matched, and comparing the reference correlation value with the optimal correlation values to determine whether relative motion occurs in the target photoelectric navigation device relative to a background, and further correctly output zero displacement when no relative motion occurs in the photoelectric navigation device. BRIEF DESCRIPTION OF DRAWINGS
[0055] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description merely show some embodiments of the application, and for those skilled in the art, other drawings can be obtained based on these drawings without creative labor.
[0056] Figure 1 A flow chart of the method for judging zero displacement in photoelectric navigation provided by the application;
[0057] Figure 2 A specific flow chart of the first method for judging zero displacement in photoelectric navigation provided by the embodiments of the application;
[0058] Figure 3 A specific flow chart of the second method for judging zero displacement in photoelectric navigation provided by the embodiments of the application;
[0059] Figure 4 A schematic diagram of the method for determining a search region provided by the application;
[0060] Figure 5 The schematic diagram of the determination method of the reference block and the target block provided by the present application is shown in the figure.
[0061] Figure 6 The module diagram of the zero displacement judgment system in the photoelectric navigation provided by the present application is shown in the figure.
[0062] Symbol explanation: current target frame determination module-1, prediction vector determination module-2, search region determination module-3, reference block determination module-4, target block determination module-5, first correlation operation module-6, current motion vector calculation module-7, second correlation operation module-8, relative motion determination module-9, displacement output module-10. DETAILED DESCRIPTION
[0063] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.
[0064] The purpose of the present application is to provide a zero displacement judgment method, system, device and medium in photoelectric navigation, so as to correctly output zero displacement when the photoelectric navigation device does not have relative motion.
[0065] In order to make the above-mentioned purposes, characteristics and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.
[0066] Embodiment one
[0067] The present embodiment provides a zero displacement judgment method in photoelectric navigation, Figure 1 The flow chart of the zero displacement judgment method in photoelectric navigation provided by the present application is shown in the figure. Figure 1 As shown, the judgment method comprises:
[0068] Step S1: taking the electronic image acquired by the target photoelectric navigation device at the current time as the current target frame.
[0069] Step S2: determining the prediction vector according to the output vector at the historical time; the prediction vector is the output vector at the last time or the linear relationship of the output vectors at the continuous historical time before the current time.
[0070] Step S3: determining a search area between the current reference frame and the current target frame according to the prediction vector; the current reference frame is a previous target frame or an original reference frame; the previous target frame is an electronic image acquired by the target photoelectric navigation device at a previous time; the original reference frame is determined according to an electronic image acquired by the target photoelectric navigation device at a historical time; the search area is an area overlapping between the current reference frame and the current target frame.
[0071] In the embodiment, step S3 specifically comprises:
[0072] Step S31: determining an actual motion vector of the previous target frame relative to the current reference frame.
[0073] Step S32: determining a prediction motion vector of the current target frame relative to the current reference frame according to the actual motion vector and the prediction vector.
[0074] Step S33: determining a search area between the current reference frame and the current target frame according to the prediction motion vector.
[0075] Step S4: determining a reference block in the search area corresponding to the current reference frame.
[0076] Step S5: determining a reference target block and a plurality of to-be-matched target blocks in the search area corresponding to the current target frame; the plurality of to-be-matched target blocks comprise a center target block and a plurality of neighborhood target blocks; the center target block is an area on the current target frame overlapping with the reference block; the neighborhood target blocks are areas obtained by moving the center target block to each neighborhood (such as 8 neighborhoods or 24 neighborhoods); the reference target block is an area on the current target frame having the same position and size as the reference block.
[0077] Step S6: performing correlation operation on each to-be-matched target block and the reference block respectively, determining an optimal correlation value, and taking the to-be-matched target block corresponding to the optimal correlation value as an optimal matching block; the optimal correlation value is a minimum value of absolute error sum or a maximum value of similarity quantity of pixel points of the to-be-matched target block and the reference block. Preferably, the correlation operation is performed by using an absolute error sum algorithm (SAD).
[0078] Step S7: calculating a current motion vector of the optimal matching block relative to the reference block.
[0079] Step S8: correlation operation is performed on the reference target block and the reference block to obtain a reference correlation value; when the optimal correlation value is the minimum value of the sum of absolute errors, the reference correlation value is the sum of absolute errors of the pixel points of the reference target block and the reference block; when the optimal correlation value is the maximum value of the similar quantity, the reference correlation value is the similar quantity of the pixel points of the reference target block and the reference block.
[0080] Step S9: according to the optimal correlation value and the reference correlation value, it is determined whether the relative motion of the target photoelectric navigation device occurs at the current time.
[0081] Step S10: if the relative motion occurs (at the current time), the current motion vector is output, i.e. the output vector (dx, dy); if the relative motion does not occur (at the current time), the zero displacement is output, i.e. the output vector (0, 0).
[0082] Further, before step S3, it further includes:
[0083] Step S11: it is judged whether the size of the search area between the previous reference frame and the current target frame is within the set size range.
[0084] Step S12: if yes, the previous target frame is taken as the current reference frame; if no, the original reference frame (i.e. the previous reference frame) is taken as the current reference frame.
[0085] Figure 2 The first photoelectric navigation zero displacement judgment method provided by the embodiment of the application has the advantages that: Figure 3 The second photoelectric navigation zero displacement judgment method provided by the embodiment of the application has the advantages that: Figure 2 And Figure 3 As shown in the actual application, the specific process of the application is as follows:
[0086] (1) a reference frame (i.e. the current reference frame) and a target frame (i.e. the current target frame) after it are obtained, image matching is performed, and the moving direction and distance of the target photoelectric navigation device (such as a mouse cursor) are obtained. The target frame needs to be obtained and updated each time, and the reference frame does not need to be updated each time. The image matching is that a reference image and a target image are prepared, the reference image and the target image are compared, and the moving direction and distance (i.e. the motion vector) of the target image relative to the reference image are determined. The photoelectric navigation device is that the target plane is irradiated by an LED lighting system, the reflected light of the target plane enters the device sensor to form an electronic image, and the device processes the electronic image (such as image matching) to obtain the motion vector. This kind of device is called a photoelectric navigation device.
[0087] (2) According to the output vector of the historical moment, a predicted vector (preDx, preDy) is determined. This predicted vector can be regarded as the predicted motion vector of the current target frame relative to the last target frame. Therefore, according to the position of the last target frame and the predicted vector, the position of the current target frame can be predicted. According to the actual motion vector of the last target frame and the current reference frame and the predicted vector, the predicted motion vector of the current target frame and the current reference frame can be obtained. See Figure 4 The overlapping area of the current reference frame and the current target frame is determined as the search area (i.e. the gray area in Figure 4 ). This area makes the current reference frame and the current target frame have the strongest correlation. The reference block and the target block are determined in the search area, and the correlation operation is performed to correct the predicted vector, so that the correct motion vector (dx, dy) of the current target frame relative to the last target frame can be obtained. The accumulation of the correct motion vector (dx, dy) obtained each time is the actual motion vector of the target frame and the reference frame. The predicted vector can be the last output correct motion vector (dx, dy), or a linear relationship of the previous motion vectors (dx, dy).
[0088] (3) The reference block and the target block (herein referred to as the target block to be matched) are determined, and the correlation operation is performed between the reference block and multiple specified target blocks (i.e. the target blocks to be matched) in the neighborhood (such as a 3x3 neighborhood or a 5x5 neighborhood). Taking the 3x3 neighborhood as an example, the correlation results can be recorded as S1-S9. By comparing S1-S9, the optimal value Sopt (i.e. the optimal correlation value) is found. The relative moving position of the target block corresponding to the optimal value (i.e. the optimal matching block) and the reference block is the motion vector value (dx, dy) of this time displacement. Among them, the reference block in the search area is fixed, and the target block is a block overlapping with the reference block in the search area on the target frame and multiple blocks in its neighborhood, such as eight blocks if the eight-neighborhood is selected, which are S1-S9. The selection of the reference block in the search area should be as large as possible, and the nine target blocks to be matched with it as the reference should not exceed the target frame. The determination method of the reference block and the target block is described in Figure 5 , wherein, Figure 5 The dark gray area in
[0089] Specifically, the optimal value is determined by a correlation operation algorithm, such as the SAD algorithm, in which two matching blocks (a reference block and a target block) are subtracted point by point, and then the absolute values are taken and summed. The smallest value among S1-S9 is taken as the optimal value. If the target block is identical to the reference block, the correlation is optimal, and the SAD value is 0. In addition, other algorithms can also be used, such as counting the number of similar points between the two matching blocks. The correlation is optimal when the number is the largest. The "point" referred to here refers to a pixel point of an electronic image.
[0090] (4) The reference block is unchanged, and a target block (i.e., a reference target block) of the same size is found at the same position in the target frame according to the position and size of the reference block, and a correlation operation is performed, and the result is recorded as S0 (i.e., a reference correlation value). Since the reference block is unchanged, and the target block corresponding to S0 has the same size as the target block corresponding to Sopt, Sopt and S0 are comparable.
[0091] (5) S0 and Sopt are compared. If S0 is more optimal than Sopt (i.e., the first implementation method), the photoelectric navigation device should not have a relative displacement, and then a zero displacement (0, 0) is output. If Sopt is more optimal, then (dx, dy) is output. Taking the SAD algorithm for example, the smaller value is more optimal.
[0092] In addition, different implementation methods can also be used, which are not limited to the present process and do not affect the idea of the present application. For example, after it is determined that Sopt is not more optimal than S0, a zero displacement is output with a lag, and the changes in the subsequent situation are observed. If Sopt is not more optimal than S0 in a continuous frame (i.e., the second implementation method) or in most frames in a certain interval (i.e., the third implementation method), a zero displacement is output.
[0093] The three implementation methods are described in detail as follows:
[0094] The first implementation method is shown in FIG. 1. Figure 2 The optimal correlation value and the reference correlation value are compared. When the optimal correlation value is the minimum sum of absolute errors, if the optimal correlation value is greater than or equal to the reference correlation value, it is determined that the target photoelectric navigation device has not moved relatively at the current time; otherwise, it is determined that the target photoelectric navigation device has moved relatively at the current time. When the optimal correlation value is the maximum number of similar points, if the optimal correlation value is less than or equal to the reference correlation value, it is determined that the target photoelectric navigation device has not moved relatively at the current time; otherwise, it is determined that the target photoelectric navigation device has moved relatively at the current time.
[0095] The second implementation method is shown in FIG. 2. Figure 3comparing the optimal correlation value with the reference correlation value; when the optimal correlation value is the minimum value of the sum of absolute errors, if the optimal correlation value at all times within a set time interval is greater than or equal to the reference correlation value, it is determined that the target photoelectric navigation device does not occur relative motion at the current time; otherwise, it is determined that the target photoelectric navigation device occurs relative motion at the current time; when the optimal correlation value is the maximum value of the similar quantity, if the optimal correlation value at all times within a set time interval is less than or equal to the reference correlation value, it is determined that the target photoelectric navigation device does not occur relative motion at the current time; otherwise, it is determined that the target photoelectric navigation device occurs relative motion at the current time; the starting time of the set time interval is the next time of the time when the target photoelectric navigation device occurs relative motion before the current time; the current time is a certain time within the set time interval. Specifically, the set time interval includes Th0 consecutive times, and a counter Cnt is used to count the number of times when Sopt is not better than S0.
[0096] The third implementation method is specifically: comparing the optimal correlation value with the reference correlation value; when the optimal correlation value is the minimum value of the sum of absolute errors, if the number of times when the optimal correlation value within a set time interval is greater than or equal to the reference correlation value is greater than a set value, it is determined that the target photoelectric navigation device does not occur relative motion at the current time; otherwise, it is determined that the target photoelectric navigation device occurs relative motion at the current time; when the optimal correlation value is the maximum value of the similar quantity, if the number of times when the optimal correlation value within a set time interval is less than or equal to the reference correlation value is greater than a set value, it is determined that the target photoelectric navigation device does not occur relative motion at the current time; otherwise, it is determined that the target photoelectric navigation device occurs relative motion at the current time; the starting time of the set time interval is the next time of the time when the target photoelectric navigation device occurs relative motion before the current time; the current time is a certain time within the set time interval.
[0097] (6) When performing the next reference frame and target frame image matching, it is necessary to consider whether the reference frame needs to be replaced. When the search area is too small, the correlation between the reference frame and the target frame is not strong, which may cause errors in matching and prediction, and then the current target frame needs to be used as the reference frame for the next matching.
[0098] Embodiment two
[0099] In order to perform the method corresponding to the above-mentioned embodiment one, to realize the corresponding function and technical effect, the following provides a photoelectric navigation zero displacement judgment system, Figure 6 The module diagram of the photoelectric navigation zero displacement judgment system provided by the application is shown in the figure. Figure 6 As shown in the figure, the judgment system comprises:
[0100] A current target frame determining module 1 is configured to obtain an electronic image acquired by the target photoelectric navigation device at a current time as a current target frame.
[0101] A prediction vector determining module 2 is configured to determine a prediction vector according to an output vector at a historical time; the prediction vector is an output vector at a previous time or a linear relationship of output vectors at a plurality of continuous historical times before the current time.
[0102] A search region determining module 3 is configured to determine a search region between a current reference frame and the current target frame according to the prediction vector; the current reference frame is a previous target frame or an original reference frame; the previous target frame is an electronic image acquired by the target photoelectric navigation device at a previous time; the original reference frame is determined according to an electronic image acquired by the target photoelectric navigation device at a historical time; and the search region is an overlapping region between the current reference frame and the current target frame.
[0103] A reference block determining module 4 is configured to determine a reference block in the search region corresponding to the current reference frame.
[0104] A target block determining module 5 is configured to determine a reference target block and a plurality of to-be-matched target blocks in the search region corresponding to the current target frame; the plurality of to-be-matched target blocks include a center target block and a plurality of neighborhood target blocks; the center target block is an overlapping region of the current target frame and the reference block; the neighborhood target block is a region obtained by moving around the center target block; and the reference target block is a region of the current target frame with the same position and size as the reference block.
[0105] A first correlation operation module 6 is configured to perform correlation operation on each of the to-be-matched target blocks and the reference block respectively, to determine an optimal correlation value, and to determine the to-be-matched target block corresponding to the optimal correlation value as an optimal matching block; the optimal correlation value is a minimum value of the sum of absolute errors of pixel points or a maximum value of the number of similarities.
[0106] A current motion vector calculating module 7 is configured to calculate a current motion vector of the optimal matching block relative to the reference block.
[0107] A second correlation operation module 8 is configured to perform correlation operation on the reference target block and the reference block to obtain a reference correlation value; when the optimal correlation value is the minimum value of the sum of absolute errors, the reference correlation value is the sum of absolute errors of pixel points of the reference target block and the reference block; and when the optimal correlation value is the maximum value of the number of similarities, the reference correlation value is the number of similarities of pixel points of the reference target block and the reference block.
[0108] The relative motion determination module 9 is configured to determine whether the target photoelectric navigation device has relative motion at the current time according to the optimal correlation value and the reference correlation value.
[0109] The displacement output module 10 is configured to output the current motion vector if the relative motion occurs, and output zero displacement if the relative motion does not occur.
[0110] Embodiment Three
[0111] The embodiment of the present application provides an electronic device, including a memory and a processor, the memory is used for storing a computer program, the processor runs the computer program to make the electronic device execute the judgment method of zero displacement in photoelectric navigation in the embodiment one.
[0112] Optionally, the electronic device can be a server.
[0113] In addition, the embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the judgment method of zero displacement in photoelectric navigation in the embodiment one.
[0114] The present application adds a zero displacement judgment process on the basis of the general flow (motion estimation algorithm), that is, the correlation matching result S0 is calculated through the target block which has the same position and size as the reference block of the general flow, and S0 is compared with the Sopt value in the general flow to determine whether the device has relative displacement relative to the background. Compared with the prior art, the present application has the following advantages:
[0115] (1) No threshold setting, all captured images under all backgrounds have consistency and are treated equally.
[0116] (2) For the device without relative motion, the S0 value is theoretically equal to 0, and in practical application, it is close to zero, which indicates that the method provided by the present application is accurate and the effect is obvious.
[0117] In the specification, each embodiment is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the related parts can be referred to the method part.
[0118] The principles and implementations of the present application are described in the specific examples in this article, and the above examples are only used to help understand the core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation and application range will be changed. Therefore, the content of the specification should not be understood as a limitation of the present application.
Claims
1. A method for determining zero displacement in photoelectric navigation, characterized in that, The judging method comprises: taking an electronic image acquired by a target photoelectric navigation device at a current time as a current target frame; determining a prediction vector according to an output vector at a historical time; the prediction vector is a linear relationship of an output vector at a previous time or output vectors at a plurality of continuous historical times before the current time; determining a search area between a current reference frame and the current target frame according to the prediction vector; the current reference frame is a previous target frame or an original reference frame; the previous target frame is an electronic image acquired by the target photoelectric navigation device at a previous time; the original reference frame is determined according to an electronic image acquired by the target photoelectric navigation device at a historical time; the search area is an area overlapping between the current reference frame and the current target frame; determining a reference block in a search area corresponding to the current reference frame; determining a reference target block and a plurality of to-be-matched target blocks in a search area corresponding to the current target frame; the plurality of to-be-matched target blocks comprise a center target block and a plurality of neighborhood target blocks; the center target block is an area overlapping with the reference block on the current target frame; the neighborhood target block is an area obtained by moving around the center target block; the reference target block is an area with the same position and size as the reference block on the current target frame; respectively performing correlation operation on each of the to-be-matched target blocks and the reference block to determine an optimal correlation value, and taking the to-be-matched target block corresponding to the optimal correlation value as an optimal matching block; the optimal correlation value is a minimum value of a sum of absolute errors of pixel points or a maximum value of a similar quantity; calculating a current motion vector of the optimal matching block relative to the reference block; performing correlation operation on the reference target block and the reference block to obtain a reference correlation value; when the optimal correlation value is the minimum value of the sum of absolute errors of pixel points, the reference correlation value is a sum of absolute errors of pixel points of the reference target block and the reference block; when the optimal correlation value is the maximum value of the similar quantity, the reference correlation value is a similar quantity of pixel points of the reference target block and the reference block; determining whether the target photoelectric navigation device has relative motion at the current time according to the optimal correlation value and the reference correlation value, specifically comprising: comparing the optimal correlation value with the reference correlation value; when the optimal correlation value is the minimum value of the sum of absolute errors of pixel points, if the optimal correlation value is greater than or equal to the reference correlation value, it is determined that the target photoelectric navigation device has no relative motion at the current time; otherwise, it is determined that the target photoelectric navigation device has relative motion at the current time; when the optimal correlation value is the maximum value of the similar quantity, if the optimal correlation value is less than or equal to the reference correlation value, it is determined that the target photoelectric navigation device has no relative motion at the current time; otherwise, it is determined that the target photoelectric navigation device has relative motion at the current time; if the relative motion occurs, outputting the current motion vector; if the relative motion does not occur, outputting zero displacement.
2. The method of claim 1, wherein the method further comprises: The method comprises the following steps: comparing the optimal correlation value with the reference correlation value; when the optimal correlation value is the minimum value of the absolute error sum, if the optimal correlation value is greater than or equal to the reference correlation value at all time points within a set time interval, it is determined that the target photoelectric navigation device does not occur relative motion at the current time point; otherwise, it is determined that the target photoelectric navigation device occurs relative motion at the current time point; when the optimal correlation value is the maximum value of the similar quantity, if the optimal correlation value is less than or equal to the reference correlation value at all time points within a set time interval, it is determined that the target photoelectric navigation device does not occur relative motion at the current time point; otherwise, it is determined that the target photoelectric navigation device occurs relative motion at the current time point; the starting time point of the set time interval is the next time point of the time point when the target photoelectric navigation device occurs relative motion before the current time point.
3. The method of claim 1, wherein the method further comprises: determining a zero displacement of the vehicle in the navigation mode based on the first and second signals. The method comprises the following steps: comparing the optimal correlation value with the reference correlation value; when the optimal correlation value is the minimum value of the absolute error sum, if the optimal correlation value is greater than or equal to the reference correlation value at all time points within a set time interval, it is determined that the target photoelectric navigation device does not occur relative motion at the current time point; otherwise, it is determined that the target photoelectric navigation device occurs relative motion at the current time point; when the optimal correlation value is the maximum value of the similar quantity, if the optimal correlation value is less than or equal to the reference correlation value at all time points within a set time interval, it is determined that the target photoelectric navigation device does not occur relative motion at the current time point; otherwise, it is determined that the target photoelectric navigation device occurs relative motion at the current time point; the starting time point of the set time interval is the next time point of the time point when the target photoelectric navigation device occurs relative motion before the current time point.
4. The method of claim 1, wherein the method further comprises: The method comprises the following steps: determining the actual motion vector of the previous target frame relative to the current reference frame; determining the predicted motion vector of the current target frame relative to the current reference frame according to the actual motion vector and the prediction vector; determining the search area between the current reference frame and the current target frame according to the predicted motion vector.
5. The method of claim 1, wherein the method further comprises: determining a zero displacement of the vehicle in the navigation mode based on the first and second signals. Before the step of determining the search area between the current reference frame and the current target frame according to the prediction vector, the method further comprises the following steps: judging whether the size of the search area between the previous reference frame and the current target frame is within a set size range; if yes, taking the previous target frame as the current reference frame; if no, taking the original reference frame as the current reference frame.
6. The method of claim 1, wherein the method further comprises: The correlation operation is performed by using the SAD algorithm.
7. A system for determining zero displacement in photoelectric guidance, characterized in that The judgment system comprises: a current target frame determination module, configured to take the electronic image acquired by the target photoelectric navigation device at the current time point as the current target frame; a prediction vector determination module configured to determine a prediction vector according to an output vector of a historical time; the prediction vector is an output vector of a previous time or a linear relationship of output vectors of a plurality of continuous historical times before a current time; a search region determination module configured to determine a search region between a current reference frame and the current target frame according to the prediction vector; the current reference frame is a previous target frame or an original reference frame; the previous target frame is an electronic image obtained by the target photoelectric navigation device at a previous time; the original reference frame is determined according to an electronic image obtained by the target photoelectric navigation device at a historical time; and the search region is an overlapping region between the current reference frame and the current target frame; a reference block determination module configured to determine a reference block in the search region corresponding to the current reference frame; a target block determination module configured to determine a reference target block and a plurality of to-be-matched target blocks in the search region corresponding to the current target frame; the plurality of to-be-matched target blocks include a center target block and a plurality of neighborhood target blocks; the center target block is an overlapping region of the current target frame and the reference block; the neighborhood target block is a region obtained by moving around the center target block; and the reference target block is a region of the current target frame with the same position and size as the reference block; a first correlation operation module configured to perform correlation operations on each of the to-be-matched target blocks and the reference block respectively, to determine an optimal correlation value, and to take the to-be-matched target block corresponding to the optimal correlation value as an optimal matching block; the optimal correlation value is a minimum value of a sum of absolute errors of pixel points or a maximum value of a number of similarities; a current motion vector calculation module configured to calculate a current motion vector of the optimal matching block relative to the reference block; a second correlation operation module configured to perform a correlation operation on the reference target block and the reference block to obtain a reference correlation value; when the optimal correlation value is the minimum value of the sum of absolute errors of pixel points, the reference correlation value is a sum of absolute errors of pixel points of the reference target block and the reference block; and when the optimal correlation value is the maximum value of the number of similarities, the reference correlation value is a number of similarities of pixel points of the reference target block and the reference block; a relative motion determination module configured to determine whether the target photoelectric navigation device has a relative motion at the current time according to the optimal correlation value and the reference correlation value, specifically including: comparing the optimal correlation value with the reference correlation value; when the optimal correlation value is the minimum value of the sum of absolute errors of pixel points, if the optimal correlation value is greater than or equal to the reference correlation value, it is determined that the target photoelectric navigation device has no relative motion at the current time; otherwise, it is determined that the target photoelectric navigation device has a relative motion at the current time; when the optimal correlation value is the maximum value of the number of similarities, if the optimal correlation value is less than or equal to the reference correlation value, it is determined that the target photoelectric navigation device has no relative motion at the current time; otherwise, it is determined that the target photoelectric navigation device has a relative motion at the current time. A displacement output module is configured to output the current motion vector if relative motion occurs, and output zero displacement if no relative motion occurs.
8. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory is configured to store a computer program, and the processor is configured to run the computer program to enable the electronic device to perform the method for judging zero displacement in photoelectric navigation according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer program is stored in the memory and is executed by the processor to implement the method for judging zero displacement in photoelectric navigation according to any one of claims 1 to 6.
Citation Information
Patent Citations
Optical movement sensing method
CN101169693A
Photoelectric navigation motion vector prediction method
CN102314694A