Track correction method of photoelectric navigation equipment, equipment, medium and product
By calculating the peak correlation values of the whole pixel and sub-pixel of the photoelectric navigation device, judging the continuous inter-frame displacement and suppressing noise interference, the problem of difficulty in determining the displacement and noise influence of the photoelectric navigation device during tiny movements is solved, and positioning accuracy and stability are improved.
Patent Information
- Application Number
- CN202411552330.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-01
- Publication Date
- 2025-01-14
AI Technical Summary
It is difficult for the photoelectric navigation device to determine displacement during slight movements, and the high-precision sub-pixel matching results are susceptible to noise, resulting in reduced positioning accuracy and jitter of the mouse pointer.
By obtaining the reference frame and the target frame, calculate the correlation peak between the entire pixel and the sub-pixel, determine whether the displacement between consecutive frames is 0, and determine whether there is noise interference based on the noise threshold to ensure the accuracy of the output displacement result.
It improves the suppression of noise interference, reduces the probability of jitter in photoelectric navigation equipment, and enhances the accuracy of output.
Smart Images
Figure CN119311136A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing, and in particular to a trajectory correction method, device, medium and product of an optoelectronic navigation device. Background Art
[0002] With the widespread application of photoelectric navigation, its performance requirements are getting higher and higher. The minimum movement that can be obtained by conventional methods is the distance between two adjacent pixels of the image device. Therefore, when the tiny movement is less than the distance of one pixel width, the tiny movement cannot be determined. The output speed is increased by multiplying the calculated displacement by a coefficient, but there will be data loss when the integer pixel displacement is multiplied. For example, if it is magnified twice, the minimum displacement of the movement is 2, and if it is magnified four times, the minimum displacement of the movement is 4. Taking the photoelectric mouse as an example, the moving trajectory is obviously jagged, many details are lost, and the positioning accuracy is reduced. At present, in order to improve the positioning accuracy of photoelectric navigation, sub-pixel matching is performed on the integer pixels to obtain the displacement at the sub-pixel level. The higher the sub-pixel accuracy, the finer the trajectory. For example, 15 pixels are inserted between any two integer pixels, and the distance between adjacent pixels is equal. Even when the user requires the resolution to be magnified 16 times, the minimum detected displacement of the photoelectric navigation device is 1.
[0003] However, the high-precision sub-pixel matching results are affected by noise. Taking the optical mouse as an example, since the image frames captured by the image sensor may contain noise, when the mouse is not in use, there is theoretically no displacement between the continuously acquired image frames, but due to the influence of noise, the values of the pixels at the corresponding positions between the two frames will not be exactly the same. Although the displacement will not be significantly affected by the position offset caused by noise when it is relatively larger than the position offset, the trajectory may be disturbed when the displacement is very small or does not move and is basically the same as the position offset or only slightly larger than the position offset. The higher the positioning accuracy, the greater the impact of noise. Therefore, when the optical mouse moves a small trajectory, it is easily affected by noise, causing the mouse pointer to jitter. Summary of the invention
[0004] The purpose of this application is to provide a trajectory correction method, device, medium and product for an optoelectronic navigation device, which can obtain accurate correlation calculation results between two frames, improve the suppression of noise interference, reduce the probability of jitter in the optoelectronic navigation device, and improve the accuracy of the output of the optoelectronic navigation device.
[0005] To achieve the above objectives, this application provides the following solutions:
[0006] In a first aspect, the present application provides a trajectory correction method for an optoelectronic navigation device, the method comprising: obtaining a pair of reference frames and a target frame; the target frame is a trajectory image collected by the optoelectronic navigation device at the current moment; the reference frame is a trajectory image collected by the optoelectronic navigation device at the previous moment of the target frame; selecting a reference block in the reference frame according to the trajectory change from the reference frame to the target frame; the reference block is used to frame the range of the trajectory change; according to the reference block, the correlation peak value of the integer pixel and the correlation peak value of the sub-pixel are calculated using the same matching method; the resolution of the sub-pixel is higher than the resolution of the integer pixel; the integer pixel displacement and the sub-pixel displacement are calculated based on the correlation peak value of the integer pixel and the correlation peak value of the sub-pixel respectively; judging whether the integer pixel displacement is 0, if not, outputting the integer pixel displacement and the sub-pixel displacement; if so, returning to "obtaining a pair of reference frames and target frames" until the integer pixel displacements of N consecutive pairs of reference frames and target frames are all 0, entering the next step; judging whether there are integer pixel and sub-pixel correlation matching peaks in the N pairs of reference frames and target frames The difference is less than the peak threshold, if not, outputting the integer pixel displacement and sub-pixel displacement, if so, not outputting the integer pixel displacement and sub-pixel displacement.
[0007] In a second aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the trajectory correction method of any one of the above-mentioned optoelectronic navigation devices.
[0008] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the trajectory correction method of any one of the above-mentioned optoelectronic navigation devices.
[0009] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the trajectory correction method of any one of the above-mentioned optoelectronic navigation devices.
[0010] According to the specific embodiments provided in this application, this application discloses the following technical effects:
[0011] The present application obtains a pair of reference frames and a target frame; selects a reference block in the reference frame according to the trajectory change from the reference frame to the target frame; calculates the correlation peak of the integer pixel and the correlation peak of the sub-pixel according to the reference block using the same matching method; calculates the integer pixel displacement and the sub-pixel displacement respectively based on the correlation peak of the integer pixel and the correlation peak of the sub-pixel; then, by judging whether there are N consecutive pairs of reference frames and target frames where the integer pixel displacements are all 0, the integer pixel displacement and the sub-pixel displacement are output as the real displacement; and further judges the situation where the integer pixel displacements of N consecutive pairs of reference frames and target frames are all 0, that is, judges whether there are N pairs of reference frames and target frames where the difference between the correlation matching peaks of the integer pixel and the sub-pixel is less than the peak threshold. If so, it indicates that it is noise interference, and the displacement is not output. Otherwise, the integer pixel displacement and the sub-pixel displacement are output as the real displacement. The present application uses the same matching method to perform correlation analysis on the target frame and the reference frame, obtains an accurate correlation calculation result between the two frames, and determines whether there is noise interference through two judgments to avoid misjudgment caused by noise, and finally determines the final displacement output result, thereby completing the correction of the trajectory, improving the suppression of noise interference, reducing the probability of jitter in the optoelectronic navigation device, and improving the accuracy of the output of the optoelectronic navigation device. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0013] Figure 1 A schematic flow chart of a trajectory correction method for an optoelectronic navigation device provided in an embodiment of the present application.
[0014] Figure 2 This is a comparison diagram of whole pixel trajectories at different resolutions provided in an embodiment of the present application.
[0015] Figure 3 A schematic diagram of a target frame and a reference frame provided in an embodiment of the present application.
[0016] Figure 4 A schematic diagram of an upper left shift operation of a target frame relative to a reference frame provided in an embodiment of the present application.
[0017] Figure 5 A schematic diagram of an upward shift operation of a target frame relative to a reference frame provided in an embodiment of the present application.
[0018] Figure 6A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0019] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0020] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.
[0021] Terminology explanation:
[0022] Integer pixel matching, the image sensor continuously collects image frames, the current frame is used as the target frame, and the reference frame is the previously output image frame. By changing the relative position of the target frame with respect to the reference frame, the optimal match (correlation peak) is determined in the search area and the displacement is calculated. Figure 2 Integer pixel traces at different resolutions are shown.
[0023] Embodiment 1, as Figure 1 As shown, this embodiment provides a trajectory correction method for an optoelectronic navigation device, the method comprising:
[0024] S1. Figure 3 As shown, a pair of reference frame (Ref1) and target frame (Tar1) are obtained; the target frame is the trajectory image collected by the optoelectronic navigation device at the current moment; the reference frame is the trajectory image collected by the optoelectronic navigation device at the previous moment of the target frame. Here, a pair of reference frame (Ref1) and target frame (Tar1) includes a reference frame and a target frame, which are in the relationship of the current frame and the previous frame and are selected in pairs.
[0025] In actual application, image frames are continuously captured by the image sensor of the optoelectronic navigation device, the current image frame is used as the target frame, and the image frame captured before the target frame is used as the reference frame.
[0026] S2. Select a reference block in the reference frame according to the trajectory change from the reference frame to the target frame; the reference block is used to frame the range of the trajectory change.
[0027] Furthermore, the reference block is a rectangular image constructed according to a range of trajectory changes of the reference frame.
[0028] In actual application, the movement speed of the mouse (photoelectric navigation device) gradually increases or decreases, so the previous movement displacement and the current movement displacement to be calculated are closely related. The previous movement displacement is used as the predicted displacement, and the reference block and target block in the current reference frame and target frame are determined according to the predicted displacement.
[0029] S3. According to the reference block, the correlation peak value of the integer pixel and the correlation peak value of the sub-pixel are calculated using the same matching method; the resolution of the sub-pixel is higher than the resolution of the integer pixel.
[0030] Among them, the whole pixel can calculate the distance that two consecutive frames of images move greater than or equal to one pixel (30μm, this parameter depends on the hardware pixel size), and the sub-pixel (sub-pixel) can calculate the distance that two consecutive frames of images move less than one pixel.
[0031] Further step S3 specifically includes:
[0032] S31. Figure 4-Figure 5 As shown, according to the reference block, multiple target blocks are selected on the target frame in a translation manner; the number of the target blocks is determined according to the specifications of the search window selected by the correlation search; the range of each target block is the same as the range of the reference block.
[0033] Furthermore, the size of the search window is 3×3.
[0034] Step S31 specifically includes:
[0035] S311. Perform a translation operation on the target frame relative to the reference frame; the translation operation specifically includes: move up, move down, move left, move right, move to the upper left, move to the lower left, move to the upper right, move to the lower right and no translation.
[0036] S312. After each operation, select the part of the target frame that overlaps with the reference block as the target block to obtain multiple target blocks.
[0037] Optionally, each operation is to select one of the following methods for translation without repetition: up, down, left, right, upper left, lower left, upper right, lower right, and no translation.
[0038] In actual application, the search area can be determined according to the prediction vector, the reference block and 9 target blocks can be determined in the search area, and the 3×3 correlation search window can be calculated by changing the position of the target frame relative to the reference frame.
[0039] S32. Perform integer pixel correlation calculation and sub-pixel correlation calculation on the reference block and each target block respectively to obtain multiple integer pixel correlation values and multiple sub-pixel correlation values.
[0040] Step S32 specifically includes:
[0041] S321. Multiply the pixel values corresponding to the integer pixels of the target block and the reference block obtained after each translation and then sum them up to obtain multiple integer pixel correlation values.
[0042] S322. Multiply the pixel values corresponding to the sub-pixels of the target block and the reference block obtained after each translation and then sum them up to obtain multiple sub-pixel correlation values.
[0043] S33. Select the correlation peak value of the integer pixel among the integer pixel correlation values, and select the correlation peak value of the sub-pixel among the sub-pixel correlation values.
[0044] In actual application, Figure 3 As shown, Tar1 is the target frame, Ref1 is the reference frame, Figure 4 The middle gray area is the reference block in the reference frame. To calculate the correlation value 8 in the 3×3 correlation search window, the target frame Tar1 is moved one pixel to the left and upward relative to the reference frame Ref1. The target block is the area in the target frame that overlaps with the reference block. The corresponding pixel data of the target block and the reference block are multiplied and summed to obtain the integer pixel correlation value and the sub-pixel correlation value. Figure 5 In order to calculate the correlation value 7 in the 3×3 correlation search window, the target frame Tar1 is moved up by one pixel relative to the reference frame Ref1. The target block is the area in the target frame that overlaps with the reference block. The corresponding pixel data of the target block and the reference block are multiplied and summed to obtain the integer pixel correlation value and the sub-pixel correlation value.
[0045] Similarly, the integer pixel displacement and sub-pixel displacement are obtained through the correlation operation between the 9 target blocks and the reference block.
[0046] S4. Calculate the integer pixel displacement and the sub-pixel displacement based on the correlation peak value of the integer pixel and the correlation peak value of the sub-pixel respectively.
[0047] S5. Determine whether the integer pixel displacement is 0. If not, output the integer pixel displacement and sub-pixel displacement; if so, return to "obtain a pair of reference frames and target frames" until the integer pixel displacements of N consecutive pairs of reference frames and target frames are all 0, and then proceed to the next step.
[0048] Furthermore, the value of N is 5.
[0049] S6. Determine whether there are N pairs of reference frames and target frames in which the difference between the integer pixel and sub-pixel correlation matching peaks is less than the peak threshold; if not, output the integer pixel displacement and sub-pixel displacement; if so, do not output the integer pixel displacement and sub-pixel displacement.
[0050] In actual application, if it is detected that there is no displacement of integer pixels for 5 consecutive matches, it is detected whether to output sub-pixel displacement. Generally, the images collected when the mouse (photoelectric navigation device) is stationary are matched, and the correlation value of the middle target block in the 3×3 correlation search window is the integer pixel peak value. The most relevant target block obtained by sub-pixel matching should also be the middle target block. However, due to the influence of noise, the correlation value of the middle target block is not the sub-pixel peak value. Therefore, it is determined whether the difference between the sub-pixel peak value and the integer pixel peak value is within the variation range of noise, and the peak threshold value th is determined according to the current image noise. If the integer pixel peak value-sub-pixel peak value <peak threshold value th, the sub-pixel displacement is not output. If it is greater than the peak threshold value th, the integer pixel displacement and sub-pixel displacement are output. If the integer pixel is detected to have displacement, it is not determined whether to output the sub-pixel displacement, and the integer pixel displacement and sub-pixel displacement are directly output. In this embodiment, the method for determining the peak threshold value th according to the current image noise is to calculate the maximum value of the change in the value of the corresponding pixel points between multiple image frames collected when the mouse is stationary, and use it as the threshold.
[0051] The technical effects of this application are as follows:
[0052] The present application obtains a pair of reference frames and a target frame; selects a reference block in the reference frame according to the trajectory change from the reference frame to the target frame; calculates the correlation peak of the integer pixel and the correlation peak of the sub-pixel according to the reference block using the same matching method; calculates the integer pixel displacement and the sub-pixel displacement respectively based on the correlation peak of the integer pixel and the correlation peak of the sub-pixel; then, by judging whether there are N consecutive pairs of reference frames and target frames where the integer pixel displacements are all 0, the integer pixel displacement and the sub-pixel displacement are output as the real displacement; and further judges the situation where the integer pixel displacements of N consecutive pairs of reference frames and target frames are all 0, that is, judges whether there are N pairs of reference frames and target frames where the difference between the correlation matching peaks of the integer pixel and the sub-pixel is less than the peak threshold. If so, it indicates that it is noise interference, and the displacement is not output. Otherwise, the integer pixel displacement and the sub-pixel displacement are output as the real displacement. The present application uses the same matching method to perform correlation analysis on the target frame and the reference frame, obtains an accurate correlation calculation result between the two frames, and determines whether there is noise interference through two judgments to avoid misjudgment caused by noise, and finally determines the final displacement output result, thereby completing the correction of the trajectory, improving the suppression of noise interference, reducing the probability of jitter in the optoelectronic navigation device, and improving the accuracy of the output of the optoelectronic navigation device.
[0053] Embodiment 2, the present application also provides a computer device, which may be a server or a terminal, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video tag processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above methods are implemented.
[0054] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0055] Embodiment 3: The present application also provides a computer-readable storage medium storing a computer program, which implements the above methods when executed by a processor.
[0056] Embodiment 4: The present application also provides a computer program product, including a computer program, which implements the above methods when executed by a processor.
[0057] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0058] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0059] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.
Claims
1. A trajectory correction method for an optoelectronic navigation device, characterized in that: The trajectory correction method of the optoelectronic navigation device comprises: Acquire a pair of reference frames and target frames; the target frame is a trajectory image collected by the optoelectronic navigation device at the current moment; the reference frame is a trajectory image collected by the optoelectronic navigation device at the previous moment of the target frame; According to the trajectory change from the reference frame to the target frame, a reference block is selected in the reference frame; the reference block is used to frame the range of the trajectory change; According to the reference block, a correlation peak value of an integer pixel and a correlation peak value of a sub-pixel are calculated by using the same matching method; the resolution of the sub-pixel is higher than the resolution of the integer pixel; The integer pixel displacement and the sub-pixel displacement are calculated based on the correlation peak value of the integer pixel and the correlation peak value of the sub-pixel respectively; Determine whether the integer pixel displacement is 0. If not, output the integer pixel displacement and sub-pixel displacement. If yes, return to "Get a pair of reference frames and target frames" until the integer pixel displacements of N consecutive pairs of reference frames and target frames are all 0, and then proceed to the next step. Determine whether there is a difference between the integer pixel and sub-pixel correlation matching peak value in N pairs of reference frames and target frames that is less than the peak threshold. If not, output the integer pixel displacement and sub-pixel displacement. If so, do not output the integer pixel displacement and sub-pixel displacement.
2. The trajectory correction method of the optoelectronic navigation device according to claim 1, characterized in that: According to the reference block, the correlation peak value of the whole pixel and the correlation peak value of the sub-pixel are calculated by the same matching method, which specifically includes: According to the reference block, multiple target blocks are selected on the target frame in a translation manner; the number of the target blocks is determined according to the specification of the search window selected by the correlation search; the range of each target block is the same as the range of the reference block; Performing an integer pixel correlation operation and a sub-pixel correlation operation on the reference block and each target block respectively to obtain a plurality of integer pixel correlation values and a plurality of sub-pixel correlation values; The correlation peak of the integer pixel is selected from the integer pixel correlation value, and the correlation peak of the sub-pixel is selected from the sub-pixel correlation value.
3. The trajectory correction method of the optoelectronic navigation device according to claim 2, characterized in that: According to the reference block, multiple target blocks are selected on the target frame in a translation manner, specifically including: The target frame is translated relative to the reference frame; the translation operation specifically includes: up, down, left, right, left-upper, left-lower, right-upper, right-lower and no translation; After each operation, the part of the target frame that overlaps with the reference block is selected as the target block to obtain multiple target blocks.
4. The trajectory correction method of the optoelectronic navigation device according to claim 2, characterized in that: Performing integer pixel correlation calculation and sub-pixel correlation calculation on the reference block and each target block respectively to obtain multiple integer pixel correlation values and multiple sub-pixel correlation values, specifically including: The pixel values corresponding to the integer pixels of the target block and the reference block obtained after each translation are multiplied and summed to obtain multiple integer pixel correlation values; The pixel values corresponding to the sub-pixels of the target block and the reference block obtained after each translation are multiplied and summed to obtain a plurality of sub-pixel correlation values.
5. The trajectory correction method of the optoelectronic navigation device according to claim 2, characterized in that: The size of the search window is 3×3.
6. The trajectory correction method of the optoelectronic navigation device according to claim 1, characterized in that: The reference block is a rectangular image constructed according to the range of trajectory changes of the reference frame.
7. The trajectory correction method of the optoelectronic navigation device according to claim 1, characterized in that: The value of N is 5.
8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the trajectory correction method of the optoelectronic navigation device described in any one of claims 1-7.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the trajectory correction method of the optoelectronic navigation device described in any one of claims 1 to 7 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the trajectory correction method of the optoelectronic navigation device described in any one of claims 1 to 7 is implemented.