Displacement determination method, apparatus, device, and medium

Abstract

The application provides a displacement determination method and device, and a medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring a first pixel array, a second pixel array and a third pixel array adjacent in frames from continuous multiple frames of images; determining a reference displacement according to the first pixel array and the second pixel array, and determining a first reference component and a second reference component according to the reference displacement; pre-processing the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array; pre-processing the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array; determining an initial displacement according to the first target pixel array and the second target pixel array; and determining a target displacement according to the initial displacement, the first reference component and the second reference component. The application improves the accuracy of displacement identification.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a displacement determination method, device, and medium. Background Art

[0002] Common image algorithms used to measure object motion include digital image correlation and optical flow. Essentially, both algorithms determine an object's displacement by analyzing the direct differences between consecutive image frames. However, when an object moves rapidly and over large distances, the differences between consecutive image frames are significant, and existing displacement identification methods often produce significant errors. Therefore, developing a more accurate displacement determination method is an urgent need. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to overcome the deficiencies in the prior art and provide a method, device, equipment and medium for determining displacement. The present invention provides the following technical solutions:

[0004] In a first aspect, the present invention provides a method for determining displacement, the method comprising:

[0005] Acquire a plurality of pixel arrays with adjacent frames from a continuous plurality of frames of images, wherein the plurality of pixel arrays are, in order of frames, a first pixel array, a second pixel array, and a third pixel array;

[0006] determining a reference displacement according to the first pixel array and the second pixel array, and determining a first reference component and a second reference component according to the reference displacement;

[0007] preprocessing the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array;

[0008] preprocessing the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array;

[0009] determining, based on the first target pixel array and the second target pixel array, an initial displacement of the target object within a target time, where the target time is a frame interval between the second pixel array and the third pixel array;

[0010] A target displacement of the target object within the target time is determined according to the initial displacement, the first reference component, and the second reference component.

[0011] In one embodiment, determining the first reference component and the second reference component based on the reference displacement includes: determining the component of the reference displacement on a first direction axis as the first reference component, the first direction axis being the same as the column arrangement direction of the first pixel array; and determining the component of the reference displacement on a second direction axis as the second reference component, the second direction axis being the same as the row arrangement direction of the first pixel array.

[0012] In one embodiment, before pre-processing the second pixel array according to the first reference component and the second reference component, the method further includes:

[0013] According to the first reference component, the column rejection value P is determined, and the calculation formula is:

[0014] P = [X + 0.5]

[0015] According to the second reference component, the row discard value Q is determined, and the calculation formula is:

[0016] Q = [Y + 0.5]

[0017] Wherein, X represents the value corresponding to the first reference component, and Y represents the value corresponding to the second reference component.

[0018] In one embodiment, the first pixel array, the second pixel array, and the third pixel array each include N rows×M columns of pixels, and preprocessing the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array includes:

[0019] For the second pixel array, if the direction of the first reference component is the same as the first direction axis, and the second reference component is the same as the second direction axis, discarding the pixels in columns 1 to P and the pixels in rows 1 to Q to obtain a first target pixel array;

[0020] If the direction of the first reference component is opposite to the first direction axis, and the second reference component is the same as the second direction axis, discarding the pixels in the (MP)th to Mth columns and the pixels in the 1st to Qth rows to obtain a first target pixel array;

[0021] If the first reference component is the same as the first direction axis, and the second reference component is opposite to the second direction axis, discarding the pixels in columns 1 to P and rows (NQ) to N to obtain a first target pixel array;

[0022] If the first reference component is opposite to the first direction axis and the second reference component is opposite to the second direction axis, pixels in the (MP)th to Mth columns and pixels in the (NQ)th to Nth rows are discarded to obtain a first target pixel array.

[0023] In an embodiment, the first pixel array, the second pixel array and the third pixel array each comprises N rows by M columns of pixels, and the pre-processing of the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array comprises:

[0024] For the third pixel array, if the direction of the first reference component is the same as the first direction axis and the second reference component is the same as the second direction axis, then the (M-P)th to Mth columns of pixels and the (N-Q)th to Nth rows of pixels are discarded to obtain the second target pixel array.

[0025] If the direction of the first reference component is opposite to the first direction axis and the second reference component is the same as the second direction axis, then the 1st to Pth columns of pixels and the (N-Q)th to Nth rows of pixels are discarded to obtain the second target pixel array.

[0026] If the first reference component is the same as the first direction axis and the second reference component is opposite to the second direction axis, then the (N-Q)th to Nth rows of pixels and the 1st to Qth rows of pixels are discarded to obtain the second target pixel array.

[0027] If the first reference component is opposite to the first direction axis and the second reference component is opposite to the second direction axis, then the 1st to Pth columns of pixels and the 1st to Qth rows of pixels are discarded to obtain the second target pixel array.

[0028] In an embodiment, the determining of the target displacement of the target object in the target time according to the initial displacement, the first reference component and the second reference component comprises: determining a first initial component of the initial displacement in the first direction axis and a second initial component of the initial displacement in the second direction axis; determining a first target component according to the first reference component and the first initial component, and determining a second target component according to the second reference component and the second initial component; and determining the target displacement of the target object in the target time according to the first target component and the second target component.

[0029] In a second aspect, the present application provides a displacement determining device, which comprises:

[0030] The obtaining module is configured to obtain a plurality of pixel arrays adjacent in frame from a plurality of continuous images, and the plurality of pixel arrays in frame order are: a first pixel array, a second pixel array and a third pixel array.

[0031] a first determining module, configured to determine a reference displacement according to the first pixel array and the second pixel array, and determine a first reference component and a second reference component according to the reference displacement;

[0032] a preprocessing module, configured to preprocess the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array; and preprocess the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array;

[0033] a second module, configured to determine an initial displacement of the target object within a target time based on the first target pixel array and the second target pixel array, where the target time is a frame interval between the second pixel array and the third pixel array;

[0034] A third determining module is configured to determine a target displacement of the target object within the target time according to the initial displacement, the first reference component, and the second reference component.

[0035] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program runs on the processor, the displacement determination method described in the first aspect is executed.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program performs the displacement determination method described in the first aspect when executed by a processor.

[0037] The present invention provides a displacement determination method, apparatus, device, and medium. The method includes: acquiring multiple pixel arrays adjacent to each other in frames from a continuous multi-frame image, wherein the multiple pixel arrays are, in frame order, a first pixel array, a second pixel array, and a third pixel array; determining a reference displacement based on the first pixel array and the second pixel array, and determining a first reference component and a second reference component based on the reference displacement; preprocessing the second pixel array based on the first reference component and the second reference component to obtain a first target pixel array; preprocessing the third pixel array based on the first reference component and the second reference component to obtain a second target pixel array; determining an initial displacement of a target object within a target time based on the first target pixel array and the second target pixel array, wherein the target time is a frame interval between the second pixel array and the third pixel array; and determining a target displacement of the target object within the target time based on the initial displacement, the first reference component, and the second reference component. The present invention improves the accuracy of displacement recognition when the object moves too fast and the displacement distance is large.

[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 A schematic diagram of a flow chart of a displacement determination method provided by an embodiment of the present invention is shown;

[0041] Figure 2 A schematic diagram of an N-row×M-column pixel array provided by an embodiment of the present invention is shown;

[0042] Figure 3 A schematic diagram showing a first direction axis and a second direction axis provided by an embodiment of the present invention is shown;

[0043] Figure 4 A schematic diagram of a movement of a pixel array corresponding to a target object provided by an embodiment of the present invention is shown;

[0044] Figure 5 Another schematic flow chart of the displacement determination method provided by an embodiment of the present invention is shown;

[0045] Figure 6 Another schematic flow chart of a displacement determination method provided by an embodiment of the present invention is shown;

[0046] Figure 7 A schematic structural diagram of a displacement determination device provided by an embodiment of the present invention is shown;

[0047] Figure 8 A schematic structural diagram of an electronic device provided by an embodiment of the present invention is shown.

[0048] Description of main component symbols:

[0049] 700 - displacement determination device; 710 - acquisition module; 720 - first determination module; 730 - preprocessing module; 740 - second determination module; 750 - third determination module; 800 - electronic device; 801 - transceiver; 802 - processor; 803 - memory. DETAILED DESCRIPTION

[0050] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0051] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. The terms used in the template description herein are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0053] Example 1

[0054] Currently, the main image algorithms used to measure object motion are digital correlation and optical flow. Digital correlation determines object motion information by collecting grayscale images before and after the object moves and performing correlation calculations based on these grayscale images. Common digital correlation methods include the difference method, correlation coefficient fitting method, and gradient method. Optical flow algorithms determine object motion information based on the temporal changes in pixels in an image sequence and inter-frame correlation. Common optical flow algorithms include the LK (Lucas-Kanade) optical flow method and the HS (Horn-Schunck) optical flow method.

[0055] It can be understood that the essence of these algorithms is to determine displacement information by analyzing the direct difference between two adjacent images. When the object moves too fast and the displacement distance is large, the difference between adjacent images is too large. At this time, there is often a large error between the displacement determined according to the existing technology and the actual displacement. Taking a 16×16 pixel array as an example, performing displacement calculations on the 16×16 pixel array based on the optical flow algorithm can ensure the calculation accuracy of the displacement of about 2 pixels. However, when the displacement exceeds 2 pixels, the calculation accuracy will drop significantly. When the displacement exceeds 3 pixels, calculation errors will occur. In this regard, in order to ensure the accuracy of displacement recognition, an embodiment of the present invention provides a displacement determination method, please refer to Figure 1 , the method includes steps S110 to S160.

[0056] Step S110 , acquiring a plurality of pixel arrays with adjacent frames from a plurality of consecutive frames of images, wherein the plurality of pixel arrays are, in order of frames, a first pixel array, a second pixel array, and a third pixel array.

[0057] A pixel array refers to an ordered arrangement of pixels. For a pixel array in a continuous frame image, if the position of a target object changes between consecutive frames, the position information of the corresponding pixel array will also change. Therefore, based on the changes in the pixel array corresponding to the target object in the continuous frame image, the position change of the target object can be determined.

[0058] In this embodiment, the first pixel array is the pixel distribution corresponding to the target object in the i-1th frame image, the second pixel array is the pixel distribution corresponding to the target object in the i-th frame image, and the third pixel array is the pixel distribution corresponding to the target object in the i+1th image, where i≥2.

[0059] Step S120 , determining a reference displacement according to the first pixel array and the second pixel array, and determining a first reference component and a second reference component according to the reference displacement.

[0060] In this embodiment, the first pixel array represents the pixel distribution of the target object in the (i-1)th frame, and the second pixel array represents the pixel distribution of the target object in the (i)th frame. Based on the first and second pixel arrays, an optical flow algorithm, feature matching, or digital correlation method is used to determine the displacement from the first pixel array to the second pixel array, i.e., the reference displacement.

[0061] Understandably, when a target object moves quickly and has a large displacement, the pixel arrays corresponding to the object before and after its movement often contain significant noise in the displacement direction. Directly determining displacement based on the pixel arrays before and after the object's movement will result in low accuracy. Therefore, to ensure accurate displacement identification, the pixel arrays before and after the object's movement must be preprocessed before determining displacement.

[0062] It can be understood that, within a given frame interval, the faster the speed and the greater the displacement, the more noise is contained in the corresponding pixel arrays before and after the target object moves. In other words, the displacement is proportional to the noise contained before and after the target object moves. In this embodiment, taking the displacement from the second pixel array to the third pixel array (i.e., the target displacement of the target object) as an example, according to the principles of kinematics, changes in speed and direction are gradual and typically do not occur suddenly. Therefore, based on the magnitude and direction of the reference displacement, the second and third pixels can be preprocessed to remove noise from the second and third pixels.

[0063] In one embodiment, determining the first reference component and the second reference component based on the reference displacement includes: determining the component of the reference displacement on a first direction axis as the first reference component, the first direction axis being the same as the column arrangement direction of the first pixel array; and determining the component of the reference displacement on a second direction axis as the second reference component, the second direction axis being the same as the row arrangement direction of the first pixel array.

[0064] In this embodiment, the reference displacement is a vector, directed from the first pixel array to the second pixel array. With the column arrangement direction of the first pixel array as the first axis and the row arrangement direction of the first pixel array as the second axis, the reference displacement is vector-decomposed to obtain the component of the reference displacement along the first axis and the component of the reference displacement along the second axis.

[0065] The pixel array consists of rows and columns of pixels. When the target object moves too quickly, the pixel array contains significant noise in the displacement direction. By decomposing the reference displacement into a first reference vector and a second reference vector, the row and column pixels that are blurred due to the target object's rapid motion can be determined.

[0066] Step S130 : Preprocessing the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array.

[0067] In this embodiment, the displacement of the target object during the frame interval between the second and third pixel arrays is determined as the target displacement. It is understood that due to the continuity of motion, the target object's speed and direction do not change suddenly during its movement. Therefore, pre-processing can be performed on the second and third pixel arrays based on the magnitude and direction of the reference displacement.

[0068] Specifically, a first reference component and a second reference component can be used according to the reference displacement, and part of the row pixels and part of the column pixels on the pixel array are determined to be discarded according to the first reference component and the second reference component, wherein the first reference component is used to determine the number of discarded row pixels, and the second reference component is used to determine the number of discarded column pixels.

[0069] In one embodiment, before pre-processing the second pixel array according to the first reference component and the second reference component, the method further includes:

[0070] According to the first reference component, the column rejection value P is determined, and the calculation formula is:

[0071] P = [X + 0.5]

[0072] According to the second reference component, the row discard value Q is determined, and the calculation formula is:

[0073] Q=[Y+0.5]

[0074] Wherein, X represents the value corresponding to the first reference component, and Y represents the value corresponding to the second reference component.

[0075] In this embodiment, [X+0.5] represents the maximum integer not exceeding X+0.5, and [Y+0.5] represents the maximum integer not exceeding Y+0.5. Taking the first reference component as 0.5 pixels and the second reference component as 1.2 pixels as an example, P=[0.5+0.5]=1, Q=[1.2+0.5]=1.

[0076] It should be noted that when the target object moves too quickly, for the second pixel array, the row pixels and / or column pixels away from the direction of motion are likely to be blurred, and for the third pixel array, the row pixels and / or column pixels close to the direction of motion are likely to be blurred. Therefore, pre-processing the second pixel array includes discarding P columns of pixels away from the direction of motion and / or Q rows of pixels away from the direction of motion. Pre-processing the third pixel array includes discarding P columns of pixels close to the direction of motion and / or Q rows of pixels close to the direction of motion.

[0077] In one embodiment, the first pixel array, the second pixel array, and the third pixel array each include N rows×M columns of pixels, and preprocessing the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array includes:

[0078] For the second pixel array, if the direction of the first reference component is the same as the first direction axis, and the second reference component is the same as the second direction axis, discarding the pixels in columns 1 to P and the pixels in rows 1 to Q to obtain a first target pixel array;

[0079] If the direction of the first reference component is opposite to the first direction axis, and the second reference component is the same as the second direction axis, discarding the pixels in the (MP)th to Mth columns and the pixels in the 1st to Qth rows to obtain a first target pixel array;

[0080] If the first reference component is the same as the first direction axis, and the second reference component is opposite to the second direction axis, discarding pixels in columns 1 to P and pixels in rows (NQ) to N to obtain a first target pixel array;

[0081] If the first reference component is opposite to the first direction axis and the second reference component is opposite to the second direction axis, pixels in the (MP)th to Mth columns and pixels in the (NQ)th to Nth rows are discarded to obtain a first target pixel array.

[0082] See Figure 2 , Figure 2 FIG1 shows a schematic diagram of an N-row×M-column pixel array. In this embodiment, the pixel array rows are arranged from top to bottom, and the columns are arranged from left to right. Figure 2 For example, the pixel array shown in FIG. 1 is sequentially arranged from top to bottom, i.e., row 1, row 2, ..., row N, and from left to right, i.e., column 1, column 2, ..., column M. It should be noted that the first direction axis is the same as the column arrangement direction, and the second direction axis is the same as the row arrangement direction. Figure 3 , Figure 3 A schematic diagram of the first direction axis and the second direction axis provided in an embodiment of the present application is shown. The direction of the first direction axis is from the first column to the Mth column, and the direction of the second direction axis is from the first row to the Nth row.

[0083] In this embodiment, if the first reference component is the same as the first direction axis, it means that the movement direction of the target object on the first direction axis is the same as the first direction axis; conversely, if the first reference component is opposite to the first direction axis, it means that the movement direction of the target object on the first direction axis is opposite to the first direction axis; if the second reference component is the same as the second direction axis, it means that the movement direction of the target object on the second direction axis is the same as the second direction axis; conversely, if the second reference component is opposite to the second direction axis, it means that the movement direction of the target object on the second direction axis is opposite to the second direction axis.

[0084] It should be noted that, for the second pixel array, row pixels and / or column pixels that are far from the motion direction are prone to blurring, and therefore, P column pixels and / or Q row pixels that are far from the motion direction need to be discarded. For the third pixel array, row pixels and / or column pixels that are close to the motion direction are prone to blurring, and therefore, P column pixels and / or Q row pixels that are close to the motion direction need to be discarded.

[0085] Step S140 : Preprocessing the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array.

[0086] As described above, pre-processing the third pixel array includes discarding P columns of pixels close to the motion direction and / or Q rows of pixels close to the motion direction.

[0087] In one embodiment, the first pixel array, the second pixel array, and the third pixel array each include N rows×M columns of pixels, and preprocessing the third pixel array according to the first reference component and the second reference component to obtain the second target pixel array includes:

[0088] For the third pixel array, if the direction of the first reference component is the same as the first direction axis, and the second reference component is the same as the second direction axis, discarding pixels in columns (MP) to M and pixels in rows (NQ) to N to obtain a second target pixel array;

[0089] If the direction of the first reference component is opposite to the first direction axis, and the second reference component is the same as the second direction axis, discarding the pixels in columns 1 to P and the pixels in rows (NQ) to N to obtain a second target pixel array;

[0090] If the first reference component is the same as the first direction axis, and the second reference component is opposite to the second direction axis, discarding the (NQ)th to Nth rows of pixels and the 1st to Qth rows of pixels to obtain a second target pixel array;

[0091] If the first reference component is opposite to the first direction axis, and the second reference component is opposite to the second direction axis, pixels in columns 1 to P and pixels in rows 1 to Q are discarded to obtain a second target pixel array.

[0092] See Figure 4 , Figure 4 A schematic diagram of a movement of a pixel array corresponding to a target object provided by an embodiment of the present application is shown. Specifically, Figure 4 The left side is the first pixel array corresponding to the target object before the reference displacement occurs. Figure 4 The right side shows the second pixel array corresponding to the target object after the reference displacement occurs. Figure 4 The reference displacement is 3 pixels, a first reference component of the reference displacement in the first direction axis is 3 pixels, and a second reference component in the second direction axis is 0 pixel.

[0093] Based on this, when determining the target displacement of the target object, that is, the displacement from the second pixel array to the third pixel array, the second pixel array must first be processed as follows: pixels in columns 1 to 3 of the second pixel array must be discarded. Then, the third pixel array must be processed as follows: pixels in columns (N-3) to N must be discarded. The column discard value is determined according to the formula P = [X + 0.5], where X = 3.

[0094] Step S150 : determining an initial displacement of the target object within a target time according to the first target pixel array and the second target pixel array, where the target time is a frame interval between the second pixel array and the third pixel array.

[0095] The first target pixel array and the second target pixel array are calculated using an optical flow algorithm to obtain an initial displacement of the target object within the target time. Since the first target pixel array and the second target pixel array both discard some row pixels and / or column pixels, the processing described in step S160 is required after obtaining the initial displacement.

[0096] Step S160 : determining a target displacement of the target object within the target time according to the initial displacement, the first reference component, and the second reference component.

[0097] In one embodiment, see Figure 5 , step S160 includes steps S161 to S163.

[0098] Step S161: Determine a first initial component of the initial displacement on the first direction axis and a second initial component on the second direction axis.

[0099] Since the initial displacement is obtained based on the first target pixel array and the second target pixel array by discarding some row pixels and / or column pixels, after obtaining the initial displacement, it is necessary to add the discarded pixel values ​​to the initial displacement. For details, please refer to steps S162 and S163.

[0100] Step S162: determining a first target component according to the first reference component and the first initial component, and determining a second target component according to the second reference component and the second initial component.

[0101] It can be understood that the initial displacement is a vector, which is decomposed into a first initial component on the first direction axis and a second initial component on the second direction axis, and then added to the discarded pixel values ​​on the corresponding direction axes to obtain the first target component and the second target component.

[0102] Step S163: determining the target displacement of the target object within the target time according to the first target component and the second target component.

[0103] The vector sum of the first target component and the second target component is the target displacement of the target object within the target time.

[0104] It can be understood that compared with the existing technology that directly calculates displacement through the pixel arrays corresponding to the target object before and after movement, the embodiments of the present invention avoid the influence of blurred pixels in the pixel array in the direction of movement caused by the object's excessive speed, thereby improving the accuracy of displacement determination.

[0105] In one embodiment, see Figure 6 , the method also includes steps S610 to S630.

[0106] Step S610, judging whether the direction of the target displacement is consistent with the direction of the reference displacement, if consistent, increasing the counter value by a first preset number of times.

[0107] The displacement determination method provided by the embodiments of the present application is that the displacement result of the previous time will affect the displacement result of the next time, therefore, in order to avoid the avalanche error, a gatekeeper mechanism is needed. The gatekeeper mechanism is specifically: there is a counter, when the direction of the target displacement is inconsistent with the direction of the reference displacement, the counter decreases by a second preset number of times, for example, 2; when the direction of the target displacement is consistent with the direction of the reference displacement, the counter increases by a first preset number of times, for example, 1. After a series of operations, if the value of the counter is equal to a preset value, it is considered that the displacement determination is wrong, and the displacement is determined again.

[0108] Step S620, if not consistent, the counter value decreases by a second preset number of times, and the first preset number of times is less than the second preset number of times.

[0109] It can be understood that if the actual motion direction starts to change rapidly, and the counter does not decrease significantly, the algorithm will not realize that it needs to be re-predicted. By adjusting the reward and punishment mechanism, the punishment for inconsistency is greater than the reward for consistency, that is, the second preset number of times is greater than the first preset number of times. In this way, even if the direction change is not immediate, the algorithm will realize that it needs to be re-predicted faster.

[0110] Step S630, when the counter value is equal to a preset value, the target displacement is determined again.

[0111] It should be noted that the initial value of the counter can be set according to actual needs, for example, 0XF. In order to avoid avalanche error, when the value of the counter is equal to a preset value, for example, 0, the target displacement is determined again.

[0112] An embodiment of the present invention provides a displacement determination method without increasing the amount of computation and without placing excessive demands on pixel arrays. The method comprises obtaining multiple pixel arrays adjacent to each other in a continuous multi-frame image, wherein the multiple pixel arrays are, in frame order, a first pixel array, a second pixel array, and a third pixel array. A reference displacement is determined based on the first pixel array and the second pixel array, and a first reference component and a second reference component are determined based on the reference displacement. The second pixel array is pre-processed based on the first reference component and the second reference component to obtain a first target pixel array. The third pixel array is pre-processed based on the first reference component and the second reference component to obtain a second target pixel array. An initial displacement of a target object within a target time is determined based on the first target pixel array and the second target pixel array, where the target time is the frame interval between the second pixel array and the third pixel array. A target displacement of the target object within the target time is determined based on the initial displacement, the first reference component, and the second reference component. The present invention improves the accuracy of displacement recognition when the object moves too fast and the displacement distance is large.

[0113] Example 2

[0114] Also, see Figure 7 The embodiment of the present invention further provides a displacement determination device 700, the device comprising:

[0115] An acquisition module 710 is configured to acquire a plurality of pixel arrays adjacent to each other in a continuous plurality of frames of images, wherein the plurality of pixel arrays are, in order of frames, a first pixel array, a second pixel array, and a third pixel array;

[0116] a first determining module 720 configured to determine a reference displacement according to the first pixel array and the second pixel array, and determine a first reference component and a second reference component according to the reference displacement;

[0117] a preprocessing module 730 configured to preprocess the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array; and preprocess the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array;

[0118] A second determining module 740 is configured to determine an initial displacement of the target object within a target time based on the first target pixel array and the second target pixel array, where the target time is a frame interval between the second pixel array and the third pixel array;

[0119] A third determining module 750 is configured to determine a target displacement of the target object in the target time according to the initial displacement, the first reference component and the second reference component.

[0120] The displacement determining apparatus provided by the embodiment of the present application obtains a plurality of pixel arrays adjacent in frames from continuous multiple frames of images, the plurality of pixel arrays are sequentially the first pixel array, the second pixel array and the third pixel array in frame order; determines a reference displacement according to the first pixel array and the second pixel array, determines a first reference component and a second reference component according to the reference displacement; pre-processes the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array; pre-processes the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array; determines an initial displacement of the target object in a target time according to the first target pixel array and the second target pixel array, the target time is an interval time between the second pixel array and the third pixel array; determines a target displacement of the target object in the target time according to the initial displacement, the first reference component and the second reference component, and improves the accuracy of displacement identification when the object moves at a high speed and the displacement distance is large.

[0121] The displacement determining apparatus provided by the embodiment of the present application is used to execute the displacement determining method described in the above embodiment 1, and details are not repeated here.

[0122] Embodiment 3

[0123] In addition, the embodiment of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the computer program executes the displacement determining method provided in the embodiment 1 when running on the processor.

[0124] Specifically, please refer to Figure 8The electronic device 800 includes: a transceiver 801, a bus interface and a processor 802, the processor 802 being configured to obtain a plurality of pixel arrays adjacent to each other in frames from a plurality of consecutive image frames, the plurality of pixel arrays being, in frame order, a first pixel array, a second pixel array and a third pixel array; determining a reference displacement based on the first pixel array and the second pixel array, and determining a first reference component and a second reference component based on the reference displacement; pre-processing the second pixel array based on the first reference component and the second reference component to obtain a first target pixel array; pre-processing the third pixel array based on the first reference component and the second reference component to obtain a second target pixel array; determining an initial displacement of the target object within a target time based on the first target pixel array and the second target pixel array, the target time being a frame interval between the second pixel array and the third pixel array; and determining a target displacement of the target object within the target time based on the initial displacement, the first reference component and the second reference component.

[0125] In the embodiment of the present invention, the electronic device 800 further includes a memory 803. Figure 8 In the embodiment, the bus architecture can include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 802 and memory represented by memory 803. The bus architecture can also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are all well known in the art and therefore will not be further described herein. The bus interface provides an interface. The transceiver 801 can be multiple components, that is, including a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. The processor 802 is responsible for managing the bus architecture and general processing, and the memory 803 can store data used by the processor 802 when performing operations.

[0126] The electronic device 800 provided in the embodiment of the present invention can execute the displacement determination method provided in the above method embodiment 1, which will not be described again here to avoid repetition.

[0127] Example 4

[0128] In addition, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the displacement determination method provided in Example 1 is implemented.

[0129] In this embodiment, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0130] The computer-readable storage medium provided in this embodiment can implement the displacement determination method provided in Example 1, and will not be described again here to avoid repetition.

[0131] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not limiting, and thus other examples of the exemplary embodiments may have different values.

[0132] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0133] The above-described embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that variations and modifications are possible without departing from the scope of the present invention, and such variations and modifications are fully within the scope of protection of the present invention.

Claims

1. A method for determining displacement, characterized in that: The method comprises: Acquire a plurality of pixel arrays with adjacent frames from a continuous plurality of frames of images, wherein the plurality of pixel arrays are, in order of frames, a first pixel array, a second pixel array, and a third pixel array; determining a reference displacement according to the first pixel array and the second pixel array, and determining a first reference component and a second reference component according to the reference displacement; preprocessing the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array; preprocessing the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array; determining an initial displacement of a target object within a target time according to the first target pixel array and the second target pixel array, where the target time is a frame interval between the second pixel array and the third pixel array; A target displacement of the target object within the target time is determined according to the initial displacement, the first reference component, and the second reference component.

2. The displacement determination method according to claim 1, characterized in that: The determining of the first reference component and the second reference component according to the reference displacement includes: Determine a component of the reference displacement in a first direction axis as a first reference component, where the first direction axis is the same as a column arrangement direction of the first pixel array; A component of the reference displacement in a second direction axis is determined as a second reference component, where the second direction axis is the same as a row arrangement direction of the first pixel array.

3. The displacement determination method according to claim 2, characterized in that: Before pre-processing the second pixel array according to the first reference component and the second reference component, the method further includes: According to the first reference component, the column rejection value P is determined, and the calculation formula is: P = [X + 0.5] According to the second reference component, the row discard value Q is determined, and the calculation formula is: Q=[Y+0.5] Wherein, X represents the value corresponding to the first reference component, and Y represents the value corresponding to the second reference component.

4. The displacement determination method according to claim 3, characterized in that: The first pixel array, the second pixel array, and the third pixel array each include N rows×M columns of pixels, and preprocessing the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array includes: For the second pixel array, if the direction of the first reference component is the same as the first direction axis, and the second reference component is the same as the second direction axis, discarding the pixels in columns 1 to P and the pixels in rows 1 to Q to obtain a first target pixel array; If the direction of the first reference component is opposite to the first direction axis, and the second reference component is the same as the second direction axis, discarding the pixels in the (MP)th to Mth columns and the pixels in the 1st to Qth rows to obtain a first target pixel array; If the first reference component is the same as the first direction axis, and the second reference component is opposite to the second direction axis, discarding pixels in columns 1 to P and pixels in rows (NQ) to N to obtain a first target pixel array; If the first reference component is opposite to the first direction axis and the second reference component is opposite to the second direction axis, pixels in the (MP)th to Mth columns and pixels in the (NQ)th to Nth rows are discarded to obtain a first target pixel array.

5. The displacement determination method according to claim 3, characterized in that: The first pixel array, the second pixel array, and the third pixel array each include N rows×M columns of pixels, and preprocessing the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array includes: For the third pixel array, if the direction of the first reference component is the same as the first direction axis, and the second reference component is the same as the second direction axis, discarding pixels in columns (MP) to M and pixels in rows (NQ) to N to obtain a second target pixel array; If the direction of the first reference component is opposite to the first direction axis, and the second reference component is the same as the second direction axis, discarding the pixels in columns 1 to P and the pixels in rows (NQ) to N to obtain a second target pixel array; If the first reference component is the same as the first direction axis, and the second reference component is opposite to the second direction axis, discarding the (NQ)th to Nth rows of pixels and the 1st to Qth rows of pixels to obtain a second target pixel array; If the first reference component is opposite to the first direction axis, and the second reference component is opposite to the second direction axis, pixels in columns 1 to P and pixels in rows 1 to Q are discarded to obtain a second target pixel array.

6. The displacement determination method according to claim 5, characterized in that: The determining, based on the initial displacement, the first reference component, and the second reference component, a target displacement of the target object within a target time includes: Determining a first initial component of the initial displacement on the first direction axis and a second initial component on the second direction axis; Determine a first target component based on the first reference component and the first initial component, and determine a second target component based on the second reference component and the second initial component; A target displacement of the target object within the target time is determined according to the first target component and the second target component.

7. The displacement determination method according to any one of claims 1 to 6, characterized in that: The method further comprises: Determining whether the directions of the target displacement and the reference displacement are consistent, and if so, increasing the counter value by a first preset number of times; If they are inconsistent, the counter value is reduced by a second preset number of times, and the first preset number of times is less than the second preset number of times; When the counter value is equal to a preset value, the target displacement is re-determined.

8. A displacement determination device, characterized in that: The device comprises: An acquisition module is used to acquire a plurality of pixel arrays adjacent to each other in frames from a plurality of consecutive frames of images, wherein the plurality of pixel arrays are, in order of frames, a first pixel array, a second pixel array, and a third pixel array; a first determining module, configured to determine a reference displacement according to the first pixel array and the second pixel array, and determine a first reference component and a second reference component according to the reference displacement; a preprocessing module, configured to preprocess the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array; and preprocess the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array; a second determining module, configured to determine an initial displacement of the target object within a target time based on the first target pixel array and the second target pixel array, where the target time is a frame interval between the second pixel array and the third pixel array; A third determining module is configured to determine a target displacement of the target object within the target time according to the initial displacement, the first reference component, and the second reference component.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is run on the processor, the displacement determination method according to any one of claims 1 to 7 is executed.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the displacement determination method according to any one of claims 1 to 7 is implemented.

Images