Displacement determination method and device, equipment and medium

Abstract

The invention provides a displacement determination method and device, equipment and a medium, and relates to the technical field of image processing, and the method comprises the steps: obtaining a first pixel array, a second pixel array and a third pixel array which are adjacent in frames from a continuous multi-frame image; determining a reference displacement according to the first pixel array and the second pixel array, and determining a first reference component and a second reference component according to the reference displacement; preprocessing the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array; preprocessing the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array; determining an initial displacement according to the first target pixel array and the second target pixel array; and determining a target displacement according to the initial displacement, the first reference component and the second reference component. The accuracy of displacement identification is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a displacement determination method, device and medium. Background Art

[0002] Common image algorithms used to measure the motion of objects include digital image correlation and optical flow. The essence of these two algorithms is to determine the displacement of an object by analyzing the direct differences between consecutive image frames. However, when the object moves too fast and the displacement distance is large, the difference between consecutive image frames is large, and the existing displacement recognition methods often produce large errors. Therefore, proposing a displacement determination method with higher accuracy has become an urgent problem to be solved. Summary of the invention

[0003] In view of this, the purpose of the present invention is to overcome the deficiencies in the prior art and provide a displacement determination method, device, equipment and medium. The present invention provides the following technical solutions:

[0004] In a first aspect, the present invention provides a displacement determination method, the method comprising:

[0005] Acquire a plurality of pixel arrays with adjacent frames from a plurality of continuous frames of images, wherein the plurality of pixel arrays are, in order of frames: a first pixel array, a second pixel array, and a third pixel array;

[0006] Determine a reference displacement according to the first pixel array and the second pixel array, and determine a first reference component and a second reference component according to the reference displacement;

[0007] Preprocessing the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array;

[0008] Preprocessing the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array;

[0009] Determining an initial displacement of the target object within a target time according to the first target pixel array and the second target pixel array, wherein the target time is a frame interval between the second pixel array and the third pixel array;

[0010] A target displacement of the target object within the target time is determined according to the initial displacement, the first reference component, and the second reference component.

[0011] In one embodiment, determining the first reference component and the second reference component based on the reference displacement includes: determining the component of the reference displacement on a first direction axis as the first reference component, the first direction axis being the same as the column arrangement direction of the first pixel array; determining the component of the reference displacement on a second direction axis as the second reference component, the second direction axis being the same as the row arrangement direction of the first pixel array.

[0012] In one embodiment, before preprocessing the second pixel array according to the first reference component and the second reference component, the method further includes:

[0013] According to the first reference component, the column discard value P is determined, and the calculation formula is:

[0014] P = [X + 0.5]

[0015] According to the second reference component, the row discard value Q is determined, and the calculation formula is:

[0016] Q = [Y + 0.5]

[0017] Wherein, X represents the value corresponding to the first reference component, and Y represents the value corresponding to the second reference component.

[0018] In one embodiment, the first pixel array, the second pixel array, and the third pixel array all include N rows×M columns of pixels, and the preprocessing of the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array includes:

[0019] For the second pixel array, if the direction of the first reference component is the same as the first direction axis, and the second reference component is the same as the second direction axis, then discard the pixels in the 1st to Pth columns and the pixels in the 1st to Qth rows to obtain a first target pixel array;

[0020] If the direction of the first reference component is opposite to the first direction axis, and the second reference component is the same as the second direction axis, discarding the pixels in the (MP)th to Mth columns and the pixels in the 1st to Qth rows to obtain a first target pixel array;

[0021] If the first reference component is the same as the first direction axis, and the second reference component is opposite to the second direction axis, discarding the pixels in the 1st to Pth columns and the pixels in the (NQ)th to Nth rows to obtain a first target pixel array;

[0022] If the first reference component is opposite to the first direction axis, and the second reference component is opposite to the second direction axis, pixels in the (MP)th to Mth columns and pixels in the (NQ)th to Nth rows are discarded to obtain a first target pixel array.

[0023] In one embodiment, the first pixel array, the second pixel array, and the third pixel array all include N rows×M columns of pixels, and the preprocessing of the third pixel array according to the first reference component and the second reference component to obtain the second target pixel array includes:

[0024] For the third pixel array, if the direction of the first reference component is the same as the first direction axis, and the second reference component is the same as the second direction axis, then discarding the (MP)th to Mth column pixels and the (NQ)th to Nth row pixels to obtain a second target pixel array;

[0025] If the direction of the first reference component is opposite to the first direction axis, and the second reference component is the same as the second direction axis, discarding the pixels in the 1st to Pth columns and the pixels in the (NQ)th to Nth rows to obtain a second target pixel array;

[0026] If the first reference component is the same as the first direction axis, and the second reference component is opposite to the second direction axis, discarding the (NQ)th to Nth rows of pixels and the 1st to Qth rows of pixels to obtain a second target pixel array;

[0027] If the first reference component is opposite to the first direction axis, and the second reference component is opposite to the second direction axis, the pixels in the 1st to Pth columns and the pixels in the 1st to Qth rows are discarded to obtain a second target pixel array.

[0028] In one embodiment, determining the target displacement of the target object within the target time based on the initial displacement, the first reference component and the second reference component includes: determining a first initial component of the initial displacement on the first direction axis, and a second initial component on the second direction axis; determining a first target component based on the first reference component and the first initial component, and determining a second target component based on the second reference component and the second initial component; determining the target displacement of the target object within the target time based on the first target component and the second target component.

[0029] In a second aspect, the present invention provides a displacement determination device, the device comprising:

[0030] An acquisition module, used for acquiring a plurality of pixel arrays with adjacent frames from a plurality of continuous frames of images, wherein the plurality of pixel arrays are, in order of frames, a first pixel array, a second pixel array, and a third pixel array;

[0031] A first determining module, configured to determine a reference displacement according to the first pixel array and the second pixel array, and determine a first reference component and a second reference component according to the reference displacement;

[0032] a preprocessing module, configured to preprocess the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array; and preprocess the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array;

[0033] A second module is used to determine the initial displacement of the target object within a target time according to the first target pixel array and the second target pixel array, where the target time is a frame interval between the second pixel array and the third pixel array;

[0034] A third determination module is configured to determine a target displacement of the target object within the target time according to the initial displacement, the first reference component and the second reference component.

[0035] In a third aspect, the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program runs on the processor, the displacement determination method described in the first aspect is executed.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to perform the displacement determination method described in the first aspect.

[0037] The present invention provides a displacement determination method, device, equipment and medium, the method comprising: acquiring a plurality of pixel arrays adjacent to each other in frames from a continuous multi-frame image, the plurality of pixel arrays being, in order of frames: a first pixel array, a second pixel array and a third pixel array; determining a reference displacement according to the first pixel array and the second pixel array, and determining a first reference component and a second reference component according to the reference displacement; preprocessing the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array; preprocessing the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array; determining an initial displacement of the target object within a target time according to the first target pixel array and the second target pixel array, the target time being a frame interval time between the second pixel array and the third pixel array; determining a target displacement of the target object within the target time according to the initial displacement, the first reference component and the second reference component, the present invention improves the accuracy of displacement recognition when the object moves too fast and the displacement distance is large.

[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 A schematic diagram of a flow chart of a displacement determination method provided by an embodiment of the present invention is shown;

[0041] Figure 2 A schematic diagram of an N-row×M-column pixel array provided by an embodiment of the present invention is shown;

[0042] Figure 3 A schematic diagram showing a first direction axis and a second direction axis provided by an embodiment of the present invention is shown;

[0043] Figure 4 A schematic diagram of a movement of a pixel array corresponding to a target object provided by an embodiment of the present invention is shown;

[0044] Figure 5 Another schematic diagram of the flow chart of the displacement determination method provided by an embodiment of the present invention is shown;

[0045] Figure 6 Another schematic diagram of a flow chart of a displacement determination method provided by an embodiment of the present invention is shown;

[0046] Figure 7 A schematic diagram of the structure of a displacement determination device provided by an embodiment of the present invention is shown;

[0047] Figure 8 A schematic structural diagram of an electronic device provided by an embodiment of the present invention is shown.

[0048] Description of main component symbols:

[0049] 700 - displacement determination device; 710 - acquisition module; 720 - first determination module; 730 - preprocessing module; 740 - second determination module; 750 - third determination module; 800 - electronic device; 801 - transceiver; 802 - processor; 803 - memory. DETAILED DESCRIPTION

[0050] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0051] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of the template are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0053] Example 1

[0054] At present, the image algorithms used to measure the motion of objects mainly include: digital correlation method and optical flow algorithm. Among them, the digital correlation method determines the object motion information by collecting grayscale images before and after the object moves, and performing correlation calculations based on the grayscale images before and after the movement. Common digital correlation methods include: difference method, correlation coefficient fitting method and gradient method; the optical flow algorithm determines the object motion information based on the pixel time change and inter-frame correlation in the image sequence. Common optical flow algorithms include: LK (Lucas-Kanade) optical flow method and HS (Horn-Schunck) optical flow method.

[0055] It can be understood that the essence of these algorithms is to determine the displacement information by analyzing the direct difference between two adjacent images. When the object moves too fast and the displacement distance is large, the difference between adjacent images is too large. At this time, the displacement determined according to the prior art often has a large error with the actual displacement. Taking the 16×16 pixel array as an example, the displacement calculation of the 16×16 pixel array based on the optical flow algorithm can ensure the calculation accuracy of the displacement of about 2 pixels. However, when the displacement exceeds 2 pixels, the calculation accuracy will drop significantly. When the displacement exceeds 3 pixels, calculation errors will occur. In this regard, in order to ensure the accuracy of displacement recognition, an embodiment of the present invention provides a displacement determination method, please refer to Figure 1 , the method includes steps S110 to S160.

[0056] Step S110 , acquiring a plurality of pixel arrays with adjacent frames from a plurality of continuous frame images, wherein the plurality of pixel arrays are, in order of frames, a first pixel array, a second pixel array, and a third pixel array.

[0057] A pixel array refers to an ordered arrangement of pixels. For a pixel array in a continuous frame image, if the position of a target object changes between continuous frames, the position information of the corresponding pixel array will also change. Therefore, based on the changes in the pixel array corresponding to the target object in the continuous frame image, the position change of the target object can be determined.

[0058] In this embodiment, the first pixel array is the pixel distribution corresponding to the target object in the i-1th frame image, the second pixel array is the pixel distribution corresponding to the target object in the i-th frame image, and the third pixel array is the pixel distribution corresponding to the target object in the i+1th image, i≥2.

[0059] Step S120 , determining a reference displacement according to the first pixel array and the second pixel array, and determining a first reference component and a second reference component according to the reference displacement.

[0060] In this embodiment, the first pixel array is the pixel distribution of the target object in the i-1th frame image, and the second pixel array is the pixel distribution of the target object in the i-th frame image. Based on the first pixel array and the second pixel array, the displacement from the first pixel array to the second pixel array, i.e., the reference displacement, is determined by an optical flow algorithm, a feature matching method, or a digital correlation method.

[0061] It can be understood that when the target object moves faster and has a larger displacement, the pixel arrays corresponding to the target object before and after the movement often contain larger noise in the displacement direction. If the displacement is determined directly based on the pixel arrays before and after the movement of the target object, the accuracy is low. Therefore, to ensure the accuracy of displacement recognition, the pixel arrays before and after the movement of the target object need to be preprocessed before determining the displacement.

[0062] It can be understood that at a certain frame interval, the faster the speed and the greater the displacement, the more noise the corresponding pixel array contains before and after the target object moves, that is, the displacement is proportional to the noise contained before and after the target object moves. In this embodiment, taking the displacement from the second pixel array to the third pixel array (that is, the target displacement of the target object) as an example, according to the principle of kinematics, the change of speed and direction is a process and usually does not occur suddenly. Therefore, according to the size and direction of the reference displacement, the second pixel and the third pixel can be pre-processed to remove the noise in the second pixel and the third pixel.

[0063] In one embodiment, determining the first reference component and the second reference component based on the reference displacement includes: determining the component of the reference displacement on a first direction axis as the first reference component, the first direction axis being the same as the column arrangement direction of the first pixel array; determining the component of the reference displacement on a second direction axis as the second reference component, the second direction axis being the same as the row arrangement direction of the first pixel array.

[0064] In this embodiment, the reference displacement is a vector, and its direction is from the first pixel array to the second pixel array. With the column arrangement direction of the first pixel array as the first direction axis and the row arrangement direction of the first pixel array as the second direction axis, the reference displacement is vector-decomposed to obtain the component of the reference displacement on the first direction axis and the component on the second direction axis.

[0065] The pixel array includes row pixels and column pixels. When the target object moves too fast, the pixel array contains large noise in the displacement direction. The reference displacement is decomposed into a first reference vector and a second reference vector to determine the row pixels and column pixels that are blurred due to the target object moving too fast.

[0066] Step S130: preprocessing the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array.

[0067] In this embodiment, the displacement of the target object within the frame interval between the second pixel array and the third pixel array is determined as the target displacement. It can be understood that due to the continuity of the motion, the speed and direction of the target object will not change suddenly during the motion. Therefore, the second pixel array and the third pixel array can be pre-processed according to the size and direction of the reference displacement.

[0068] Specifically, a first reference component and a second reference component can be used according to the reference displacement, and part of the row pixels and part of the column pixels on the pixel array are discarded according to the first reference component and the second reference component, wherein the first reference component is used to determine the number of discarded row pixels, and the second reference component is used to determine the number of discarded column pixels.

[0069] In one embodiment, before preprocessing the second pixel array according to the first reference component and the second reference component, the method further includes:

[0070] According to the first reference component, the column discard value P is determined, and the calculation formula is:

[0071] P = [X + 0.5]

[0072] According to the second reference component, the row discard value Q is determined, and the calculation formula is:

[0073] Q = [Y + 0.5]

[0074] Wherein, X represents the value corresponding to the first reference component, and Y represents the value corresponding to the second reference component.

[0075] In this embodiment, [X+0.5] represents the maximum integer not exceeding X+0.5, and [Y+0.5] represents the maximum integer not exceeding Y+0.5. Taking the first reference component as 0.5 pixels and the second reference component as 1.2 pixels as an example, P=[0.5+0.5]=1, Q=[1.2+0.5]=1.

[0076] It should be noted that when the target object moves too fast, for the second pixel array, the row pixels and / or column pixels far from the moving direction are prone to blurring, and for the third pixel array, the row pixels and / or column pixels close to the moving direction are prone to blurring. Therefore, preprocessing the second pixel array includes discarding P column pixels far from the moving direction and / or Q row pixels far from the moving direction. Preprocessing the third pixel array includes discarding P column pixels close to the moving direction and / or Q row pixels close to the moving direction.

[0077] In one embodiment, the first pixel array, the second pixel array, and the third pixel array all include N rows×M columns of pixels, and the preprocessing of the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array includes:

[0078] For the second pixel array, if the direction of the first reference component is the same as the first direction axis, and the second reference component is the same as the second direction axis, then discard the pixels in the 1st to Pth columns and the pixels in the 1st to Qth rows to obtain a first target pixel array;

[0079] If the direction of the first reference component is opposite to the first direction axis, and the second reference component is the same as the second direction axis, discarding the pixels in the (MP)th to Mth columns and the pixels in the 1st to Qth rows to obtain a first target pixel array;

[0080] If the first reference component is the same as the first direction axis, and the second reference component is opposite to the second direction axis, discarding the pixels in the 1st to Pth columns and the pixels in the (NQ)th to Nth rows to obtain a first target pixel array;

[0081] If the first reference component is opposite to the first direction axis, and the second reference component is opposite to the second direction axis, pixels in the (MP)th to Mth columns and pixels in the (NQ)th to Nth rows are discarded to obtain a first target pixel array.

[0082] See also Figure 2 , Figure 2 FIG. 1 shows a schematic diagram of an N-row×M-column pixel array. In this embodiment, the pixel array rows are arranged from top to bottom, and the columns are arranged from left to right. Figure 2 Taking the pixel array shown in FIG. 1 as an example, from top to bottom, they are the 1st row, the 2nd row, ..., the Nth row, and from left to right, they are the 1st column, the 2nd column, ..., the Mth column. It should be noted that the first direction axis is the same as the column arrangement direction, and the second direction axis is the same as the row arrangement direction, see Figure 3 , Figure 3 A schematic diagram of the first direction axis and the second direction axis provided in an embodiment of the present application is shown, wherein the direction of the first direction axis is from the first column to the Mth column, and the direction of the second direction axis is from the first row to the Nth row.

[0083] In this embodiment, if the first reference component is the same as the first direction axis, it means that the movement direction of the target object on the first direction axis is the same as the first direction axis; conversely, if the first reference component is opposite to the first direction axis, it means that the movement direction of the target object on the first direction axis is opposite to the first direction axis; if the second reference component is the same as the second direction axis, it means that the movement direction of the target object on the second direction axis is the same as the second direction axis; conversely, if the second reference component is opposite to the second direction axis, it means that the movement direction of the target object on the second direction axis is opposite to the second direction axis.

[0084] It should be noted that, for the second pixel array, row pixels and / or column pixels far from the motion direction are prone to blurring, and P column pixels far from the motion direction and / or Q row pixels far from the motion direction need to be discarded. For the third pixel array, row pixels and / or column pixels close to the motion direction are prone to blurring, and P column pixels close to the motion direction and / or Q row pixels close to the motion direction need to be discarded.

[0085] Step S140: pre-processing the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array.

[0086] As described above, pre-processing the third pixel array includes discarding P columns of pixels close to the moving direction and / or Q rows of pixels close to the moving direction.

[0087] In one embodiment, the first pixel array, the second pixel array, and the third pixel array all include N rows×M columns of pixels, and the preprocessing of the third pixel array according to the first reference component and the second reference component to obtain the second target pixel array includes:

[0088] For the third pixel array, if the direction of the first reference component is the same as the first direction axis, and the second reference component is the same as the second direction axis, then discarding the (MP)th to Mth column pixels and the (NQ)th to Nth row pixels to obtain a second target pixel array;

[0089] If the direction of the first reference component is opposite to the first direction axis, and the second reference component is the same as the second direction axis, discarding the pixels in the 1st to Pth columns and the pixels in the (NQ)th to Nth rows to obtain a second target pixel array;

[0090] If the first reference component is the same as the first direction axis, and the second reference component is opposite to the second direction axis, discarding the (NQ)th to Nth rows of pixels and the 1st to Qth rows of pixels to obtain a second target pixel array;

[0091] If the first reference component is opposite to the first direction axis, and the second reference component is opposite to the second direction axis, the pixels in the 1st to Pth columns and the pixels in the 1st to Qth rows are discarded to obtain a second target pixel array.

[0092] See also Figure 4 , Figure 4 A schematic diagram of a movement of a pixel array corresponding to a target object provided by an embodiment of the present application is shown. Specifically, Figure 4 The left side is the first pixel array corresponding to the target object before the reference displacement occurs. Figure 4 The right side is the second pixel array corresponding to the target object after the reference displacement occurs. Figure 4 The reference displacement is 3 pixels, the first reference component of the reference displacement in the first direction axis is 3 pixels, and the second reference component in the second direction axis is 0 pixel.

[0093] Based on this, when determining the target displacement of the target object, that is, the displacement from the second pixel array to the third pixel array, the second pixel array needs to be processed as follows: discard the pixels in the 1st to 3rd columns of the second pixel array, and the third pixel array needs to be processed as follows: discard the pixels in the (N-3)th to Nth columns. The column discard value is determined according to the formula P=[X+0.5], where X=3.

[0094] Step S150, determining an initial displacement of the target object within a target time according to the first target pixel array and the second target pixel array, where the target time is a frame interval between the second pixel array and the third pixel array.

[0095] The first target pixel array and the second target pixel array are calculated by an optical flow algorithm to obtain an initial displacement of the target object within the target time. Since the first target pixel array and the second target pixel array both discard some row pixels and / or column pixels, the processing described in step S160 needs to be performed after obtaining the initial displacement.

[0096] Step S160: determining a target displacement of the target object within the target time according to the initial displacement, the first reference component and the second reference component.

[0097] In one embodiment, see Figure 5 , step S160 includes steps S161 to S163.

[0098] Step S161, determining a first initial component of the initial displacement in the first direction axis and a second initial component of the initial displacement in the second direction axis.

[0099] Since the initial displacement is obtained based on the first target pixel array and the second target pixel array by discarding some row pixels and / or column pixels, after obtaining the initial displacement, it is necessary to add the discarded pixel values ​​to the initial displacement. For details, please refer to steps S162 and S163.

[0100] Step S162: determining a first target component according to the first reference component and the first initial component, and determining a second target component according to the second reference component and the second initial component.

[0101] It can be understood that the initial displacement is a vector, which is decomposed into a first initial component on the first direction axis and a second initial component on the second direction axis, and then added to the discarded pixel values ​​on the corresponding direction axis to obtain the first target component and the second target component.

[0102] Step S163: determining the target displacement of the target object within the target time according to the first target component and the second target component.

[0103] The vector sum of the first target component and the second target component is the target displacement of the target object within the target time.

[0104] It can be understood that compared with the existing technology that directly calculates displacement through the pixel arrays corresponding to the target object before and after movement, the embodiments of the present invention avoid the influence of blurred pixels in the pixel array in the direction of movement caused by the object's excessive speed, thereby improving the accuracy of displacement determination.

[0105] In one embodiment, see Figure 6 , the method also includes steps S610 to S630.

[0106] Step S610, determining whether the directions of the target displacement and the reference displacement are consistent, if they are consistent, the counter value is increased by a first preset number of times.

[0107] In the displacement determination method provided in the embodiment of the present application, the previous displacement result will affect the next displacement result. Therefore, in order to avoid avalanche-like errors, a gatekeeping mechanism needs to be added. The gatekeeping mechanism is specifically: there is a counter. When the direction of the target displacement is inconsistent with the direction of the reference displacement, the counter decreases by a second preset number, such as 2; when the direction of the target displacement is consistent with the direction of the reference displacement, the counter increases by a first preset number, such as 1. After a series of operations, if the value of the counter is equal to the preset value, it is considered that the displacement determination is wrong, and the displacement determination is restarted.

[0108] Step S620: If they are inconsistent, the counter value is reduced by a second preset number of times, and the first preset number of times is less than the second preset number of times.

[0109] It is understandable that if the actual direction of motion starts to change rapidly and the counter does not drop significantly, the algorithm will not realize that a re-prediction is needed. By adjusting the reward and penalty mechanism, the penalty for inconsistency is greater than the reward for consistency, i.e., the second preset number is greater than the first preset number. In this way, even if the direction change is not immediate, the algorithm will realize that a re-prediction is needed more quickly.

[0110] Step S630: when the counter value is equal to the preset value, the target displacement is re-determined.

[0111] It should be noted that the initial value of the counter can be set according to actual needs, such as 0XF. To avoid avalanche errors, when the value of the counter is equal to a preset value, such as 0, the target displacement is re-determined.

[0112] The embodiment of the present invention provides a displacement determination method without increasing the amount of calculation and without making excessive requirements on the pixel array. The method comprises: obtaining a plurality of pixel arrays adjacent to each other in frames from a continuous multi-frame image, wherein the plurality of pixel arrays are, in order of frames, a first pixel array, a second pixel array and a third pixel array; determining a reference displacement according to the first pixel array and the second pixel array, and determining a first reference component and a second reference component according to the reference displacement; preprocessing the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array; preprocessing the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array; determining an initial displacement of the target object within a target time according to the first target pixel array and the second target pixel array, wherein the target time is a frame interval time between the second pixel array and the third pixel array; determining a target displacement of the target object within the target time according to the initial displacement, the first reference component and the second reference component. The present invention improves the accuracy of displacement recognition when the object moves too fast and the displacement distance is large.

[0113] Example 2

[0114] Also, see Figure 7 The embodiment of the present invention further provides a displacement determination device 700, the device comprising:

[0115] An acquisition module 710 is used to acquire a plurality of pixel arrays with adjacent frames from a plurality of continuous frames of images, wherein the plurality of pixel arrays are, in order of frames: a first pixel array, a second pixel array, and a third pixel array;

[0116] A first determination module 720, configured to determine a reference displacement according to the first pixel array and the second pixel array, and determine a first reference component and a second reference component according to the reference displacement;

[0117] A preprocessing module 730 is used to preprocess the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array; and preprocess the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array;

[0118] A second determination module 740 is used to determine the initial displacement of the target object within a target time according to the first target pixel array and the second target pixel array, where the target time is a frame interval between the second pixel array and the third pixel array;

[0119] The third determination module 750 is configured to determine a target displacement of the target object within the target time according to the initial displacement, the first reference component and the second reference component.

[0120] The displacement determination device provided by the embodiment of the present invention obtains a plurality of pixel arrays adjacent to each other in frames from a continuous plurality of frames of images, wherein the plurality of pixel arrays are, in order of frames: a first pixel array, a second pixel array, and a third pixel array; determines a reference displacement according to the first pixel array and the second pixel array, and determines a first reference component and a second reference component according to the reference displacement; pre-processes the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array; pre-processes the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array; determines an initial displacement of the target object within a target time according to the first target pixel array and the second target pixel array, wherein the target time is a frame interval time between the second pixel array and the third pixel array; determines a target displacement of the target object within the target time according to the initial displacement, the first reference component, and the second reference component, thereby improving the accuracy of displacement recognition when the object moves too fast and the displacement distance is large.

[0121] The displacement determination device provided in the embodiment of the present invention is used to execute the displacement determination method described in the above-mentioned embodiment 1, and will not be described again here to avoid repetition.

[0122] Example 3

[0123] In addition, an embodiment of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program runs on the processor, the displacement determination method provided in Embodiment 1 is executed.

[0124] For details, see Figure 8The electronic device 800 includes: a transceiver 801, a bus interface and a processor 802, wherein the processor 802 is used to obtain a plurality of pixel arrays adjacent to each other in frames from a plurality of continuous frames of images, wherein the plurality of pixel arrays are, in order of frames, a first pixel array, a second pixel array and a third pixel array; determine a reference displacement according to the first pixel array and the second pixel array, and determine a first reference component and a second reference component according to the reference displacement; pre-process the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array; pre-process the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array; determine an initial displacement of the target object within a target time according to the first target pixel array and the second target pixel array, wherein the target time is a frame interval between the second pixel array and the third pixel array; determine a target displacement of the target object within the target time according to the initial displacement, the first reference component and the second reference component.

[0125] In the embodiment of the present invention, the electronic device 800 further includes a memory 803. Figure 8 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 802 and memory represented by memory 803. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 801 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. The processor 802 is responsible for managing the bus architecture and general processing, and the memory 803 may store data used by the processor 802 when performing operations.

[0126] The electronic device 800 provided in the embodiment of the present invention can execute the displacement determination method provided in the above method embodiment 1, which will not be described again here to avoid repetition.

[0127] Example 4

[0128] In addition, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the displacement determination method provided in Embodiment 1 is implemented.

[0129] In this embodiment, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0130] The computer-readable storage medium provided in this embodiment can implement the displacement determination method provided in Embodiment 1, and will not be described again here to avoid repetition.

[0131] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.

[0132] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0133] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A displacement determination method, characterized in that: The method comprises: Acquire a plurality of pixel arrays with adjacent frames from a plurality of continuous frames of images, wherein the plurality of pixel arrays are, in order of frames: a first pixel array, a second pixel array, and a third pixel array; Determine a reference displacement according to the first pixel array and the second pixel array, and determine a first reference component and a second reference component according to the reference displacement; Preprocessing the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array; Preprocessing the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array; Determining an initial displacement of a target object within a target time according to the first target pixel array and the second target pixel array, wherein the target time is a frame interval between the second pixel array and the third pixel array; A target displacement of the target object within the target time is determined according to the initial displacement, the first reference component, and the second reference component.

2. The displacement determination method according to claim 1, characterized in that: The determining of the first reference component and the second reference component according to the reference displacement comprises: Determine a component of the reference displacement in a first direction axis as a first reference component, wherein the first direction axis is the same as a column arrangement direction of the first pixel array; A component of the reference displacement in a second direction axis is determined as a second reference component, where the second direction axis is the same as a row arrangement direction of the first pixel array.

3. The displacement determination method according to claim 2, characterized in that: Before preprocessing the second pixel array according to the first reference component and the second reference component, the method further includes: According to the first reference component, the column discard value P is determined, and the calculation formula is: P = [X + 0.5] According to the second reference component, the row discard value Q is determined, and the calculation formula is: Q = [Y + 0.5] Wherein, X represents the value corresponding to the first reference component, and Y represents the value corresponding to the second reference component.

4. The displacement determination method according to claim 3, characterized in that: The first pixel array, the second pixel array, and the third pixel array all include N rows×M columns of pixels, and the second pixel array is preprocessed according to the first reference component and the second reference component to obtain a first target pixel array, including: For the second pixel array, if the direction of the first reference component is the same as the first direction axis, and the second reference component is the same as the second direction axis, then discard the pixels in the 1st to Pth columns and the pixels in the 1st to Qth rows to obtain a first target pixel array; If the direction of the first reference component is opposite to the first direction axis, and the second reference component is the same as the second direction axis, discarding the pixels in the (MP)th to Mth columns and the pixels in the 1st to Qth rows to obtain a first target pixel array; If the first reference component is the same as the first direction axis, and the second reference component is opposite to the second direction axis, discarding the pixels in the 1st to Pth columns and the pixels in the (NQ)th to Nth rows to obtain a first target pixel array; If the first reference component is opposite to the first direction axis, and the second reference component is opposite to the second direction axis, pixels in the (MP)th to Mth columns and pixels in the (NQ)th to Nth rows are discarded to obtain a first target pixel array.

5. The displacement determination method according to claim 3, characterized in that: The first pixel array, the second pixel array, and the third pixel array all include N rows×M columns of pixels, and the preprocessing of the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array includes: For the third pixel array, if the direction of the first reference component is the same as the first direction axis, and the second reference component is the same as the second direction axis, then discarding the (MP)th to Mth column pixels and the (NQ)th to Nth row pixels to obtain a second target pixel array; If the direction of the first reference component is opposite to the first direction axis, and the second reference component is the same as the second direction axis, discarding the pixels in the 1st to Pth columns and the pixels in the (NQ)th to Nth rows to obtain a second target pixel array; If the first reference component is the same as the first direction axis, and the second reference component is opposite to the second direction axis, discarding the (NQ)th to Nth rows of pixels and the 1st to Qth rows of pixels to obtain a second target pixel array; If the first reference component is opposite to the first direction axis, and the second reference component is opposite to the second direction axis, the pixels in the 1st to Pth columns and the pixels in the 1st to Qth rows are discarded to obtain a second target pixel array.

6. The displacement determination method according to claim 5, characterized in that: The step of determining the target displacement of the target object within a target time according to the initial displacement, the first reference component, and the second reference component includes: Determine a first initial component of the initial displacement on the first direction axis and a second initial component of the initial displacement on the second direction axis; Determine a first target component according to the first reference component and the first initial component, and determine a second target component according to the second reference component and the second initial component; A target displacement of the target object within the target time is determined according to the first target component and the second target component.

7. The displacement determination method according to any one of claims 1 to 6, characterized in that: The method further comprises: Determine whether the directions of the target displacement and the reference displacement are consistent, and if they are consistent, increase the counter value by a first preset number of times; If they are inconsistent, the counter value is reduced by a second preset number of times, and the first preset number of times is less than the second preset number of times; When the counter value is equal to a preset value, the target displacement is re-determined.

8. A displacement determination device, characterized in that: The device comprises: An acquisition module, used for acquiring a plurality of pixel arrays with adjacent frames from a plurality of continuous frames of images, wherein the plurality of pixel arrays are, in order of frames, a first pixel array, a second pixel array, and a third pixel array; A first determining module, configured to determine a reference displacement according to the first pixel array and the second pixel array, and determine a first reference component and a second reference component according to the reference displacement; a preprocessing module, configured to preprocess the second pixel array according to the first reference component and the second reference component to obtain a first target pixel array; and preprocess the third pixel array according to the first reference component and the second reference component to obtain a second target pixel array; A second determination module, configured to determine an initial displacement of the target object within a target time according to the first target pixel array and the second target pixel array, wherein the target time is a frame interval between the second pixel array and the third pixel array; A third determination module is configured to determine a target displacement of the target object within the target time according to the initial displacement, the first reference component and the second reference component.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is run on the processor, the displacement determination method according to any one of claims 1 to 7 is executed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the displacement determination method according to any one of claims 1 to 7 is implemented.

Images