A method and device for controlling cursor movement of intelligent interactive display device

Through the cursor movement control method of the intelligent interactive display device, nonlinear calculation and rounding processing are used to solve the problem of cursor jitter and inaccurate positioning in gesture control, and the precise movement and high-precision control of the cursor are achieved.

CN114153349BActive Publication Date: 2024-11-12SHENZHEN HONGHE INNOVATION INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111413979.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-25
Publication Date
2024-11-12
Estimated Expiration
2041-11-25

AI Technical Summary

Technical Problem

In the prior art, when the cursor is controlled to move through gestures, the cursor is prone to jitter and cannot be accurately positioned, resulting in low accuracy.

Method used

By adopting a cursor movement control method of an intelligent interactive display device, the coordinate increment of the user control part is calculated by acquiring the target image frame of the user control part collected by the image acquisition device, and the coordinate increment of the cursor on the display screen is calculated based on this increment. The amount is used to filter out jitter using nonlinear calculation methods (such as α*Δx+β*Δx2+γ*Δx3) and perform rounding calculations when necessary to achieve accurate cursor movement.

Benefits of technology

The precise movement of the cursor to any position is achieved, the jitter in gesture control is reduced, the smoothness and accuracy of the cursor movement is improved, and the positioning accuracy at a single pixel level can be achieved when needed.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114153349B_ABST
    Figure CN114153349B_ABST
Patent Text Reader

Abstract

The present invention discloses a cursor movement control method for an intelligent interactive display device, comprising: acquiring a target image frame of a user-controlled part; calculating the current coordinates (x, y) of the user-controlled part; subtracting the current coordinates from the coordinates at the previous moment to obtain a coordinate increment (Δx, Δy); calculating a coordinate increment (Dx, Dy) of a cursor on a display screen from the coordinate increment (Δx, Δy); when the absolute value of either of the coordinate increments Dx and Dy of the cursor is greater than or equal to a coordinate increment threshold T, moving the cursor from a current position (X, Y) to (X+Dx, Y+Dy); otherwise, accumulating the coordinate increments of the cursor; when the absolute value of either of the accumulated values ​​is greater than or equal to the coordinate increment threshold T, rounding the accumulated value of the cursor coordinate increment to obtain a cursor coordinate increment integer value DX and / or DY, moving the cursor from the current position (X, Y) to (X+DX, Y+DY), filtering out cursor jitter, and realizing accurate movement of the cursor to any position.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data processing of intelligent interactive display devices, and in particular to a cursor movement control method and device for intelligent interactive display devices. Background Art

[0002] With the development of computer vision technology, people have come up with the idea of ​​using cameras to track human movements and send control instructions to devices through human movements and gestures. This makes it possible to remotely control devices with bare hands within a certain distance without the need for control devices such as remote controls or traditional input and output devices such as mouse and keyboard.

[0003] A very common and very important function is to control the movement of the cursor on the screen through gestures, thereby simulating the behavior of a regular mouse, such as moving the cursor, drawing curves on a whiteboard, etc.

[0004] In the process of controlling the movement of the cursor, the positioning accuracy of the human hand in space is far from comparable to that of the mouse. Under existing technical conditions, the hand positioning obtained by various target detection technologies cannot be very accurate. When the hand moves slowly, if it is directly mapped to the cursor position, the cursor will shake and cannot be accurately positioned. Summary of the invention

[0005] Based on the above situation, the main purpose of the present invention is to provide a cursor movement control method and device for an intelligent interactive display device that can filter out cursor jitter and accurately move the cursor to any position.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0007] A method for controlling cursor movement of an intelligent interactive display device, wherein the intelligent interactive display device comprises an image acquisition device and a display screen, and the control method comprises:

[0008] Acquire a target image frame of the user-controlled part acquired by the image acquisition device;

[0009] Calculating the current coordinates (x, y) of the user-controlled part in the frame image;

[0010] Subtracting the current coordinates (x, y) from the coordinates at the previous moment to obtain a coordinate increment (Δx, Δy) of the user-controlled part; the coordinates at the previous moment are the coordinates of the user-controlled part in the previous adjacent image frame of the target image frame, or the coordinates of the user-controlled part in the previous image frame that is multiple frames away from the target image frame;

[0011] Calculating a coordinate increment (Dx, Dy) of a cursor on the display screen according to a coordinate increment (Δx, Δy) of the user-controlled part;

[0012] If the absolute value of either the coordinate increment Dx or Dy of the cursor is less than a preset coordinate increment threshold T, the coordinate increments Dx and Dy of the cursor are accumulated. When the absolute value of either accumulated value is greater than or equal to the coordinate increment threshold T, the accumulated values ​​of the coordinate increments Dx and / or Dy of the cursor are rounded to obtain the cursor coordinate increment integer values ​​DX and / or DY, and the cursor is moved from the current position (X, Y) to (X+DX, Y+DY).

[0013] In a possible implementation, if the absolute value of either the coordinate increment Dx or Dy of the cursor is greater than or equal to the preset coordinate increment threshold T, the cursor is moved from the current position (X, Y) to (X+Dx, Y+Dy).

[0014] In a possible implementation, the following calculation method is used to calculate the coordinate increment (Dx, Dy) of the cursor on the display screen according to the coordinate increment (Δx, Δy) of the user-controlled part:

[0015] Dx=α*Δx+β*Δx 2 +γ*Δx 3 ,

[0016] Dy=α*Δy+β*Δy 2 +γ*Δy 3 ,

[0017] Among them, α, β, and γ are constants.

[0018] In a possible implementation, the α, β, and γ are determined based on the field of view (FoV) of the image acquisition device, the frames per second (fps) of the image acquisition device, the resolution of the image acquired by the image acquisition device, and the display resolution of the display screen.

[0019] In a possible implementation, the values ​​of α, β, and γ are such that when the coordinate increment Δx or Δy of the user-controlled part is within a jitter threshold, the coordinate increment Dx or Dy of the cursor on the display screen is less than the coordinate increment threshold T.

[0020] In a possible implementation, when the absolute value of the accumulated value of any one is greater than or equal to the coordinate increment threshold T, rounding the accumulated value of the cursor coordinate increment Dx and / or Dy to obtain the cursor coordinate increment integer value DX and / or DY includes:

[0021] If the absolute value of the accumulated value of the cursor coordinate increment Dx is greater than or equal to the coordinate increment threshold T, then:

[0022] The cursor coordinate increment integer value DX=int (the accumulated value of Dx),

[0023] The updated value of the accumulated value of the coordinate increment Dx of the cursor=the accumulated value of the coordinate increment Dx-DX;

[0024] If the accumulated value of the coordinate increment Dy of the cursor is greater than or equal to the coordinate increment threshold T, then:

[0025] The cursor coordinate increment integer value DY=int (accumulated value of Dy);

[0026] The update value of the accumulated value of the coordinate increment Dy of the cursor=the accumulated value of the coordinate increment Dy-DY;

[0027] Among them, the int(accumulated value of Dx) and the int(accumulated value of Dy) are respectively the integer part of the accumulated value of the cursor coordinate increment Dx and the integer part of the accumulated value of the cursor coordinate increment Dy.

[0028] The present invention also provides an intelligent interactive display device, which can adopt the cursor movement control method of the intelligent interactive display device of the present invention to enable a user to control a cursor on a display screen of the intelligent interactive display device.

[0029] The present invention also provides a cursor movement control device for an intelligent interactive display device, wherein the intelligent interactive display device comprises an image acquisition device and a display screen, and the control device comprises:

[0030] An image acquisition module, used to acquire a target image frame of the user-controlled part acquired by the image acquisition device;

[0031] A target detection and positioning module, used to calculate the current coordinates (x, y) of the user-controlled part in the target image frame;

[0032] A coordinate increment calculation module, used for subtracting the current coordinate (x, y) from the coordinate at the previous moment to obtain the coordinate increment (Δx, Δy) of the user-controlled part; and calculating the coordinate increment (Dx, Dy) of the cursor on the display screen according to the coordinate increment (Δx, Δy) of the user-controlled part; wherein the coordinate at the previous moment is the coordinate of the user-controlled part in the previous adjacent image frame of the target image frame, or the coordinate of the user-controlled part in the previous image frame that is separated from the target image frame by multiple frames;

[0033] An execution module, comprising a judgment unit, an accumulation unit, a rounding calculation unit and a cursor moving unit, wherein the judgment unit is used to judge whether the absolute value of any one of the coordinate increments Dx and Dy of the cursor is less than a preset coordinate increment threshold T;

[0034] When the determination result of the determination unit is that the absolute value of either the coordinate increment Dx or Dy of the cursor is less than the preset coordinate increment threshold T, the accumulation unit accumulates the coordinate increment Dx and Dy of the cursor;

[0035] When the judgment unit judges that the absolute value of any one of the accumulated values ​​is greater than or equal to the preset coordinate increment threshold T, the rounding calculation unit can round the accumulated value of the cursor coordinate increment Dx and / or Dy to obtain the cursor coordinate increment integer value DX and / or DY;

[0036] The cursor moving unit moves the cursor from the current position (X, Y) to (X+DX, Y+DY).

[0037] In a possible implementation, when the judgment unit determines that the absolute value of either the coordinate increment Dx and Dy of the cursor is greater than or equal to a preset coordinate increment threshold T, the cursor moving unit moves the cursor from the current position (X, Y) to (X+Dx, Y+Dy).

[0038] In a possible implementation, the coordinate increment calculation module includes a cursor coordinate increment calculation unit, which is used to calculate the coordinate increment (Dx, Dy) of the cursor on the display screen according to the coordinate increment (Δx, Δy) of the user-controlled part using the following calculation method:

[0039] Dx=α*Δx+β*Δx2+γ*Δx3,

[0040] Dy=α*Δy+β*Δy2+γ*Δy3,

[0041] Among them, α, β, and γ are constants.

[0042] In a possible implementation, the rounding calculation unit includes a rounding unit and an updating unit.

[0043] When the determination unit determines that the absolute value of the accumulated value of the coordinate increment Dx of the cursor is greater than or equal to the preset coordinate increment threshold T,

[0044] The rounding unit can calculate the cursor coordinate increment integer value DX=int (the accumulated value of Dx),

[0045] The updating unit can calculate the update value of the accumulated value of the coordinate increment Dx of the cursor = the accumulated value of the coordinate increment Dx - DX;

[0046] When the determination unit determines that the absolute value of the accumulated value of the coordinate increment Dy of the cursor is greater than or equal to the preset coordinate increment threshold T,

[0047] The rounding unit can calculate the cursor coordinate increment integer value DY=int (the accumulated value of Dy),

[0048] The updating unit can calculate the update value of the accumulated value of the coordinate increment Dy of the cursor = the accumulated value of the coordinate increment Dy - DY;

[0049] Among them, the int(accumulated value of Dx) and the int(accumulated value of Dy) are the integer part of the accumulated value of the cursor coordinate increment Dx and the integer part of the accumulated value of the cursor coordinate increment Dy.

[0050] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the cursor movement control method of the intelligent interactive display device of the present invention can be implemented.

[0051] The cursor movement control method and device of the intelligent interactive display device of the present invention can filter out cursor jitter and realize the precise movement of the cursor to any position, making the cursor movement controlled by gesture smoother and more natural and in line with people's habits. When precise positioning is required, it can filter out slight hand jitters. It only needs to make the hand movement speed slow enough to obtain single pixel level positioning accuracy, which can ensure the response of low-speed hand movement. As long as the hand continues to move in a certain direction, the cursor will definitely move in the corresponding direction.

[0052] The intelligent interactive display device of the present invention can ensure that the continuous small movement of the target will not be filtered out as jitter, and can achieve precise fine adjustment of the cursor position.

[0053] Other beneficial effects of the present invention will be explained in the specific implementation manner through the introduction of specific technical features and technical solutions. Through the introduction of these technical features and technical solutions, those skilled in the art should be able to understand the beneficial technical effects brought about by the technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The following will describe the preferred implementation of the cursor movement control method and device of the intelligent interactive display device according to the present invention with reference to the accompanying drawings.

[0055] Figure 1 A flow chart of a cursor movement control method for an intelligent interactive display device according to an embodiment of the present invention;

[0056] Figure 2 A flow chart of a cursor movement control method of an intelligent interactive display device according to another embodiment of the present invention;

[0057] Figure 3A flowchart of cumulative calculation in a cursor movement control method of an intelligent interactive display device according to an embodiment of the present invention;

[0058] Figure 4 A block diagram of a cursor movement control device for an intelligent interactive display device according to an embodiment of the present invention;

[0059] Figure 5 A block diagram of a cursor movement control device for an intelligent interactive display device according to another embodiment of the present invention;

[0060] Figure 6 A block diagram of a target detection and positioning module according to an embodiment of the present invention;

[0061] Figure 7 FIG. 4 is a block diagram of a rounding calculation unit according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] Figure 1 The flowchart of the cursor movement control method of the intelligent interactive display device according to one embodiment of the present invention is as follows. The intelligent interactive display device includes an image acquisition device and a display screen, wherein the control method includes:

[0063] Step 1: Obtain a target image frame of the user-controlled part acquired by an image acquisition device; the target image frame generally refers to each current image frame in the user-controlled part action video;

[0064] Step 2: Calculate the current coordinates (x, y) of the user-controlled part in the target image frame. The user-controlled part is usually a hand, especially a finger, such as the index finger.

[0065] Step 3: Subtract the current coordinate (x, y) from the coordinate at the previous moment to obtain the coordinate increment (Δx, Δy) of the user-controlled part;

[0066] Step 4: Calculate the coordinate increment (Dx, Dy) of the cursor on the display screen according to the coordinate increment (Δx, Δy) of the user-controlled part;

[0067] Step 5: Determine whether the absolute value of either the coordinate increment Dx or Dy of the cursor is less than a preset coordinate increment threshold T;

[0068] If yes, then execute step 6: accumulate the cursor coordinate increments Dx and Dy;

[0069] Step 7: Determine whether the absolute value of any accumulated value is greater than or equal to the coordinate increment threshold T;

[0070] If yes, execute step 8: round the accumulated value of the coordinate increment Dx and / or Dy to obtain the cursor coordinate increment integer value DX and / or DY;

[0071] Step 9: Move the cursor from the current position (X, Y) to (X+DX, Y+DY).

[0072] The cursor movement control method of the intelligent interactive display device of the present invention can filter out the cursor jitter and realize the precise movement of the cursor to any position, making the cursor movement controlled by gestures smoother and more natural and in line with people's habits. When precise positioning is required, in addition to filtering out slight hand jitters, the positioning accuracy at the level of a single pixel can be obtained by simply moving the hand at a low enough speed, which can ensure the response of the hand at a low speed. As long as the hand continues to move in a certain direction, the cursor will surely move in the corresponding direction.

[0073] Usually, before step 2, that is, before calculating the current coordinates (x, y) of the user-controlled part in the image frame, the image frame is preprocessed, such as scaling, enhancement, etc. After step 2, the motion trajectory of the user-controlled part is smoothed to obtain the smoothed current coordinates (x, y) of the user-controlled part. The motion trajectory of the user-controlled part can be smoothed by filtering calculation, such as Kalman filtering.

[0074] In a specific implementation, the coordinates of the previous moment in step 3 may be the coordinates of the user-controlled part in the previous adjacent image frame of the target image frame, or may be the coordinates of the user-controlled part in the previous image frame that is multiple frames apart from the target image frame. This is usually determined based on the image frame acquisition frequency of the image acquisition device and the image frame acquisition frequency required by the specific implementation scheme, and can be determined by a person skilled in the art as required. Usually, when the image frame acquisition frequency of the image acquisition device is low, the sampling rate of the image frames in the above control method may be the same as the image frame acquisition frequency of the image acquisition device. When the image frame acquisition frequency of the image acquisition device is high, the sampling rate of the image frames in the above control method does not have a high requirement, and there may be several interval image frames between the sampled front and back image frames.

[0075] In one embodiment, a deep neural network model can be used to calculate the size and position of the user-controlled part, and the number and position of pixels are used to represent the size and position of the user-controlled part; the deep neural network model also calculates the current coordinates (x, y) of the user-controlled part in the image frame based on the size and position of the user-controlled part. Using a deep neural network model to calculate the coordinates of the current position of the user-controlled part is more accurate than other calculation methods.

[0076] In one embodiment, if Figure 2As shown, the cursor movement control method of the intelligent interactive display device may also include step 10, that is, if the absolute value of either the cursor coordinate increment Dx and Dy is greater than or equal to the preset coordinate increment threshold T, the cursor is moved from the current position (X, Y) to (X+Dx, Y+Dy).

[0077] In one embodiment, the following calculation method can be used to calculate the coordinate increment (Dx, Dy) of the cursor on the display screen according to the coordinate increment (Δx, Δy) of the user-controlled part:

[0078] Dx=α*Δx+β*Δx 2 +γ*Δx 3 ,

[0079] Dy=α*Δy+β*Δy 2 +γ*Δy 3 ,

[0080] Among them, α, β, and γ are constants.

[0081] By using the above calculation method, a nonlinear transformation of the hand movement distance to the cursor movement distance on the screen can be achieved, and the jitter can be filtered out. It can be used to roughly control the movement of the cursor. When the cursor is moved by fast action control, the cursor movement can be further accelerated. The same hand movement distance can expand the cursor movement range, and the cursor movement is smoother and more natural.

[0082] In one embodiment, α, β, and γ may be determined based on the field of view (FoV) of the image acquisition device, the frames per second (fps) of the image acquisition device, the resolution of the image acquired by the image acquisition device, and the display resolution of the display screen.

[0083] In the prior art, the coordinate increment (Dx, Dy) of the cursor on the display screen is calculated according to the coordinate increment (Δx, Δy) of the user-controlled part, and only the constant α value is set, and β and γ are both 0, that is, the coordinate increment of the user-controlled part is directly linearly mapped to the coordinate increment of the cursor on the display screen, and simple scaling is performed, which is prone to cursor jitter and inaccurate positioning problems. The technical solution takes into account the speed factor of the movement of the user-controlled part, thereby adding β and γ factors, as well as the corresponding quadratic and cubic increments of the coordinate increment of the user-controlled part (Δx 2 and Δx 3 , Δy 2 and Δy 3 ) to achieve nonlinear correction rather than simple direct linear mapping. These three values ​​are adjusted based on experience. According to the changes in the high, low and medium speeds of the hand, the corresponding values ​​are obtained through experiments. Ultimately, when the hand moves fast, the cursor moves at one ratio, when the hand moves slowly, the cursor moves at another ratio, and when the hand speed is moderate, the cursor moves at another ratio.

[0084] In order to meet the needs of different users, for example, some users require direct linear mapping. Then different α, β, and γ can be fitted for (Dx, Dy) of different size ranges. For example, within the range of customer needs, only the α value is set, while β and γ are set to 0. In other ranges, α, β, and γ are still set.

[0085] In one embodiment, α, β, and γ are selected so that when the coordinate increment Δx or Δy of the user-controlled part is within the jitter threshold, the coordinate increment Dx or Dy of the cursor on the display screen is less than the coordinate increment threshold T. The jitter threshold is reflected in the jitter range of the hand's spatial coordinates, which may vary slightly from person to person, and is reflected in the positioning accuracy. This value can be determined experimentally. In other words, the selection of α, β, and γ should not make the range of cursor movement too large. When the coordinate increment Δx or Δy of the user-controlled part is within the jitter threshold, the cursor should not move due to the values ​​of α, β, and γ, thereby achieving the effect of filtering jitter.

[0086] In one embodiment, the preset coordinate increment threshold T is usually set to 1 pixel. The selection of this value determines the precision of cursor movement. If T is 1, the precision can be as fine as 1 pixel.

[0087] In one embodiment, if Figure 3 As shown, in step 6-9, when the absolute value of the accumulated value of any one is greater than or equal to the preset coordinate increment threshold T, the accumulated value of the coordinate increment Dx and / or Dy is rounded to obtain the cursor coordinate increment integer value DX and / or DY, which may include:

[0088] Step 71: if the absolute value of the accumulated value of the cursor coordinate increment Dx is greater than or equal to the preset coordinate increment threshold T, then execute step 81: cursor coordinate increment integer value DX=int(accumulated value of Dx); updated value of the accumulated value of the cursor coordinate increment Dx=accumulated value of the coordinate increment Dx-DX;

[0089] Step 72: If the absolute value of the accumulated value of the cursor coordinate increment Dy is greater than or equal to the preset coordinate increment threshold T, then execute step 82: cursor coordinate increment integer value DY=int(accumulated value of Dy); updated value of the accumulated value of the cursor coordinate increment Dy=accumulated value of coordinate increment Dy-DY;

[0090] Among them, int(Dx accumulated value) and int(Dy accumulated value) are the evidence part of the accumulated value of the cursor coordinate increment Dx and the integer part of the accumulated value of the cursor coordinate increment Dy. The updated value of the cursor coordinate increment Dx accumulated value and the updated value of the coordinate increment Dy accumulated value are subsequently replaced by the previous round of coordinate increment Dx accumulated value and the previous round of coordinate increment Dy accumulated value, and then a new round of accumulation starts again.

[0091] When the user's controlled part moves slowly, the coordinate increments Dx and Dy are accumulated and calculated. When the absolute value of the accumulated value is greater than or equal to the preset coordinate increment threshold T, step 9 is executed, i.e., the cursor is moved, so as to ensure that the user can control the cursor even when the action is slow, and the cursor will not be mistaken for jitter and filtered out. At the same time, the residual value after rounding the accumulated value is updated to the updated value of the coordinate increment accumulated value, so as to ensure that the residual value is still recorded, so as to maintain continuous accumulation in the next accumulation calculation, and the residual value of the previous accumulation will not be lost, and the continuity of the cursor movement can also be ensured.

[0092] The present invention also provides an intelligent interactive display device, which uses the cursor movement control method of the intelligent interactive display device of the present invention to enable the user to control the intelligent interactive display device. Since the screen of the intelligent interactive display device is very large, such as a 65-86-inch screen, it is usually controlled by the user's hand. During the user's use, the hand sometimes needs to move slowly or in a small range to control the cursor. In order to prevent the cursor from shaking, the prior art usually ignores the situation of slow or small-range movement of the hand, which will lead to the problem of inaccurate cursor positioning. By adopting the control method of the present invention, it can be ensured that the continuous small movement of the target will not be filtered out as jitter, and the fine adjustment of the cursor position can be achieved. In a specific embodiment, the degree of precision depends on the threshold value T. If T is 1, it can be as fine as 1 pixel.

[0093] Furthermore, during operation, when the hand moves quickly, the cursor usually needs to move a large range of distances. If the direct mapping of the prior art is maintained, then when the hand moves quickly, the mapped cursor movement unit is the same as the cursor movement range of the normal hand speed, and the effect of moving the cursor over a large range of distances cannot be achieved. By adopting the control method of the present invention, when the cursor is moved by fast action control, the cursor movement can be further accelerated, the same hand movement distance can expand the cursor movement range, and the cursor movement is smoother and more natural.

[0094] The present invention also provides a cursor movement control device for an intelligent interactive display device, such as Figure 4 As shown, the intelligent interactive display device of the present invention includes an image acquisition device and a display screen, and the control device includes:

[0095] The image acquisition module 20 is used to acquire a target image frame of the user-controlled body part movement acquired by an image acquisition device.

[0096] The target detection and positioning module 30 is used to calculate the current coordinates (x, y) of the user-controlled part in the target image frame; the user-controlled part is usually the user's hand, especially the finger, such as the index finger.

[0097] The coordinate increment calculation module 40 is used to subtract the current coordinates (x, y) from the coordinates at the previous moment to obtain the coordinate increment (Δx, Δy) of the user-controlled part; and calculate the coordinate increment (Dx, Dy) of the cursor on the display screen based on the coordinate increment (Δx, Δy) of the user-controlled part; wherein the coordinates at the previous moment are the coordinates of the user-controlled part in the previous adjacent image frame of the target image frame, or the coordinates of the user-controlled part in the previous image frame that is multiple frames apart from the target image frame.

[0098] The execution module 70 includes a judgment unit 71, an accumulation unit 72, a rounding calculation unit 73 and a cursor moving unit 74. The judgment unit 71 is used to judge whether the absolute value of any one of the cursor coordinate increments Dx and Dy is less than a preset coordinate increment threshold T; when the judgment unit 71 judges that the absolute value of any one of the cursor coordinate increments Dx and Dy is less than the preset coordinate increment threshold T, the accumulation unit 72 accumulates the cursor coordinate increments Dx and Dy; when the judgment unit 71 judges that the absolute value of any one of the accumulated values ​​is greater than or equal to the preset coordinate increment threshold T, the rounding calculation unit 73 can round the accumulated values ​​of the cursor coordinate increments Dx and / or Dy to obtain the cursor coordinate increment integer values ​​DX and / or DY; the cursor moving unit 74 moves the cursor from the current position (X, Y) to (X+DX, Y+DY).

[0099] The control device of the intelligent interactive display device of the present invention can filter out the cursor jitter and realize the precise movement of the cursor to any position, making the cursor movement controlled by gestures smoother and more natural and in line with people's habits. When precise positioning is required, in addition to filtering out slight hand jitters, it is only necessary to keep the hand moving at a low enough speed to obtain positioning accuracy at the level of a single pixel, which can ensure the response of the hand's slow movement. As long as the hand continues to move in a certain direction, the cursor will surely move in the corresponding direction. The conversion of the hand movement distance to the cursor movement distance on the screen can be realized, and the jitter can be filtered out. It can ensure that the continuous slight movement of the target will not be filtered out as jitter, and fine adjustment of the cursor position can be achieved.

[0100] Usually, if Figure 5 As shown, the control device may further include a preprocessing module 50 and a filtering module 60. The preprocessing module 50 is used to perform preprocessing on the image frame, such as scaling, enhancement and the like. The preprocessed image frame is then output to the target detection and positioning module 30. The filtering module 60 is used to smooth the motion trajectory of the user-controlled part, and obtain the current smoothed coordinates (x, y) of the user-controlled part, which are then output to the coordinate increment calculation module 40. Usually, filtering calculation may be used to smooth the motion trajectory of the user-controlled part, such as Kalman filtering.

[0101] In one embodiment, if Figure 6As shown, the target detection and positioning module 30 may include a deep neural network unit, which includes a user control part calculation unit 31 and a coordinate calculation unit 32. The user control part calculation unit 31 is used to calculate the size and position of the user control part, and the number and position of pixels are used to represent the size and position of the user control part; the coordinate calculation unit 32 calculates the current coordinates (x, y) of the user control part in the image frame according to the size and position of the user control part. The use of a deep neural network unit to calculate the coordinates of the current position of the user control part has a higher accuracy than other calculations.

[0102] In one embodiment, if the determination unit determines that the absolute value of either the coordinate increment Dx or Dy of the cursor is not less than a preset coordinate increment threshold T, the cursor moving unit moves the cursor from the current position (X, Y) to (X+Dx, Y+Dy).

[0103] In one embodiment, the coordinate increment calculation module may include a cursor coordinate increment calculation unit, which is used to calculate the coordinate increment (Dx, Dy) of the cursor on the display screen according to the coordinate increment (Δx, Δy) of the user-controlled part using the following calculation method:

[0104] Dx=α*Δx+β*Δx 2 +γ*Δx 3 ,

[0105] Dy=α*Δy+β*Δy 2 +γ*Δy 3 ,

[0106] Among them, α, β, and γ are constants.

[0107] By using the above calculation method, a nonlinear transformation of the hand movement distance to the cursor movement distance on the screen can be achieved, and the jitter can be filtered out. It can be used to roughly control the movement of the cursor. When the cursor is moved by fast action control, the cursor movement can be further accelerated. The same hand movement distance can expand the cursor movement range, and the cursor movement is smoother and more natural.

[0108] In one embodiment, the preset coordinate increment threshold T is usually set to 1 pixel. The selection of this value determines the precision of cursor movement. If T is 1, the precision can be as fine as 1 pixel.

[0109] In one embodiment, if Figure 7As shown, the rounding calculation unit 73 includes a rounding unit 731 and an updating unit 732. When the judging unit 71 judges that the absolute value of the accumulated value of the cursor coordinate increment Dx is greater than or equal to the coordinate increment threshold T, the rounding unit 731 can calculate the cursor coordinate increment integer value DX=int(accumulated value of Dx), and the updating unit 732 can calculate the updated value of the accumulated value of the cursor coordinate increment Dx=accumulated value of coordinate increment Dx-DX; when the judging unit 71 judges that the absolute value of the accumulated value of the cursor coordinate increment Dy is greater than or equal to the coordinate increment threshold T, the rounding unit 731 can calculate the cursor coordinate increment integer value DY=int(accumulated value of Dy), and the updating unit 732 can calculate the updated value of the accumulated value of the cursor coordinate increment Dy=accumulated value of coordinate increment Dy-DY; wherein, int(accumulated value of Dx) and int(accumulated value of Dy) are the integer part of the accumulated value of the cursor coordinate increment Dx and the integer part of the accumulated value of the cursor coordinate increment Dy.

[0110] When the user's controlled part moves slowly, the coordinate increments Dx and Dy are accumulated and calculated. When the absolute value of the accumulated value is greater than or equal to the coordinate increment threshold T, the cursor is moved, thereby ensuring that the user can control the cursor even when the action is slow, and the cursor will not be mistaken for jitter and filtered out. At the same time, the residual value after rounding the accumulated value is updated to the updated value of the coordinate increment accumulated value, ensuring that the residual value is still recorded, so that continuous accumulation is maintained in the next accumulation calculation, and the residual value of the previous accumulation will not be lost, and the continuity of the cursor movement can also be ensured.

[0111] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the cursor movement control method of the intelligent interactive display device of the present invention can be implemented.

[0112] It should be noted that the use of step numbers (letters or numbers) to refer to certain specific method steps in the present invention is only for the purpose of convenience and brevity of description, and is by no means intended to limit the order of these method steps. Those skilled in the art will appreciate that the order of the relevant method steps should be determined by the technology itself and should not be inappropriately limited by the existence of step numbers.

[0113] Those skilled in the art will appreciate that, without conflict, the above-mentioned preferred solutions can be freely combined and superimposed.

[0114] It should be understood that the above-mentioned embodiments are merely illustrative and not restrictive. Without departing from the basic principles of the present invention, various obvious or equivalent modifications or substitutions that can be made by those skilled in the art to the above-mentioned details will all be included in the scope of the claims of the present invention.

Claims

1. A method for controlling cursor movement of an intelligent interactive display device, wherein the intelligent interactive display device comprises an image acquisition device and a display screen, characterized in that: The control method comprises: Acquire a target image frame of the user-controlled part acquired by the image acquisition device; Calculating the current coordinates (x, y) of the user-controlled part in the target image frame; Subtracting the current coordinates (x, y) from the coordinates at the previous moment to obtain a coordinate increment (Δx, Δy) of the user-controlled part; the coordinates at the previous moment are the coordinates of the user-controlled part in the previous adjacent image frame of the target image frame, or the coordinates of the user-controlled part in the previous image frame that is multiple frames away from the target image frame; Calculating the coordinate increment (Dx, Dy) of the cursor on the display screen according to the coordinate increment (Δx, Δy) of the user-controlled part includes: Dx=α*Δx+β*Δx 2 +γ*Δx 3 , Dy=α*Δy+β*Δy 2 +γ*Δy 3 , Among them, α, β, and γ are constants. The values ​​of α, β, and γ are such that when the coordinate increment Δx or Δy of the user-controlled part is within the jitter threshold, the coordinate increment Dx or Dy of the cursor on the display screen is less than the coordinate increment threshold T; If the absolute value of either the coordinate increment Dx or Dy of the cursor is less than a preset coordinate increment threshold T, the coordinate increments Dx and Dy of the cursor are accumulated. When the absolute value of either accumulated value is greater than or equal to the coordinate increment threshold T, the accumulated values ​​of the coordinate increments Dx and / or Dy of the cursor are rounded to obtain the cursor coordinate increment integer values ​​DX and / or DY, and the cursor is moved from the current position (X, Y) to (X+DX, Y+DY).

2. The cursor movement control method of the intelligent interactive display device according to claim 1, characterized in that: The α, β, and γ are determined according to the field of view (FoV) of the image acquisition device, the frames per second (fps) of the image acquisition device, the resolution of the image acquired by the image acquisition device, and the display resolution of the display screen.

3. The cursor movement control method of the intelligent interactive display device according to any one of claims 1-2, characterized in that: When the absolute value of the accumulated value of any one is greater than or equal to the coordinate increment threshold T, the accumulated value of the cursor coordinate increment Dx and / or Dy is rounded to obtain the cursor coordinate increment integer value DX and / or DY, including: If the absolute value of the accumulated value of the cursor coordinate increment Dx is greater than or equal to the coordinate increment threshold T, then: The cursor coordinate increment integer value DX=int (accumulated value of Dx); The updated value of the accumulated value of the coordinate increment Dx of the cursor=the accumulated value of the coordinate increment Dx-DX; If the absolute value of the accumulated value of the coordinate increment Dy of the cursor is greater than or equal to the coordinate increment threshold T, then: The cursor coordinate increment integer value DY=int (accumulated value of Dy); The update value of the accumulated value of the coordinate increment Dy of the cursor=the accumulated value of the coordinate increment Dy-DY; Among them, the int(accumulated value of Dx) and the int(accumulated value of Dy) are respectively the integer part of the accumulated value of the cursor coordinate increment Dx and the integer part of the accumulated value of the cursor coordinate increment Dy.

4. An intelligent interactive display device, characterized in that: The cursor movement control method of the intelligent interactive display device as described in any one of claims 1 to 3 is adopted to enable a user to control a cursor on a display screen of the intelligent interactive display device.

5. A cursor movement control device for an intelligent interactive display device, the intelligent interactive display device comprising an image acquisition device and a display screen, characterized in that: The control device comprises: An image acquisition module, used to acquire a target image frame of the user-controlled part acquired by the image acquisition device; A target detection and positioning module, used to calculate the current coordinates (x, y) of the user-controlled part in the target image frame; A coordinate increment calculation module, used for subtracting the current coordinate (x, y) from the coordinate at the previous moment to obtain the coordinate increment (Δx, Δy) of the user-controlled part; and calculating the coordinate increment (Dx, Dy) of the cursor on the display screen according to the coordinate increment (Δx, Δy) of the user-controlled part; wherein the coordinate at the previous moment is the coordinate of the user-controlled part in the previous adjacent image frame of the target image frame, or the coordinate of the user-controlled part in the previous image frame that is separated from the target image frame by multiple frames; An execution module, comprising a judgment unit, an accumulation unit, a rounding calculation unit and a cursor moving unit, wherein the judgment unit is used to judge whether the absolute value of any one of the coordinate increments Dx and Dy of the cursor is less than a preset coordinate increment threshold T; When the determination result of the determination unit is that the absolute value of either the coordinate increment Dx or Dy of the cursor is less than the preset coordinate increment threshold T, the accumulation unit accumulates the coordinate increment Dx and Dy of the cursor; When the judgment unit judges that the absolute value of any accumulated value is greater than or equal to the preset coordinate increment threshold T, the rounding calculation unit rounds the accumulated value of the cursor coordinate increment Dx and / or Dy to obtain the cursor coordinate increment integer value DX and / or DY; The cursor moving unit moves the cursor from the current position (X, Y) to (X+DX, Y+DY); The coordinate increment calculation module includes a cursor coordinate increment calculation unit, which is used to calculate the coordinate increment (Dx, Dy) of the cursor on the display screen according to the coordinate increment (Δx, Δy) of the user-controlled part using the following calculation method: Dx=α*Δx+β*Δx2+γ*Δx3, Dy=α*Δy+β*Δy2+γ*Δy3, Among them, α, β, and γ are constants. The values ​​of α, β, and γ are such that when the coordinate increment Δx or Δy of the user-controlled part is within the jitter threshold, the coordinate increment Dx or Dy of the cursor on the display screen is less than the coordinate increment threshold T.

6. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the cursor movement control method of the intelligent interactive display device as described in any one of claims 1-3 can be implemented.