A method and device for rapid hand detection, identification and tracking

Abstract

The present invention relates to a method and device for rapid human hand detection, identification and tracking, comprising: acquiring a detection video; determining the initial position of a skin positioning hand in the detection video according to the two-dimensional Gaussian distribution of the skin YCbCr color space; setting initial particles to track the next position of the skin positioning hand based on the initial position, and correcting it according to the two-dimensional Gaussian distribution of the skin YCbCr color space; determining the human hand contour position of the skin positioning hand according to the Gaussian filter differential formula of the image of the skin positioning hand in different scale spaces, and controlling the mechanical arm to move toward the human hand contour position. The present invention uses the two-dimensional Gaussian distribution of the YCbCr color space for particle filtering, reduces the influence of the particle filter on illumination, and enhances the tracking accuracy. At the same time, the Gaussian filter differential formula in different scale spaces is used to detect the human hand contour position, avoiding complex derivative calculations, and having faster calculation speed, higher efficiency, and better real-time performance.

Description

Technical Field

[0001] The present invention relates to the field of collaborative robots, and in particular to a method and device for rapid human hand detection, recognition and tracking. Background Art

[0002] Target detection and tracking is an important topic in machine vision, and has important applications in many areas such as human-machine interaction, military guidance, video surveillance, robot vision navigation, and medical diagnosis. The interaction channel between collaborative robots and the human body is mainly through the human hand, so human hand recognition and tracking is an important step in the normal operation of collaborative robots.

[0003] In the existing technology, there are two main methods for hand recognition and tracking: one is to use data gloves, but since it is necessary to wear complex data gloves and cumbersome sensors, it affects the user experience, and the data gloves are expensive, so it is not conducive to popularization and use; the other is bare-hand detection and tracking. Although there are many methods for object detection and tracking at home and abroad, they also have various disadvantages, such as sensitivity to the influence of lighting, inaccurate models, and low real-time tracking. Therefore, how to accurately, efficiently and quickly recognize and track human hands is an urgent problem to be solved. Summary of the invention

[0004] In view of this, it is necessary to provide a method and device for rapid hand detection, identification and tracking to solve the problem that the identification and tracking of hands in the prior art are not accurate, efficient and fast enough.

[0005] In order to solve the above technical problems, the present invention provides a method for rapid hand detection, recognition and tracking, comprising:

[0006] Get the detection video;

[0007] Determining the initial position of the skin positioning hand in the detection video according to the two-dimensional Gaussian distribution of the skin YCbCr color space;

[0008] Based on the initial position, initial particles are set to track the next position of the skin positioning hand, and correction is performed according to the two-dimensional Gaussian distribution of the skin YCbCr color space;

[0009] According to the Gaussian filter difference formula of the image of the skin positioning hand in different scale spaces, the human hand contour position of the skin positioning hand is determined, and the mechanical arm is controlled to move to the human hand contour position.

[0010] Further, determining the initial position of the skin positioning hand in the detection video according to the two-dimensional Gaussian distribution of the skin YCbCr color space includes:

[0011] Substituting the YCbCr value of each pixel point in each frame of the detection video into the two-dimensional Gaussian distribution of the skin YCbCr color space, and calculating the corresponding probability value;

[0012] If the probability value meets the preset threshold condition, it belongs to the initial position.

[0013] Furthermore, the determination of the two-dimensional Gaussian distribution of the skin YCbCr color space includes:

[0014] Determine multiple groups of two-dimensional variables according to Cb values ​​and Cr values ​​in YCbCr values ​​of multiple types of skin images;

[0015] Substituting the multiple groups of two-dimensional variables into a two-dimensional Gaussian distribution formula to obtain the color mean and color variance of the skin;

[0016] A two-dimensional Gaussian distribution of the skin YCbCr color space is constructed according to the color mean and the color variance.

[0017] Furthermore, the step of setting initial particles to track the next position of the skin positioning hand based on the initial position includes:

[0018] Setting initial particles in the detection video to perform particle filter tracking;

[0019] Based on the initial position, the initial particle is used to track the next position of the skin positioning hand, wherein the motion equation of the skin positioning hand serves as the system model of the initial particle and the two-dimensional Gaussian distribution of the skin of the skin positioning hand serves as a correction measurement of the initial particle.

[0020] Furthermore, the correction is performed according to the two-dimensional Gaussian distribution of the skin YCbCr color space, including:

[0021] Determine the probability value of each particle according to the two-dimensional Gaussian distribution probability of the YCbCr color space of each particle in the particle filter;

[0022] The correction similarity is determined according to the probability value, and particle resampling is performed on the initial particles to achieve tracking measurement correction.

[0023] Further, the determining of the human hand contour position of the skin positioning hand according to the Gaussian filter difference formula of the image of the skin positioning hand in different scale spaces includes:

[0024] Determine the DoG operator according to the Gaussian filter difference formula of the image of the skin positioning hand in different scale spaces;

[0025] The image of the skin positioning hand is convolved with the DoG operator to determine the human hand contour position of the skin positioning hand.

[0026] Further, determining the DoG operator according to the Gaussian filter difference formula of the image of the skin positioning hand in different scale spaces includes:

[0027] Filtering the image of the skin positioning hand according to the Gaussian filtering formula under the first parameter to obtain a first Gaussian filtering result;

[0028] Filtering the image of the skin positioning hand according to the Gaussian filtering formula under the second parameter to obtain a second Gaussian filtering result;

[0029] The DoG operator is determined according to a difference between the first Gaussian filtering result and the second Gaussian filtering result.

[0030] Further, the convolving the image of the skin positioning hand with the DoG operator to determine the human hand contour position of the skin positioning hand includes:

[0031] Determining the similarity between the human hand in the image of the skin positioning hand and the DoG operator;

[0032] According to the similarity, the human hand contour position of the skin positioning hand is determined.

[0033] Furthermore, controlling the robot arm to move toward the human hand contour position includes:

[0034] Controlling the mechanical arm of the collaborative robot to move toward the outline position of the human hand;

[0035] Determining whether the robotic arm delivers the object to the human hand at the human hand outline position;

[0036] If yes, then end the task;

[0037] If not, the initial position is updated, and the process returns to the step of setting the initial particles to track the next position of the skin positioning hand based on the initial position, and performing correction according to the two-dimensional Gaussian distribution of the skin YCbCr color space.

[0038] The present invention also provides a rapid human hand detection, identification and tracking device, comprising:

[0039] An acquisition unit, used for acquiring a detection video;

[0040] A processing unit, used to determine the initial position of the skin positioning hand in the detection video according to the two-dimensional Gaussian distribution of the skin YCbCr color space; and also used to set initial particles to track the next position of the skin positioning hand based on the initial position, and perform correction according to the two-dimensional Gaussian distribution of the skin YCbCr color space;

[0041] The control unit is used to determine the human hand contour position of the skin positioning hand according to the Gaussian filter difference formula of the image of the skin positioning hand in different scale spaces, and control the mechanical arm to move to the human hand contour position.

[0042] Compared with the prior art, the beneficial effects of the present invention include: first, effectively acquiring the detection video; then, based on the two-dimensional Gaussian distribution of the skin YCbCr color space, the initial position of the human hand is initially located, and the segmentation is effectively performed to ensure the subsequent tracking accuracy; then, based on the initial position, the two-dimensional Gaussian distribution of the YCbCr color space is used for particle filtering to effectively track the next position of the human hand, reduce the influence of the particle filter on the illumination, and enhance the tracking accuracy; finally, using the Gaussian filter differential formula of the skin positioning hand image in different scale spaces, only simple calculations are required to detect the human hand contour position according to the similarity of the DoG operator, and accurate positioning is performed to ensure that the robot arm can accurately and in real time move to the precise human hand contour position. In summary, the present invention uses the two-dimensional Gaussian distribution of the YCbCr color space for particle filtering, reduces the influence of the particle filter on the illumination, and enhances the tracking accuracy. At the same time, the Gaussian filter differential formula in different scale spaces is used to detect the human hand contour position, avoiding complex derivative calculations, and the calculation speed is faster, the efficiency is higher, and the real-time performance is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of a flow chart of an embodiment of a method for rapid hand detection, recognition and tracking provided by the present invention;

[0044] Figure 2 The present invention provides Figure 1 A flow chart of an embodiment of step S102;

[0045] Figure 3 A schematic diagram of a process for determining a two-dimensional Gaussian distribution of a skin YCbCr color space provided by the present invention;

[0046] Figure 4 A schematic diagram of the segmentation result of an embodiment of segmenting a human hand using Gaussian distribution of the skin provided by the present invention;

[0047] Figure 5 The present invention provides Figure 1 A schematic diagram of a flow chart of an embodiment of step S103;

[0048] Figure 6 The present invention provides Figure 1 A schematic diagram of another embodiment of step S103;

[0049] Figure 7 The present invention provides Figure 1 A schematic diagram of a flow chart of an embodiment of step S104;

[0050] Figure 8 The present invention provides Figure 7 A flow chart of an embodiment of step S701;

[0051] Fig. 9 A schematic diagram of a flow chart of an embodiment of a LoG operator and a DoG operator curve provided by the present invention;

[0052] Fig.10 The present invention provides Figure 1 A schematic diagram of a flow chart of an embodiment of step S104;

[0053] Fig.11 A detection schematic diagram of an embodiment of detecting and tracking the back of a hand provided by the present invention;

[0054] Fig.12 A detection schematic diagram of an embodiment of palm detection and tracking provided by the present invention;

[0055] Fig.13 A schematic diagram of the structure of an embodiment of a fast human hand detection, identification and tracking device provided by the present invention;

[0056] Fig.14 A schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0058] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0059] In the description of the present invention, reference to an "embodiment" means that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the described embodiments may be combined with other embodiments.

[0060] The present invention provides a method and device for rapid hand detection, recognition and tracking, which combines Gaussian filter difference formulas in different scale spaces and two-dimensional Gaussian distribution in YCbCr color space, providing a new idea for further improving the accuracy and efficiency of hand recognition and tracking.

[0061] Before describing the embodiments, the following definitions are given for the relevant terms:

[0062] YCbCr value: recorded as YCBCR or Y'CBCR, is a color space, usually used in continuous image processing in movies or digital photography systems. Y' is the brightness (luma) component of the color, while CB and CR are the concentration offset components of blue and red. Y' and Y are different, and Y is the so-called brightness (luminance), which represents the concentration of light and is nonlinear, using gamma correction (gamma correction) encoding processing;

[0063] Particle filtering: The process of approximating the probability density function by finding a set of random samples propagating in the state space, replacing the integral operation with the sample mean, and then obtaining the minimum variance estimate of the system state. These samples are figuratively called "particles", so it is called particle filtering.

[0064] Based on the description of the above technical terms, in the prior art, data gloves are often used for hand tracking detection, but data gloves are expensive, which greatly increases the detection cost and affects the user experience, so they are not widely used; or bare-hand detection and tracking are performed, but a variety of complex operators are often used. The most typical one is the use of the LoG operator (Laplacian of Gaussian) to calculate the extreme value of image features to detect spots on the hands, and the spots reflect the areas in the two-dimensional image that have color differences and grayscale differences with the surrounding colors. The formula of the LoG operator is as follows:

[0065] (1)

[0066] Among them, when the image signal response is the largest, the Laplacian function is equal to 0, that is:

[0067] (2)

[0068] Export ,in, ;

[0069] It should be noted that here r is the radius of the circle, that is, The characteristic spots with a radius of r can be detected. It can be seen that the LoG operator needs to calculate the second-order derivative, which is very time-consuming and inefficient.

[0070] Therefore, existing methods for identifying and tracking human hands have problems such as complex algorithms, high costs, and low accuracy. The present invention aims to propose an efficient and accurate method for fast human hand detection, identification, and tracking to solve the above problems.

[0071] The specific embodiments are described in detail below:

[0072] The embodiment of the present invention provides a method for rapid human hand detection, recognition and tracking. Figure 1 Come and see, Figure 1 The flowchart of an embodiment of the method for rapid hand detection, identification and tracking provided by the present invention includes steps S101 to S104, wherein:

[0073] In step S101, a detection video is obtained;

[0074] In step S102, the initial position of the skin positioning hand in the detection video is determined according to the two-dimensional Gaussian distribution of the skin YCbCr color space;

[0075] In step S103, based on the initial position, initial particles are set to track the next position of the skin positioning hand, and correction is performed according to the two-dimensional Gaussian distribution of the skin YCbCr color space;

[0076] In step S104, the human hand contour position of the skin positioning hand is determined according to the Gaussian filter difference formula of the image of the skin positioning hand in different scale spaces, and the robot arm is controlled to move to the human hand contour position.

[0077] In an embodiment of the present invention, first, the detection video is effectively acquired; then, based on the two-dimensional Gaussian distribution of the skin YCbCr color space, the initial position of the human hand is initially located, and the segmentation is effectively performed to ensure the subsequent tracking accuracy; further, based on the initial position, the two-dimensional Gaussian distribution of the YCbCr color space is used for particle filtering to effectively track the next position of the human hand, reduce the impact of the particle filter on the illumination, and enhance the tracking accuracy; finally, using the Gaussian filter difference formula of the skin positioning hand image in different scale spaces, only simple calculations are required to detect the human hand contour position according to the similarity of the DoG operator, and accurate positioning is performed to ensure that the robotic arm can accurately and real-time move to the precise human hand contour position.

[0078] It should be further explained that the present invention preliminarily detects the human hand by acquiring the two-dimensional Gaussian distribution of the skin YCbCr color space, then tracks the human hand through particle filtering, and finally determines the contour of the human hand by generating relevant operators through the Gaussian filter difference formula of the image of the skin-located hand in different scale spaces, which can effectively overcome the morphological and scale changes of the human hand as a non-rigid object in motion.

[0079] As a preferred embodiment, Figure 2 Come and see, Figure 2 The present invention provides Figure 1 The flowchart of step S102 in the embodiment of the present invention is as follows, step S102 specifically includes step S201 to step S202, wherein:

[0080] In step S201, the YCbCr value of each pixel point in each frame of the detection video is brought into the two-dimensional Gaussian distribution of the skin YCbCr color space to calculate the corresponding probability value;

[0081] In step S202, if the probability value meets the preset threshold condition, it belongs to the initial position.

[0082] In the embodiment of the present invention, the two-dimensional Gaussian distribution of the skin YCbCr color space is obtained to preliminarily detect the human hand, which can effectively determine the preliminary location of the human hand and effectively improve the foreground / background separation accuracy.

[0083] As a preferred embodiment, Figure 3 Come and see, Figure 3 The present invention provides a flow chart for determining the two-dimensional Gaussian distribution of the skin YCbCr color space, which specifically includes steps S301 to S303, wherein:

[0084] In step S301, multiple groups of two-dimensional variables are determined according to Cb values ​​and Cr values ​​in YCbCr values ​​of multiple types of skin images;

[0085] In step S302, the plurality of groups of two-dimensional variables are substituted into a two-dimensional Gaussian distribution formula to obtain the color mean and color variance of the skin;

[0086] In step S303, a two-dimensional Gaussian distribution of the skin YCbCr color space is constructed according to the color mean and the color variance.

[0087] In the embodiment of the present invention, Gaussian distribution is used to describe the skin, and the skin is extracted by probability estimation detection, thereby achieving a more accurate and efficient extraction effect.

[0088] In a specific embodiment of the present invention, Figure 4 Come and see, Figure 4 This is a schematic diagram of the segmentation result of an embodiment of the present invention for segmenting a human hand using a two-dimensional Gaussian distribution of skin. The formula for converting RGB color space to YCbCr color space is as follows:

[0089] (3)

[0090] Among them, Cb and Cr of multiple types of skin images obtained in advance are used to form a two-dimensional variable , substitute into the two-dimensional Gaussian distribution formula to obtain the mean value of skin color And variance ∑:

[0091] Among them, the two-dimensional Gaussian (normal) distribution formula is as follows:

[0092] (4)

[0093] It should be noted that, when detecting the image in the video, the YCbCr value of each pixel is substituted into the above formula for calculation, and the probability value greater than the set threshold is detected as skin. The embodiment of the present invention detects the initial position of the hand by detecting the skin, thereby eliminating the disadvantage of adjusting different thresholds for hands of different skin colors, saving time and achieving better results.

[0094] As a preferred embodiment, Figure 5 Come and see, Figure 5 The present invention provides Figure 1 The flowchart of step S103 in an embodiment of the present invention is shown in FIG. 1 , wherein step S103 specifically includes steps S501 to S502, wherein:

[0095] In step S501, initial particles are set in the detection video to perform particle filter tracking;

[0096] In step S502, based on the initial position, the initial particles are used to track the next position of the skin positioning hand, wherein the motion equation of the skin positioning hand is used as the system model of the initial particles, and the two-dimensional Gaussian distribution of the skin of the skin positioning hand is used as the correction measurement of the initial particles.

[0097] In the embodiment of the present invention, the two-dimensional Gaussian distribution of the skin of the skin positioning hand is used for particle filtering, which reduces the influence of particle filtering on illumination and enhances the tracking accuracy.

[0098] As a preferred embodiment, Figure 6 Come and see, Figure 6 The present invention provides Figure 1 The flowchart of another embodiment of step S103 in step S103 specifically includes steps S601 to S602, wherein:

[0099] In step S601, the probability value of each particle is determined according to the two-dimensional Gaussian distribution probability of the YCbCr color space of the pixel point where each particle in the particle filtering is located;

[0100] In step S602, the correction similarity is determined according to the probability value, and particle resampling is performed on the initial particles to achieve tracking measurement correction.

[0101] In the embodiment of the present invention, the two-dimensional Gaussian distribution probability of the YCbCr color space (see the corresponding formulas (3) to (5) of the above-mentioned embodiment of the skin Gaussian distribution segmentation) estimates the probability of each particle in the particle filter, and resamples the particles to track the target. The advantages of this color space are that it is less affected by brightness changes, and it is a two-dimensional independent distribution, which can better limit the skin color distribution area. The multivariate Gaussian distribution describes the skin more accurately, which improves the robustness of particle tracking.

[0102] As a preferred embodiment, Figure 7 Come and see, Figure 7 The present invention provides Figure 1 The flowchart of step S104 in the embodiment of FIG. 1 further includes steps S701 to S702, wherein:

[0103] In step S701, a DoG operator is determined according to Gaussian filter difference formulas of the skin positioning hand image in different scale spaces;

[0104] In step S702, the image of the skin positioning hand is convolved with the DoG operator to determine the position of the human hand contour of the skin positioning hand.

[0105] In the embodiment of the present invention, the Gaussian filter difference formula in different scale spaces is used to generate the DoG (Differece of Gaussian) operator to detect the hand model, which has a simple calculation process, faster calculation speed, higher efficiency and better real-time performance.

[0106] As a preferred embodiment, Figure 8 Come and see, Figure 8 The present invention provides Figure 7 The flowchart of step S701 in the embodiment of FIG. 1 further includes steps S801 to S803, wherein:

[0107] In step S801, the image of the skin positioning hand is filtered according to the Gaussian filtering formula under the first parameter to obtain a first Gaussian filtering result;

[0108] In step S802, the image of the skin positioning hand is filtered according to the Gaussian filtering formula under the second parameter to obtain a second Gaussian filtering result;

[0109] In step S803, the DoG operator is determined according to the difference between the first Gaussian filtering result and the second Gaussian filtering result.

[0110] In the embodiment of the present invention, the corresponding DoG operator is effectively generated according to the Gaussian filter difference formula in different scale spaces.

[0111] In a specific embodiment of the present invention, the DoG operator is the difference of Gaussian functions. Specifically, in an image, the Gaussian filtering results of the image in different scale spaces are subtracted to obtain a difference image. The expression of the DoG operator is as follows:

[0112] (5)

[0113] For the above Gaussian function The derivative is:

[0114] (6)

[0115] Then the following equation exists:

[0116] (7)

[0117] Therefore, we have:

[0118] (8)

[0119] The best approximation of DoG to LoG can be obtained under the following conditions:

[0120] (9)

[0121] As a preferred embodiment, step S702 specifically includes:

[0122] Determining the similarity between the human hand in the image of the skin positioning hand and the DoG operator;

[0123] According to the similarity, the human hand contour position of the skin positioning hand is determined.

[0124] In the embodiment of the present invention, based on the convolution result of the DoG operator and the image of the skin-positioned hand, the corresponding similarity is determined to perform efficient human hand contour position detection.

[0125] In a specific embodiment of the present invention, Fig. 9 Come and see, Fig. 9 FIG. 1 is a flow chart of an embodiment of a LoG operator and a DoG operator curve provided by the present invention, and a comparison of the function waveforms of the LoG operator and the DoG operator. Fig. 9 As shown, the display is very similar. The present invention uses the Gaussian difference in two different scale spaces to approximate the equivalent of LoG, and can achieve the same effect. Since only the convolution needs to be calculated, and the second-order derivative does not need to be calculated, the calculation speed is faster, the efficiency is higher, and the real-time performance is better. In addition, the DoG proposed by the present invention calculates the extreme values ​​of different scale spaces, so that the target detection is not affected by the scale change.

[0126] As a preferred embodiment, Fig.10 Come and see, Fig.10 The present invention provides Figure 1 The flowchart of step S104 in the embodiment of FIG. 1 further includes steps S1001 to S1004, wherein:

[0127] In step S1001, the robot arm of the collaborative robot is controlled to move toward the outline position of the human hand;

[0128] In step S1002, it is determined whether the robot arm delivers the object to the human hand at the human hand contour position;

[0129] In step S1003, if yes, the task ends;

[0130] In step S1004, if not, the initial position is updated, and the process returns to the step of setting the initial particles to track the next position of the skin positioning hand based on the initial position, and performing correction according to the two-dimensional Gaussian distribution of the skin YCbCr color space.

[0131] In the embodiment of the present invention, the front-end arm of the collaborative robot is effectively controlled to move toward the human hand to determine whether the robot arm hands the object to the human hand. If so, the process ends; otherwise, the process returns to step S103 and repeats the cycle until the process ends.

[0132] Combine the following Fig.11 , Fig.12 Come and see, Fig.11 A detection schematic diagram of an embodiment of detecting and tracking the back of a hand provided by the present invention, Fig.12 The detection schematic diagram of an embodiment of palm detection and tracking provided by the present invention is used to better illustrate the technical solution of the present invention with a specific application example:

[0133] The first step is to pre-train the skin parameters under Gaussian distribution;

[0134] The second step is to input the video, detect the skin through the trained parameters, and initially locate the human hand;

[0135] The third step is to set the initial particles in the video image;

[0136] In the fourth step, a particle filter is used to track the next position of the hand. The equation of motion is used as the system model to predict the particles, and the two-dimensional Gaussian distribution of the skin YCbCr color space is used as the correction measurement for the particle filter.

[0137] Step 5: resampling of particles based on the corrected similarity of the skin;

[0138] In the sixth step, DoG determines the outline of the hand and gives the hand position;

[0139] Step 7: Control the front arm of the collaborative robot to move toward the human hand;

[0140] Step 8: Determine whether the robot arm has handed the object to the person. If so, the process ends. Otherwise, return to step 4 and repeat the cycle until the process ends.

[0141] The embodiment of the present invention also provides a fast human hand detection, identification and tracking device, combined with Fig.13 Come and see, Fig.13 This is a schematic diagram of the structure of an embodiment of a fast hand detection, identification and tracking device provided by the present invention. The fast hand detection, identification and tracking device 1300 includes:

[0142] An acquisition unit 1301 is used to acquire a detection video;

[0143] The processing unit 1302 is used to determine the initial position of the skin positioning hand in the detection video according to the two-dimensional Gaussian distribution of the skin YCbCr color space; and is also used to set the initial particles to track the next position of the skin positioning hand based on the initial position, and perform correction according to the two-dimensional Gaussian distribution of the skin YCbCr color space;

[0144] The control unit 1303 is used to determine the human hand contour position of the skin positioning hand according to the Gaussian filter difference formula of the image of the skin positioning hand in different scale spaces, and control the mechanical arm to move to the human hand contour position.

[0145] The more specific implementation methods of each unit of the fast hand detection, recognition and tracking device can refer to the description of the above-mentioned fast hand detection, recognition and tracking method, and have similar beneficial effects, which will not be repeated here.

[0146] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for rapid human hand detection, recognition and tracking as described above is implemented.

[0147] Generally speaking, the computer instructions for implementing the method of the present invention can be carried by any combination of one or more computer-readable storage media. Non-transitory computer-readable storage media can include any computer-readable media except for the signal itself that is temporarily propagating.

[0148] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0149] Computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages, in particular, Python suitable for neural network computing and platform frameworks based on TensorFlow, PyTorch, etc. can be used. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0150] The embodiment of the present invention further provides an electronic device, Fig.14 Come and see, Fig.14 This is a structural diagram of an embodiment of an electronic device provided by the present invention. The electronic device 1400 includes a processor 1401, a memory 1402, and a computer program stored in the memory 1402 and executable on the processor 1401. When the processor 1401 executes the program, the fast human hand detection, identification and tracking method as described above and / or the fast human hand detection, identification and tracking method as described above are implemented.

[0151] As a preferred embodiment, the electronic device 1400 further includes a display 1403 for displaying that the processor 1401 executes the fast human hand detection, recognition and tracking method as described above.

[0152] Exemplarily, the computer program may be divided into one or more modules / units, one or more modules / units are stored in the memory 1402, and executed by the processor 1401 to complete the present invention. One or more modules / units may be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device 1400. For example, the computer program may be divided into the acquisition unit 1301, the processing unit 1302, and the control unit 1303 in the above embodiment. The specific functions of each unit are as described above, and are not repeated here.

[0153] The electronic device 1400 may be a desktop computer, a notebook, a PDA, a smart phone or the like with an adjustable camera module.

[0154] Among them, the processor 1401 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor 1401 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present invention can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0155] The memory 1402 may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The memory 1402 is used to store programs, and the processor 1401 executes the program after receiving an execution instruction. The process definition method disclosed in any of the embodiments of the present invention may be applied to the processor 1401 or implemented by the processor 1401.

[0156] The display 1403 may be an LCD display screen or an LED display screen, for example, a display screen on a mobile phone.

[0157] Understandably, Fig.14 The structure shown is only a schematic diagram of the structure of the electronic device 1400. The electronic device 1400 may also include Fig.14 More or fewer components as shown. Fig.14 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0158] The computer-readable storage medium and electronic device provided according to the above embodiments of the present invention can be implemented with reference to the specific description of the method for fast hand detection, identification and tracking as described above according to the present invention, and have similar beneficial effects as the method for fast hand detection, identification and tracking as described above, which will not be repeated here.

[0159] The present invention discloses a method and device for rapid human hand detection, recognition and tracking. Firstly, the detection video is effectively acquired; then, based on the two-dimensional Gaussian distribution of the skin YCbCr color space, the initial position of the human hand is initially located, and the segmentation is effectively performed to ensure the subsequent tracking accuracy; further, based on the initial position, the two-dimensional Gaussian distribution of the YCbCr color space is used for particle filtering to effectively track the next position of the human hand, reduce the influence of the particle filtering on the illumination, and enhance the tracking accuracy; finally, the Gaussian filter difference formula of the image of the skin positioning hand in different scale spaces is used, and the human hand contour position can be detected according to the similarity of the DoG operator with only simple calculation, and accurate positioning is performed to ensure that the robot arm can accurately and real-time move to the precise human hand contour position.

[0160] The technical solution of the present invention uses the two-dimensional Gaussian distribution of the YCbCr color space for particle filtering, reduces the impact of particle filtering on illumination, and enhances tracking accuracy. At the same time, the Gaussian filter differential formula in different scale spaces is used to detect the contour position of the human hand, avoiding complex derivative calculations, and achieving faster calculation speed, higher efficiency, and better real-time performance.

[0161] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A fast human hand detection, recognition and tracking method, characterized in that: include: Get the detection video; Determining the initial position of the skin positioning hand in the detection video according to the two-dimensional Gaussian distribution of the skin YCbCr color space; Based on the initial position, initial particles are set to track the next position of the skin positioning hand, and correction is performed according to the two-dimensional Gaussian distribution of the skin YCbCr color space; Determine the human hand contour position of the skin positioning hand according to the Gaussian filter difference formula of the image of the skin positioning hand in different scale spaces, and control the robot arm to move to the human hand contour position; The step of setting initial particles to track the next position of the skin positioning hand based on the initial position includes: Setting initial particles in the detection video to perform particle filter tracking; Based on the initial position, using the initial particle to track the next position of the skin positioning hand, wherein the motion equation of the skin positioning hand is used as the system model of the initial particle, and the two-dimensional Gaussian distribution of the skin color of the skin positioning hand is used as the correction measurement of the initial particle; The correction is performed according to the two-dimensional Gaussian distribution of the skin YCbCr color space, including: Determine the probability value of each particle according to the two-dimensional Gaussian distribution probability of the YCbCr color space of each particle in the particle filter; Determine the correction similarity according to the probability value, perform particle resampling on the initial particles, and implement tracking measurement correction; Determining the contour position of the skin positioning hand according to the Gaussian filter difference formula of the image of the skin positioning hand in different scale spaces includes: Determine the DoG operator according to the Gaussian filter difference formula of the image of the skin positioning hand in different scale spaces; Convolving the image of the skin positioning hand with the DoG operator to determine the human hand contour position of the skin positioning hand; Determining the DoG operator according to the Gaussian filter difference formula of the image of the skin positioning hand in different scale spaces includes: Filtering the image of the skin positioning hand according to the Gaussian filtering formula under the first parameter to obtain a first Gaussian filtering result; Filtering the image of the skin positioning hand according to the Gaussian filtering formula under the second parameter to obtain a second Gaussian filtering result; Determining the DoG operator according to the difference between the first Gaussian filtering result and the second Gaussian filtering result; The step of convolving the image of the skin positioning hand with the DoG operator to determine the contour position of the human hand of the skin positioning hand includes: Determining the similarity between the human hand in the image of the skin positioning hand and the DoG operator; According to the similarity, the human hand contour position of the skin positioning hand is determined.

2. The fast hand detection, recognition and tracking method according to claim 1, characterized in that: The step of determining the initial position of the skin positioning hand in the detection video according to the two-dimensional Gaussian distribution of the skin YCbCr color space includes: Substituting the YCbCr value of each pixel point in each frame of the detection video into the two-dimensional Gaussian distribution of the skin YCbCr color space, and calculating the corresponding probability value; If the probability value meets the preset threshold condition, it belongs to the initial position.

3. The fast human hand detection, recognition and tracking method according to claim 2, characterized in that: The determination of the two-dimensional Gaussian distribution of the skin YCbCr color space includes: Determine multiple groups of two-dimensional variables according to Cb values ​​and Cr values ​​in YCbCr values ​​of multiple types of skin images; Substituting the multiple groups of two-dimensional variables into a two-dimensional Gaussian distribution formula to obtain the color mean and color variance of the skin; A two-dimensional Gaussian distribution of the skin YCbCr color space is constructed according to the color mean and the color variance.

4. The fast human hand detection, recognition and tracking method according to claim 1, characterized in that: The method of controlling the robot arm to move toward the human hand contour position includes: Controlling the mechanical arm of the collaborative robot to move toward the outline position of the human hand; Determining whether the robotic arm delivers the object to the human hand at the human hand outline position; If yes, then end the task; If not, the initial position is updated, and the process returns to the step of setting the initial particles to track the next position of the skin positioning hand based on the initial position, and performing correction according to the two-dimensional Gaussian distribution of the skin YCbCr color space.

5. A fast human hand detection, identification and tracking device, used to execute the fast human hand detection, identification and tracking method according to any one of claims 1 to 4, characterized in that: include: An acquisition unit, used for acquiring a detection video; A processing unit, used to determine the initial position of the skin positioning hand in the detection video according to the two-dimensional Gaussian distribution of the skin YCbCr color space; and also used to set initial particles to track the next position of the skin positioning hand based on the initial position, and perform correction according to the two-dimensional Gaussian distribution of the skin YCbCr color space; The control unit is used to determine the human hand contour position of the skin positioning hand according to the Gaussian filter difference formula of the image of the skin positioning hand in different scale spaces, and control the mechanical arm to move to the human hand contour position.

Images