Method, device and equipment for expressing arc shape of image target object

Abstract

This invention discloses a method, apparatus, and device for representing target objects in images using arc shapes. It generates the center coordinates of local features based on the 3D coordinates of numerous key points in the original dataset file. Then, target key points are selected from the current key points, and an arc shape representing the local features is generated based on these center coordinates and the 3D coordinates of the target key points. This arc shape effectively represents the line contour features of the target object. The target key points are then superimposed onto the target object in the original image and displayed on the interface, allowing users to easily revise the positions of the target key points, thus making the representation of the target object more accurate. This method can correct key points and represent target objects in images using arc shapes, and can accurately label even when the target object is occluded. When the labeled image is used as a training set image and input into a model for training, the training accuracy of the model can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device and equipment for expressing the arc shape of an image target object. Background Art

[0002] With the development of the information age, people have begun to explore the metaverse, and people's interaction with computers is no longer satisfied with traditional input devices such as mice and keyboards. In such an era, the detection of human hands has become particularly important. At present, some teams have conducted research in this regard. At the 2020 IEEE Conference on Computer Vision and Pattern Recognition, the Google team and others published a paper MediaPipeHands:On-device Real-time Hand Tracking, which created a method for detecting palms and then detecting hand skeleton points. This method uses an efficient two-stage hand tracking pipeline, which can track multiple hands in real time on mobile devices, and a hand posture estimation model that only needs RGB input to predict 2.5D hand posture and an open source hand tracking pipeline. The coordinates of its key points of the hand are expressed in a Cartesian coordinate system. However, the method proposed in this paper is applicable to the case where the hand is not occluded. Once the hand is occluded by an object, the detection accuracy is reduced. If this detection result is directly input into the model for training, the trained model has low accuracy. For occluded hand images, the occluded part cannot be predicted well. Summary of the invention

[0003] The purpose of the embodiments of the present invention is to provide a method, device and equipment for expressing the arc shape of a target object in an image, which can correct the key points and express the arc shape of the target object in the image, and can accurately mark the target object even when the target object is occluded. When the marked image is input into the model as a training set image for training, the training accuracy of the model can be improved.

[0004] To achieve the above object, an embodiment of the present invention provides a method for expressing an arc shape of an image target object, comprising:

[0005] Acquire an original data set file; wherein the original data set file includes an original image, three-dimensional coordinates of several key points of a target object in the original image, and camera intrinsic parameters;

[0006] Generating the coordinates of the center of a circle in the target object for characterizing local features according to the three-dimensional coordinates of the key points;

[0007] Acquire the three-dimensional coordinates of several target key points from the three-dimensional coordinates of the key points;

[0008] Generate an arc expression representing the local feature according to the three-dimensional coordinates of the target key point and the center coordinates of the circle;

[0009] Performing a two-dimensional coordinate transformation on the target key points and the arc expression according to the camera internal parameters, and displaying an original image superimposed with the target key points and the arc expression in a display interface;

[0010] A movement instruction of the target key point is received, and the position of the target key point and the arc expression are corrected according to the movement instruction.

[0011] As an improvement of the above solution, the target object is a hand, and the local features are fingers.

[0012] As an improvement of the above scheme, the target key point includes an occluded key point and an unoccluded key point; wherein the occluded key point is a key point when the local feature is occluded; then, after correcting the position of the target key point and the arc expression according to the movement instruction, it also includes:

[0013] The occluded key points, the unoccluded key points and the original image are input as training set data into a pre-built target object recognition model for training.

[0014] As an improvement of the above scheme, the key point is the key point of the local feature, each of the local features corresponds to N key points, N is an integer and N≥3; then, generating the center coordinates of the circle for characterizing the local feature in the target object according to the three-dimensional coordinates of the key point includes:

[0015] Get the three-dimensional coordinates of N key points corresponding to the current local feature;

[0016] Select the three-dimensional coordinates of three key points from the three-dimensional coordinates of N key points to construct a first plane;

[0017] constructing a second plane perpendicular to the first plane;

[0018] A set of equations regarding the coordinates of the center of a circle corresponding to the current local feature is constructed based on the first plane and the second plane, and the coordinates of the center of the circle are solved according to the set of equations.

[0019] As an improvement of the above solution, after displaying the original image superimposed with the target key point and the arc expression in the display interface, it also includes:

[0020] receiving a local display instruction, and displaying a target key point and an arc expression of a corresponding local feature according to the local display instruction;

[0021] A local hiding instruction is received, and target key points and arc expressions of corresponding local features are hidden according to the local hiding instruction.

[0022] As an improvement of the above solution, after displaying the original image superimposed with the target key point and the arc expression in the display interface, it also includes:

[0023] receiving an arc display instruction, and displaying an arc expression of the target object according to the arc display instruction;

[0024] An arc hiding instruction is received, and an arc expression of the target object is hidden according to the arc hiding instruction.

[0025] As an improvement of the above scheme, when the target key points and arc expressions of the target object in the original image are displayed in the display interface, the target key points corresponding to each local feature are displayed in the same color, and the target key points and arc expressions corresponding to different local features are displayed in different colors.

[0026] To achieve the above object, an embodiment of the present invention further provides an arc-shaped expression device for an image target object, comprising:

[0027] The original data set file acquisition module is used to acquire the original data set file; wherein the original data set file includes the original image, the three-dimensional coordinates of several key points of the target object in the original image, and the camera intrinsic parameters;

[0028] A circle center coordinate calculation module, used to generate the circle center coordinates for characterizing local features in the target object according to the three-dimensional coordinates of the key points;

[0029] A target key point acquisition module, used to acquire the three-dimensional coordinates of several target key points from the three-dimensional coordinates of the key points;

[0030] An arc expression generation module, used to generate an arc expression representing the local feature according to the three-dimensional coordinates of the target key point and the center coordinates of the circle;

[0031] A display module, used for performing two-dimensional coordinate transformation on the target key points and the arc expression according to the camera internal parameters, and displaying an original image superimposed with the target key points and the arc expression in a display interface;

[0032] The correction module is used to receive the movement instruction of the target key point and correct the position of the target key point and the arc expression according to the movement instruction.

[0033] As an improvement of the above solution, the target object is a hand, and the local features are fingers.

[0034] To achieve the above-mentioned purpose, an embodiment of the present invention also provides an arc expression device for an image target object, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the arc expression method for the image target object as described in any of the above-mentioned embodiments.

[0035] Compared with the prior art, the arc expression method, device and equipment of the target object in the image disclosed in the embodiment of the present invention generates the center coordinates of the local features based on the three-dimensional coordinates of more key points in the original data set file, and then selects the target key points from the current key points, and generates an arc expression representing the local features based on the center coordinates and the three-dimensional coordinates of the target key points. The arc expression can well represent the line contour features of the target object, and then presents it in the target object of the original image in combination with the target key points, and displays it in the display interface, so that the user can revise the position of the target key points, thereby making the representation of the target object more accurate. It is possible to correct the key points and express the arc of the target object in the image, and accurately mark the target object when it is blocked. When the marked image is input into the model as a training set image for training, the training accuracy of the model can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart of a method for expressing an arc shape of an image target object provided by an embodiment of the present invention;

[0037] Figure 2 is a schematic diagram of an operation interface of a hand model annotation tool provided by an embodiment of the present invention;

[0038] Figure 3 It is a structural block diagram of an arc-shaped expression device for an image target object provided by an embodiment of the present invention;

[0039] Figure 4 It is a structural block diagram of an arc-shaped expression device for an image target object provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] See also Figure 1 , Figure 1: is a flow chart of a method for expressing an arc shape of an image target object provided by an embodiment of the present invention, wherein the method for expressing an arc shape of an image target object comprises:

[0042] S1. Obtaining an original data set file; wherein the original data set file includes an original image, three-dimensional coordinates of several key points of a target object in the original image, and camera intrinsic parameters;

[0043] S2, generating the coordinates of the center of a circle in the target object for characterizing local features according to the three-dimensional coordinates of the key points;

[0044] S3, acquiring the three-dimensional coordinates of several target key points from the three-dimensional coordinates of the key points;

[0045] S4, generating an arc expression representing the local feature according to the three-dimensional coordinates of the target key point and the center coordinates of the circle;

[0046] S5, performing two-dimensional coordinate transformation on the target key points and the arc expression according to the camera internal parameters, and displaying the original image superimposed with the target key points and the arc expression in a display interface;

[0047] S6. Receive a movement instruction of the target key point, and correct the position of the target key point and the arc expression according to the movement instruction.

[0048] It is worth noting that the arc expression method of the target object in the image described in the embodiment of the present invention is implemented by a labeling tool installed on the client, and the labeling tool can be an application. In the embodiment of the present invention, the target object is a hand, and the local feature is a finger.

[0049] Specifically, in step S1, in an embodiment of the present invention, a data set file of a target object in an original image is generated using an existing arc expression method, such as a hand feature image of 19 key points and 2 feature points is generated using a Cartesian coordinate system, wherein the key points are the key points of the local features, and each of the local features corresponds to N key points, where N is an integer and N≥3. That is, the key points are the key points of the fingers. In an embodiment of the present invention, there are 19 key points, namely: except for the thumb, the remaining four thumbs have 4 key points from the finger tip to the bottom (each of the two joints has 1 key point), and the thumb has 3 key points; the 2 feature points are: there is 1 feature point from the bottom of the thumb to the middle of the wrist, and there is 1 feature point at the wrist. Since the 21 points generated at this time are generated by the model, there may be occluded parts that cannot be accurately represented, so the annotation tool provided by the present invention is required to re-annotate the key points of the hand to improve the accuracy. And 2 of the feature points can be reused.

[0050] Specifically, in step S2, an arc expression is constructed based on the current hand skeleton point data set, and a circle center expression of local features is generated based on the three-dimensional coordinate information of the 19 key points of the hand. Then, the step of generating the circle center coordinates for characterizing local features in the target object based on the three-dimensional coordinates of the key points includes the following steps:

[0051] S21, obtaining the three-dimensional coordinates of N key points corresponding to the current local feature;

[0052] S22, selecting the three-dimensional coordinates of three key points from the three-dimensional coordinates of the N key points to construct a first plane;

[0053] S23, constructing a second plane perpendicular to the first plane;

[0054] S24. Construct a group of equations regarding the circle center coordinates corresponding to the current local feature based on the first plane and the second plane, and solve the circle center coordinates according to the group of equations.

[0055] For example, let the coordinates of the key points of a finger from the base to the fingertip be M1: (x1, y1, z1), M2: (x2, y2, z2), M3: (x3, y3, z3), M4: (x4, y4, z4), and the coordinates are expressed in the camera coordinate system. First, select any three points to construct the plane where the finger is located, such as M1, M2, and M3. Here, the point normal form is used to construct the plane. The vectors L1 = M3-M1, L2 = M3-M2, and the vector perpendicular to the plane L = L1×L2 = (A, B, C). The "plane equation" refers to the equation corresponding to all points in the same plane in space. Its general form is Ax+By+Cz+D=0. The point normal form equation of a plane is a form of plane equation. In the spatial rectangular coordinate system, given a point M3: (x3, y3, z3) and a normal vector L on the plane, the expression of the first plane S1 can be determined as: A(X-x3)+B(Y-y3)+C(z-z3)=0.

[0056] Then, construct a second plane S2, which is perpendicular to plane S1 and line M1M4, and is on the perpendicular bisector of line M1M4. Vector n = M4-M1 = (A1, B1, C1), according to the point normal equation, the expression of the second plane S2 can be obtained as: According to the first plane S1 and the second plane S2, we can get the space line {S1, S2}. Finally, we can solve the coordinates of the center of the circle according to the equation group. Let the coordinates of the center of the circle be (c1, c2, c3), which satisfies the following equation group:

[0057]

[0058] The first equation is to bring the center of the circle into the first plane S1. The second and third equations indicate that the radius distance from the center of the circle to the point on the arc is equal. The fourth equation indicates that the center of the circle is on the straight line {s1, s2}. This uses two planes to represent the straight line in space. Based on the above equations, the coordinates of the center of the circle (c1, c2, c3) can be solved.

[0059] Specifically, in step S3, the three-dimensional coordinates of 10 target key points are obtained from the three-dimensional coordinates of the 19 key points. At this time, one finger is represented by two target key points, and the two feature points in the original image can be reused. The two feature points will not generate arc expressions, that is, only fingers have arc expressions. For example, the two target key points of the index finger are the tip of the index finger and the bottom of the index finger. The arc connecting the two target key points is the arc expression of the index finger.

[0060] Specifically, in step S4, after knowing the three-dimensional coordinates of the two target key points in the current finger and the center coordinates of the circle, an arc expression representing the current finger can be generated, and the arc expression is now displayed in three-dimensional form. The two target key points are points on the circle formed by the center coordinates of the circle, and the arc is a line segment of the circle intercepted by the two target key points. It can be understood that when the human hand is straightened, the center coordinates of the circle calculated at this time are far away from the two corresponding target key points, so that the arc formed by connecting the two target key points is close to a straight line, which better represents the straight shape of the finger; when the human hand is bent, because the degree of bending of the finger in reality is similar to the arc, the bending state of the human finger can be well presented by the arc.

[0061] Specifically, in step S5, since the display interface is flat, it is necessary to convert the target key points and arc expressions into two-dimensional coordinates so that the arc expressions can be better presented in the display interface. At this time, the three-dimensional coordinates of the target key points and the three-dimensional coordinates of the arc expressions are converted into two-dimensional coordinates according to the camera internal parameters. After obtaining the target key points and arc expressions of each local feature in the target object, the user can modify the target key points in the annotation tool. At this time, the operation interface of the annotation tool is as follows: Figure 2 As shown in the figure, the operation interface is divided into 4 parts. The upper part of the left half is the dataset image and annotation point display area, the lower part of the left half is a number of operation buttons, the upper part of the right half is for selecting whether the target key points and feature points are visible, and the lower part of the right half includes an arc display box and a search input box. Enter the image id in the input box and click the "OK" button to jump to the last annotated image.

[0062] The display area is used to display the original image superimposed with the target key points and the arc expression, and the display effect is as follows Figure 2 As shown in FIG. 1 , based on the hand in the figure, after the operations of steps S1 to S4, the generated target key points are: thumb target key points a1 and a2, index finger target key points b1 and b2, middle finger target key points c1 and c2, ring finger target key points d1 and d2, little finger target key points e1 and e2, and two reused feature points: wrist point G and thumb 1F, a total of 12 points. The 10 key points have corresponding arc expressions based on the fingers, namely thumb arc expression a1-a2, index finger arc expression b1-b2, middle finger arc expression c1-c2, ring finger arc expression d1-d2, and little finger arc expression e1-e2.

[0063] Furthermore, when the target key points and arc expressions of the target object in the original image are displayed in the display interface, the target key points corresponding to each local feature are displayed in the same color, and the target key points and arc expressions corresponding to different local features are displayed in different colors. Each arc expression is represented by a dotted line of a different color, and the feature points can also be represented by a color different from the target key points. In the embodiment of the present invention, by using different colors to distinguish the target key points of different local features, when there are many overlapping parts between fingers, the feature points representing each finger can be well distinguished without causing visual confusion.

[0064] Specifically, in step S6, the user can use the mouse to move the target key point in the operation interface to correct the position of the target key point. After the target key point is moved, a new arc expression will be regenerated. The corrected image can be input as training set data into a pre-built target object recognition model for training to improve the training accuracy of the model.

[0065] Further, the target key point includes an occluded key point and an unoccluded key point; wherein the occluded key point is a key point when the local feature is occluded; then, after correcting the position of the target key point and the arc expression according to the movement instruction, it also includes:

[0066] The occluded key points, the unoccluded key points and the original image are input as training set data into a pre-built target object recognition model for training.

[0067] Exemplarily, hourglass is used to complete the training of the network, the labeled data set is loaded using the pytorch framework, and the data set is divided into a test set and a validation set. The loaded data includes RGB images, depth images, three-dimensional key points, and two-dimensional key points (including occluded key points and unoccluded key points). The network structure is written, and the existing image recognition network structure can be used to set network parameters, including learning rate, model preservation, etc. The model is trained, the loss curve is observed, and the parameters are modified until the model converges to complete the model training. In an embodiment of the present invention, by inputting the occluded key points into the model, the hand image of the partially occluded fingers can be focused on during the model training process, and the prediction accuracy of the model can be improved. The model can also well predict the contour of the hand image with partially occluded fingers in actual application, thereby accurately displaying the key points and arc expressions of the occluded fingers.

[0068] Furthermore, in an embodiment of the present invention, the operation interface can also display and hide different local features separately, and display and hide arc expressions. Then, after displaying the original image superimposed with the target key points and the arc expressions in the display interface, the steps further include:

[0069] S7, receiving a local display instruction, and displaying a target key point and an arc expression of a corresponding local feature according to the local display instruction;

[0070] S8, receiving a local hiding instruction, and hiding the target key points and arc expressions of the corresponding local features according to the local hiding instruction;

[0071] S9, receiving an arc display instruction, and displaying an arc expression of the target object according to the arc display instruction;

[0072] S10, receiving an arc hiding instruction, and hiding the arc expression of the target object according to the arc hiding instruction.

[0073] For example, Figure 2In the upper part of the middle right half, there is an operation toolbar for target key points and feature points. When selected, the corresponding key points or feature points will be presented. When unselected, the corresponding key points or feature points will be hidden. At the same time, selecting arcsShow can display the corresponding arc expression. For example, if the user needs to observe whether the target key points of the index finger are accurately represented, only the bottom b2 of the index finger and the tip b1 of the index finger can be selected at this time, and the remaining key points and feature points can be unselected. Only the target key points b1 and b2 will be displayed in the display area. When arcsShow is selected, the corresponding arc expression b1-b2 can also be displayed. In an embodiment of the present invention, by displaying the target key points of some local features, it is convenient for users to correct the target key points without being disturbed by other target key points, thereby improving the user's operating experience and making higher correction progress.

[0074] Compared with the prior art, the arc expression method of the target object in the image disclosed in the embodiment of the present invention generates the center coordinates of the local features based on the three-dimensional coordinates of more key points in the original data set file, and then selects the target key points from the current key points, and generates an arc expression representing the local features based on the center coordinates and the three-dimensional coordinates of the target key points. The arc expression can well represent the line contour features of the target object, and then combine the target key points to be superimposed on the target object in the original image and displayed in the display interface, so that the user can revise the position of the target key points, thereby making the representation of the target object more accurate. The target object in the image can be corrected for key points and arc expression, and can be accurately labeled when the target object is blocked. When the labeled image is input into the model as a training set image for training, the training accuracy of the model can be improved.

[0075] See also Figure 3 , Figure 3 1 is a structural block diagram of an arc-shaped expression device 10 for an image target object provided by an embodiment of the present invention. The arc-shaped expression device 10 for an image target object includes:

[0076] The original data set file acquisition module 11 is used to acquire the original data set file; wherein the original data set file includes the original image, the three-dimensional coordinates of several key points of the target object in the original image, and the camera intrinsic parameters;

[0077] A circle center coordinate calculation module 12, used to generate the circle center coordinates for characterizing local features in the target object according to the three-dimensional coordinates of the key points;

[0078] A target key point acquisition module 13 is used to acquire the three-dimensional coordinates of several target key points from the three-dimensional coordinates of the key points;

[0079] An arc expression generating module 14, configured to generate an arc expression representing the local feature according to the three-dimensional coordinates of the target key point and the center coordinates of the circle;

[0080] A display module 15, configured to perform a two-dimensional coordinate transformation on the target key points and the arc expression according to the camera internal parameters, and display an original image superimposed with the target key points and the arc expression in a display interface;

[0081] The correction module 16 is used to receive the movement instruction of the target key point, and correct the position of the target key point and the arc expression according to the movement instruction.

[0082] Specifically, the target object is a hand, and the local features are fingers.

[0083] Specifically, the key point is the key point of the local feature, each of the local features corresponds to N key points, N is an integer and N≥3; then, the circle center coordinate calculation module 12 is used to:

[0084] Get the three-dimensional coordinates of N key points corresponding to the current local feature;

[0085] Select the three-dimensional coordinates of three key points from the three-dimensional coordinates of N key points to construct a first plane;

[0086] constructing a second plane perpendicular to the first plane;

[0087] A set of equations regarding the coordinates of the center of a circle corresponding to the current local feature is constructed based on the first plane and the second plane, and the coordinates of the center of the circle are solved according to the set of equations.

[0088] Specifically, the display module 15 is also used for:

[0089] receiving a local display instruction, and displaying a target key point and an arc expression of a corresponding local feature according to the local display instruction;

[0090] Receiving a local hiding instruction, and hiding a target key point and an arc expression of a corresponding local feature according to the local hiding instruction;

[0091] receiving an arc display instruction, and displaying an arc expression of the target object according to the arc display instruction;

[0092] An arc hiding instruction is received, and an arc expression of the target object is hidden according to the arc hiding instruction.

[0093] Specifically, when the target key points and arc expressions of the target object in the original image are displayed in the display interface, the target key points corresponding to each local feature are displayed in the same color, and the target key points and arc expressions corresponding to different local features are displayed in different colors.

[0094] Further, the target key point includes an occluded key point and an unoccluded key point; wherein the occluded key point is a key point when the local feature is occluded; then, after correcting the position of the target key point and the arc expression according to the movement instruction, the correction module 16 is further used to:

[0095] The occluded key points, the unoccluded key points and the original image are input as training set data into a pre-built target object recognition model for training.

[0096] It is worth noting that the working process of each module in the image target object arc expression device 10 described in the embodiment of the present invention can refer to the working process of the image target object arc expression method described in the above embodiment, and will not be repeated here.

[0097] Compared with the prior art, the arc expression device 10 of the target object in the image disclosed in the embodiment of the present invention generates the center coordinates of the local features based on the three-dimensional coordinates of more key points in the original data set file, and then selects the target key points from the current key points, and generates an arc expression representing the local features based on the center coordinates and the three-dimensional coordinates of the target key points. The arc expression can well represent the line contour features of the target object, and then combine the target key points to be superimposed on the target object in the original image and displayed in the display interface, so that the user can revise the position of the target key points, thereby making the representation of the target object more accurate. The target object in the image can be corrected for key points and arc expression, and can be accurately labeled when the target object is blocked. When the labeled image is input into the model as a training set image for training, the training accuracy of the model can be improved.

[0098] See also Figure 4 , Figure 4 1 is a structural block diagram of an arc expression device 20 for an image target object provided by an embodiment of the present invention, wherein the arc expression device 20 for an image target object comprises a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, the steps in the above-mentioned arc expression method embodiments of the image target object are implemented. Alternatively, when the processor 21 executes the computer program, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0099] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the arc expression device 20 of the image target object.

[0100] The arc expression device 20 of the image target object can be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The arc expression device 20 of the image target object can include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art can understand that the schematic diagram is only an example of the arc expression device 20 of the image target object, and does not constitute a limitation on the arc expression device 20 of the image target object. It can include more or less components than shown in the figure, or combine certain components, or different components. For example, the arc expression device 20 of the image target object can also include input and output devices, network access devices, buses, etc.

[0101] The processor 21 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 21 is the control center of the arc expression device 20 of the image target object, and uses various interfaces and lines to connect various parts of the arc expression device 20 of the entire image target object.

[0102] The memory 22 can be used to store the computer program and / or module. The processor 21 realizes various functions of the arc expression device 20 of the image target object by running or executing the computer program and / or module stored in the memory 22 and calling the data stored in the memory 22. The memory 22 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 22 can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0103] Wherein, if the module / unit integrated in the arc expression device 20 of the image target object is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor 21, the steps of the above-mentioned various method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0104] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0105] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for expressing an arc shape of an image target object, characterized in that: include: Acquire an original data set file; wherein the original data set file includes an original image, three-dimensional coordinates of several key points of a target object in the original image, and camera intrinsic parameters; Generating the coordinates of the center of a circle in the target object for characterizing local features according to the three-dimensional coordinates of the key points; Acquire the three-dimensional coordinates of several target key points from the three-dimensional coordinates of the key points; Generate an arc expression representing the local feature according to the three-dimensional coordinates of the target key point and the center coordinates of the circle; Performing a two-dimensional coordinate transformation on the target key points and the arc expression according to the camera internal parameters, and displaying an original image superimposed with the target key points and the arc expression in a display interface; Receive a movement instruction of the target key point, and correct the position of the target key point and the arc expression according to the movement instruction; wherein the target key point includes an occluded key point and an unoccluded key point, and the occluded key point is a key point when the local feature is occluded; Inputting the occluded key points, the unoccluded key points and the original image as training set data into a pre-built target object recognition model for training; Among them, the key point is the key point of the local feature, each of the local features corresponds to N key points, N is an integer and N≥3; then, generating the center coordinates of the circle used to characterize the local features in the target object according to the three-dimensional coordinates of the key points includes: obtaining the three-dimensional coordinates of N key points corresponding to the current local feature; selecting the three-dimensional coordinates of 3 key points from the three-dimensional coordinates of the N key points to construct a first plane; constructing a second plane perpendicular to the first plane; constructing a group of equations about the center coordinates of the circle corresponding to the current local feature based on the first plane and the second plane, and solving the center coordinates according to the group of equations.

2. The arc expression method of an image target object according to claim 1, characterized in that: The target object is a hand, and the local features are fingers.

3. The arc expression method of an image target object according to claim 1, characterized in that: After the original image with the target key point and the arc expression superimposed thereon is displayed in the display interface, the method further comprises: receiving a local display instruction, and displaying a target key point and an arc expression of a corresponding local feature according to the local display instruction; A local hiding instruction is received, and target key points and arc expressions of corresponding local features are hidden according to the local hiding instruction.

4. The arc expression method of an image target object according to claim 1, characterized in that: After the original image with the target key point and the arc expression superimposed thereon is displayed in the display interface, the method further comprises: receiving an arc display instruction, and displaying an arc expression of the target object according to the arc display instruction; An arc hiding instruction is received, and an arc expression of the target object is hidden according to the arc hiding instruction.

5. The arc-shaped expression method of an image target object according to claim 1, characterized in that: When the target key points and arc expressions of the target object in the original image are displayed in the display interface, the target key points corresponding to each local feature are displayed in the same color, and the target key points and arc expressions corresponding to different local features are displayed in different colors.

6. A device for expressing an arc shape of an image target object, characterized in that: include: The original data set file acquisition module is used to acquire the original data set file; wherein the original data set file includes the original image, the three-dimensional coordinates of several key points of the target object in the original image, and the camera intrinsic parameters; A circle center coordinate calculation module, used to generate the circle center coordinates for characterizing local features in the target object according to the three-dimensional coordinates of the key points; A target key point acquisition module, used to acquire the three-dimensional coordinates of several target key points from the three-dimensional coordinates of the key points; An arc expression generation module, used to generate an arc expression representing the local feature according to the three-dimensional coordinates of the target key point and the center coordinates of the circle; A display module, used for performing two-dimensional coordinate transformation on the target key points and the arc expression according to the camera internal parameters, and displaying an original image superimposed with the target key points and the arc expression in a display interface; A correction module, used for receiving a movement instruction of the target key point, and correcting the position of the target key point and the arc expression according to the movement instruction; wherein the target key point includes an occluded key point and an unoccluded key point, and the occluded key point is a key point when the local feature is occluded; and also used for inputting the occluded key point, the unoccluded key point and the original image as training set data into a pre-built target object recognition model for training; Among them, the key point is the key point of the local feature, each of the local features corresponds to N key points, N is an integer and N≥3; then, the center coordinate calculation module is used to: obtain the three-dimensional coordinates of the N key points corresponding to the current local feature; select the three-dimensional coordinates of 3 key points from the three-dimensional coordinates of the N key points to construct a first plane; construct a second plane perpendicular to the first plane; based on the first plane and the second plane, construct a group of equations about the center coordinates of the current local feature, and solve the center coordinates according to the group of equations.

7. The arc-shaped expression device of an image target object according to claim 6, characterized in that: The target object is a hand, and the local features are fingers.

8. A device for expressing the arc shape of an image target object, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the arc expression method of the image target object as described in any one of claims 1 to 5 when executing the computer program.

Images