Method and device for matching same hand in multi-view camera, computer device and storage medium

Abstract

The application relates to the field of multi-view camera algorithm design, in particular to a same-hand matching method and device in a multi-view camera, computer equipment and a storage medium, the method comprising the following steps: acquiring a first image shot by a first view of the multi-view camera, wherein the first image contains a first hand and a second hand; acquiring a second image shot by a second view, wherein the second image contains a third hand and a fourth hand; performing key point triangulation calculation according to the first hand, the third hand and the fourth hand to obtain a gesture space position of the first hand; performing key point triangulation calculation according to the second hand, the third hand and the fourth hand to obtain a gesture space position of the second hand; calculating an average depth and a re-projection error according to the gesture space position of the first hand and the gesture space position of the second hand; and selecting two hands meeting a constraint condition as the same hands in the multi-view camera according to the average depth and the re-projection error. The application can improve the matching accuracy and efficiency of the same hands in the multi-view camera scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of multi-view camera algorithm design, in particular to a same hand matching method and device in a multi-view camera, a computer device and a storage medium. BACKGROUND

[0002] With the development of imaging technology, virtual reality technology and augmented reality technology have become the mainstream of current research. Gesture interaction plays an indispensable role in these technologies due to its flexibility and convenience. At present, visual-based gesture interaction technology can easily identify the gesture state of a single hand. However, when multiple hands appear in a multi-view camera, the gesture interaction system may misjudge the user's hand, resulting in the inability to accurately match the same hand in the case of multiple hands in the multi-view camera. SUMMARY

[0003] The main purpose of the present application is to provide a same hand matching method and screen projection connection control method, device, computer device and storage medium in a multi-view camera, aiming to solve the problem of low matching accuracy of the same hand in the current multi-view camera scenario.

[0004] In order to achieve the above-mentioned purpose of the application, the present application provides a same hand matching method in a multi-view camera, comprising:

[0005] obtaining a first image shot by a first view of a multi-view camera, the first image containing a first hand and a second hand;

[0006] obtaining a second image shot by a second view of the multi-view camera, the second image containing a third hand and a fourth hand;

[0007] performing a key point triangulation calculation according to the first hand, the third hand and the fourth hand to obtain a gesture space position of the first hand;

[0008] performing a key point triangulation calculation according to the second hand, the third hand and the fourth hand to obtain a gesture space position of the second hand;

[0009] calculating an average depth and a re-projection error according to the gesture space position of the first hand and the gesture space position of the second hand;

[0010] selecting two hands that meet the constraint condition as the same hands in the multi-view camera according to the average depth and the re-projection error.

[0011] The present application also provides a same hand matching device in a multi-view camera, comprising:

[0012] a first image module for obtaining a first image shot by a first view of a multi-view camera, the first image containing a first hand and a second hand;

[0013] a second image module configured to acquire a second image captured by a second view of the multi-view camera, the second image containing a third hand and a fourth hand;

[0014] a first conversion module configured to perform triangulation calculation of key points according to the first hand, the third hand and the fourth hand to obtain a gesture space position of the first hand;

[0015] a second conversion module configured to perform triangulation calculation of key points according to the second hand, the third hand and the fourth hand to obtain a gesture space position of the second hand;

[0016] a depth projection module configured to calculate an average depth and a reprojection error according to the gesture space position of the first hand and the gesture space position of the second hand;

[0017] a constraint condition module configured to select two hands meeting a constraint condition as the same hands in the multi-view camera according to the average depth and the reprojection error.

[0018] The application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the matching method of the same hands in the multi-view camera when executing the computer program.

[0019] The application further provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the matching method of the same hands in the multi-view camera when executed by a processor.

[0020] This application provides a method for matching identical hands in a multi-view camera scene using depth distance and reprojection error. First, a first image captured by the first view of the multi-view camera is acquired, containing a first hand and a second hand. Then, a second image captured by the second view of the multi-view camera is acquired, containing a third hand and a fourth hand. Triangulation calculations of key points are performed based on the first hand, the third hand, and the fourth hand to obtain the gesture spatial position of the first hand. Similarly, triangulation calculations of key points are performed based on the second hand, the third hand, and the fourth hand to obtain the gesture spatial position of the second hand. For each calculated set of gesture spatial positions... The average depth and reprojection error corresponding to the spatial position of each gesture are calculated to obtain the average depth and reprojection error of the hand in the image captured by each lens of the multi-view camera. Based on the average depth and reprojection error, two hands that meet the constraints are selected as the same hand in the multi-view camera. That is, if the average depth and reprojection error meet the constraints, one hand in the first image and one hand in the second image are selected as the same hand in the multi-view camera. In this way, the correspondence of the same hand in the image can be accurately and quickly determined from the images captured by the multi-view camera through spatial position and reprojection error, thereby improving the matching accuracy and efficiency of the same hand in the multi-view camera scene. Attached Figure Description

[0021] Figure 1 This is a schematic flowchart of an embodiment of the matching method for the same hand in a multi-view camera according to this application;

[0022] Figure 2 This is a schematic diagram of an embodiment of the matching device for the same hand in the multi-view camera of this application;

[0023] Figure 3 This is a schematic block diagram of an embodiment of the computer device of this application.

[0024] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0026] Reference Figure 1 This application provides a method for matching identical hands in a multi-view camera. The method includes steps S10-S60, and the detailed description of each step of the method is as follows.

[0027] S10, acquire a first image shot by a first view of a multi-view camera, the first image containing a first hand and a second hand.

[0028] The embodiment is applied to matching of the same hand in the scene of the multi-view camera. In the multi-view camera, multiple cameras are used to shoot images. In the embodiment, the number of views of the multi-view camera is not limited, and the first view and the second view are only used to define different cameras in the multi-view camera. After multiple hands are shot by different views of the multi-view camera, the identity and the corresponding relationship of the hands shot by different views of the multi-view camera need to be determined, so as to match the hands and track the same hand. First, a first image shot by a first view of a multi-view camera is acquired, the first image containing a first hand and a second hand. That is, one of the cameras in the multi-view camera is defined as the first view, and the image shot by the first view contains multiple hands, which are defined as the first hand and the second hand.

[0029] S20, acquire a second image shot by a second view of the multi-view camera, the second image containing a third hand and a fourth hand.

[0030] In the embodiment, after the first image shot by the first view of the multi-view camera is acquired, the second image shot by the second view of the multi-view camera is acquired, the second image containing a third hand and a fourth hand. That is, for other cameras in the multi-view camera, the image shot by the other view is acquired, which is defined as the second image shot by the second view, and the second image also contains multiple hands, which are defined as the third hand and the fourth hand. For example, the first image shot by the first view cam0 contains a first hand hand0 and a second hand hand1, and the second image shot by the second view cam1 contains a third hand hand2 and a fourth hand hand3. At this time, hand0 and hand2 or hand3 need to be matched, or hand1 and hand2 or hand3 need to be matched, so as to track the observation of the same hand under the multi-view camera.

[0031] S30, perform key point triangulation calculation according to the first hand, the third hand, and the fourth hand, to obtain a gesture space position of the first hand.

[0032] In this embodiment, after obtaining a first image captured by a first camera of the multi-camera system, the first image containing a first hand and a second hand, and a second image captured by a second camera of the multi-camera system, the second image containing a third hand and a fourth hand, a key point triangulation calculation is performed according to the first hand, the third hand, and the fourth hand to obtain a gesture space position of the first hand, i.e., a key point triangulation calculation is performed on the first hand hand0 of the first camera cam0 observing hand2 and hand3 of the second camera cam1 to obtain two different gesture space positions XYZ 02 and XYZ 03 , the two different gesture space positions XYZ 02 and XYZ 03 are defined as the gesture space position of the first hand.

[0033] S40, a key point triangulation calculation is performed according to the second hand, the third hand, and the fourth hand to obtain a gesture space position of the second hand.

[0034] In this embodiment, after obtaining a first image captured by a first camera of the multi-camera system, the first image containing a first hand and a second hand, and a second image captured by a second camera of the multi-camera system, the second image containing a third hand and a fourth hand, a key point triangulation calculation is performed according to the second hand, the third hand, and the fourth hand to obtain a gesture space position of the second hand, i.e., a key point triangulation calculation is performed on the second hand hand1 of the first camera cam1 observing hand2 and hand3 of the second camera cam1 to obtain two different gesture space positions XYZ 12 and XYZ 13 , the two different gesture space positions XYZ 12 and XYZ 13 are defined as the gesture space position of the second hand.

[0035] S50, an average depth and a reprojection error are calculated according to the gesture space position of the first hand and the gesture space position of the second hand.

[0036] In this embodiment, after obtaining a gesture space position of the first hand and a gesture space position of the second hand, an average depth and a reprojection error are calculated according to the gesture space position of the first hand and the gesture space position of the second hand. Specifically, for each group of gesture space positions calculated, an average depth and a reprojection error corresponding to the group of gesture space positions are calculated, i.e., for the gesture space positions XYZ 02 and XYZ 03 of the first hand, an average depth and a reprojection error of XYZ 02 are calculated, and an average depth and a reprojection error of XYZ 03the average depth and reprojection error of XYZ 12 and XYZ 13 , the average depth and reprojection error of XYZ 12 and the average depth and reprojection error of XYZ 13 , so as to obtain the average depth and reprojection error of the hand in the image captured by each camera of the multi-camera.

[0037] S60, selecting two hands meeting the constraint condition as the same hand in the multi-camera according to the average depth and reprojection error.

[0038] In the embodiment, after the average depth and reprojection error is calculated according to the gesture space position of the first hand and the gesture space position of the second hand, two hands meeting the constraint condition are selected as the same hand in the multi-camera according to the average depth and reprojection error. Specifically, if the average depth and reprojection error meet the constraint condition, one hand in the first image and one hand in the second image are selected as the same hand in the multi-camera, so that the corresponding relationship of the same hand in the image is accurately and quickly determined from the images captured by the multi-camera through the space position and the reprojection error.

[0039] The embodiment provides a method for matching the same hand in a multi-camera scene through a depth distance and a reprojection error. First, a first image captured by a first camera of a multi-camera is obtained, the first image containing a first hand and a second hand. A second image captured by a second camera of the multi-camera is obtained, the second image containing a third hand and a fourth hand. Triangulation calculation of key points is performed according to the first hand, the third hand and the fourth hand, so as to obtain the gesture space position of the first hand. Triangulation calculation of key points is performed according to the second hand, the third hand and the fourth hand, so as to obtain the gesture space position of the second hand. For each group of gesture space positions obtained, the average depth and reprojection error corresponding to the group of gesture space positions are calculated, so as to obtain the average depth and reprojection error of the hand in the image captured by each camera of the multi-camera. If the average depth and reprojection error meet the constraint condition, one hand in the first image and one hand in the second image are selected as the same hand in the multi-camera, so that the corresponding relationship of the same hand in the image is accurately and quickly determined from the images captured by the multi-camera through the space position and the reprojection error, and the matching accuracy and efficiency of the same hand in the multi-camera scene are improved.

[0040] In one embodiment, the first image captured by the first camera of the multi-camera contains the first hand and the second hand, and the method comprises the following steps.

[0041] obtaining a first image captured by a first camera of the multi-camera system;

[0042] identifying a number of hands contained in the first image;

[0043] if the number of hands is greater than a preset value, performing hand detection on the first image to fuse the extra hands until the number of hands contained in the first image is less than or equal to the preset value.

[0044] In the embodiment, in the process of obtaining a first image captured by a first camera of the multi-camera system, the first image containing a first hand and a second hand, the constraint of only having two hands in the gesture scene is used by default, i.e., only the case of two hands is processed, and when there are extra hands in the first image, the extra hands need to be processed first so that the number of hands in the first image is 2. Specifically, a first image captured by a first camera of the multi-camera system is obtained, the number of hands contained in the first image is identified, and if the number of hands is greater than a preset value, hand detection is performed on the first image to fuse the extra hands until the number of hands contained in the first image is less than or equal to the preset value. Taking a binocular camera as an example: the number of hands observed by the left camera, the camera observing more than two hands is fused by IOU, and the extra hand detection frame is fused to make the number of hands less than or equal to 2, so that different scenes are converted into the same scene for calculation, which can reduce the complexity of subsequent algorithm logic and improve the efficiency of calculation.

[0045] In one embodiment, before the two hands meeting the constraint condition are selected as the same hands in the multi-camera system according to the average depth and the reprojection error, the method further comprises:

[0046] obtaining a natural interaction distance of the gesture;

[0047] configuring the constraint condition of the average depth according to the interaction distance.

[0048] In the embodiment, before the two hands meeting the constraint condition are selected as the same hands in the multi-camera system according to the average depth and the reprojection error, the constraint condition of the average depth and the reprojection error meeting the constraint condition need to be determined, wherein the constraint condition of the average depth is determined by the natural interaction distance of the gesture, i.e., the natural interaction distance of the gesture is obtained, and the constraint condition of the average depth is configured according to the interaction distance, wherein the natural interaction distance of the gesture is determined by big data collection, and in one implementation, the range of the natural interaction distance of the gesture is 100≤Depth avg ≤800, i.e., the average depth needs to be within the range of the natural interaction distance of the gesture to preliminarily confirm that the corresponding two hands match each other, and the accuracy of hand matching is improved by configuring the natural interaction distance of the gesture.

[0049] In one embodiment, the selecting two hands meeting the constraint condition as the same hand in the multi-view camera according to the average depth and the re-projection error comprises:

[0050] selecting the candidate hands whose average depth is within the natural interactive distance of the hand gesture;

[0051] selecting two hands with a re-projection error less than a preset error from the candidate hands as the same hand in the multi-view camera.

[0052] In the embodiment, in the process of selecting two hands meeting the constraint condition as the same hand in the multi-view camera according to the average depth and the re-projection error, the candidate hands whose average depth is within the natural interactive distance of the hand gesture are selected first, that is, the average depth is screened first, and then two hands with a re-projection error less than a preset error are selected from the candidate hands as the same hand in the multi-view camera, which can effectively reduce the calculation amount and improve the matching efficiency of the same hand.

[0053] In one embodiment, the selecting two hands with a re-projection error less than a preset error from the candidate hands as the same hand in the multi-view camera comprises:

[0054] if the number of hands with a re-projection error less than a preset error selected from the candidate hands is greater than two;

[0055] selecting two hands with the minimum re-projection error from the candidate hands as the same hand in the multi-view camera.

[0056] In the embodiment, in the process of selecting two hands with a re-projection error less than a preset error from the candidate hands as the same hand in the multi-view camera, if the number of hands with a re-projection error less than a preset error selected from the candidate hands is greater than two, that is, the number of hands with a re-projection error less than a preset error is greater than two, in order to accurately screen the target hand, two hands with the minimum re-projection error are selected from the candidate hands as the same hand in the multi-view camera, so as to eliminate all false matches and improve the matching accuracy of the same hand.

[0057] In one embodiment, the calculating the average depth and the re-projection error according to the spatial position of the hand gesture of the first hand and the spatial position of the hand gesture of the second hand comprises:

[0058] obtaining an imaging model of the multi-view camera;

[0059] obtaining internal and external parameters of the multi-view camera;

[0060] calculating the epipolar plane position of the image according to the imaging model and the internal and external parameters;

[0061] According to the phase plane position respectively with the gesture space position of the first hand, the gesture space position of the second hand, the re-projection error is calculated.

[0062] In the embodiment, in the process of calculating the average depth and the re-projection error according to the gesture space position of the first hand and the gesture space position of the second hand, first, the imaging model of the multi-view camera is obtained, and the internal and external parameters of the multi-view camera are obtained, then the phase plane position of the image is calculated according to the imaging model and the internal and external parameters, and then the re-projection error is calculated according to the phase plane position respectively with the gesture space position of the first hand, the gesture space position of the second hand, that is, the error of the 3D space point re-projecting back to the phase plane of the left and right view cameras is calculated as the re-projection error. Since the parameters of different multi-view cameras are different, the re-projection error calculated in the scene of different multi-view cameras is closer to the actual scene, thereby improving the matching accuracy of the same hand in the multi-view camera scene.

[0063] In one embodiment, the first hand gesture space position is calculated by triangulation of the key points of the first hand, the third hand and the fourth hand, comprising:

[0064] Obtaining the coordinates of the hand key points corresponding to the first hand, the third hand and the fourth hand;

[0065] According to the coordinates of the hand key points, the first hand gesture space position is calculated by triangulation of the key points of the first hand, the third hand and the fourth hand.

[0066] In the embodiment, in the process of calculating the first hand gesture space position by triangulation of the key points of the first hand, the third hand and the fourth hand, first, the coordinates of the hand key points corresponding to the first hand, the third hand and the fourth hand are obtained, that is, the coordinates of the hand key points of the first hand, the coordinates of the hand key points of the third hand and the coordinates of the hand key points of the fourth hand are obtained, and the hand key points are corresponded, then the first hand gesture space position is calculated by triangulation of the key points of the first hand, the third hand and the fourth hand according to the coordinates of the hand key points. Similarly, the second hand gesture space position is calculated in the same way, thereby obtaining the second hand gesture space position, thereby improving the matching efficiency of the same hand in the multi-view camera scene.

[0067] Referring to Figure 2 The application also provides a multi-view camera same hand matching device, comprising:

[0068] An image receiving module 10 is configured to obtain an input medical image.

[0069] a high-order feature module 20, configured to perform feature extraction on the medical image to obtain a high-order feature map;

[0070] a graph segmentation module 30, configured to divide the high-order feature map into a plurality of grids and construct a convolution kernel for each of the grids;

[0071] a convolution calculation module 40, configured to calculate a convolution result of a feature value of each of the grids and the corresponding convolution kernel;

[0072] a convolution dimension reduction module 50, configured to perform dimension reduction on the convolution result to obtain a convolution vector corresponding to each of the grids;

[0073] a feature vector module 60, configured to splice the convolution vectors to obtain a target vector representing the medical image.

[0074] As described above, it can be understood that each component of the same hand matching device in the multi-view camera provided in the present application can implement the function of any one of the same hand matching methods in the multi-view camera as described above.

[0075] Reference Figure 3 In the embodiments of the present application, a computer device is also provided, which can be a mobile terminal, and the internal structure thereof can be as shown in Figure 3 The computer device includes a processor, a memory, a network interface, a display device and an input device connected through a system bus. The network interface of the computer device is configured to communicate with an external terminal through network connection. The display device of the computer device is configured to display an offline application. The input device of the computer device is configured to receive the input of a user in the offline application. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium. The non-volatile storage medium stores an operating system, a computer program and a database. The database of the computer device is configured to store raw data. The computer program is executed by the processor to implement a same hand matching method in a multi-view camera.

[0076] The processor executes the same hand matching method in the multi-view camera as described above, and the method includes: acquiring an input medical image; performing feature extraction on the medical image to obtain a high-order feature map; dividing the high-order feature map into a plurality of grids and constructing a convolution kernel for each of the grids; calculating a convolution result of a feature value of each of the grids and the corresponding convolution kernel; performing dimension reduction on the convolution result to obtain a convolution vector corresponding to each of the grids; and splicing the convolution vectors to obtain a target vector representing the medical image.

[0077] The computer device provides a method for matching the same hand in a multi-view camera scene through depth distance and re-projection error, which comprises the following steps: obtaining a first image captured by a first view of a multi-view camera, the first image containing a first hand and a second hand; obtaining a second image captured by a second view of the multi-view camera, the second image containing a third hand and a fourth hand; performing a key point triangulation calculation according to the first hand, the third hand and the fourth hand to obtain a gesture space position of the first hand; performing a key point triangulation calculation according to the second hand, the third hand and the fourth hand to obtain a gesture space position of the second hand; calculating the average depth and the re-projection error corresponding to each group of gesture space positions, thereby obtaining the average depth and the re-projection error of the hands in the images captured by each view of the multi-view camera; selecting two hands that meet the constraint conditions as the same hands in the multi-view camera according to the average depth and the re-projection error, i.e., if the average depth and the re-projection error meet the constraint conditions, selecting a hand in the first image and a hand in the second image as the same hands in the multi-view camera, thereby accurately and quickly determining the corresponding relationship of the same hands in the images captured by the multi-view camera through the spatial position and the re-projection error, and improving the matching accuracy and efficiency of the same hands in the multi-view camera scene.

[0078] An embodiment of the present application also provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by the processor to implement a matching method for the same hands in a multi-view camera, comprising the following steps: obtaining an input medical image; performing feature extraction on the medical image to obtain a high-order feature map; dividing the high-order feature map into a plurality of grids and constructing a convolution kernel for each grid; calculating the feature value of each grid and the convolution result of the corresponding convolution kernel; reducing the dimension of the convolution result to obtain a convolution vector corresponding to each grid; and splicing the convolution vectors to obtain a target vector representing the medical image.

[0079] The computer readable storage medium provides a method for matching the same hand in a multi-view camera scene through depth distance and re-projection error. First, a first image captured by a first view of a multi-view camera is obtained, the first image containing a first hand and a second hand. A second image captured by a second view of the multi-view camera is obtained, the second image containing a third hand and a fourth hand. Triangulation calculation of key points is performed according to the first hand, the third hand and the fourth hand, to obtain a gesture space position of the first hand. Triangulation calculation of key points is performed according to the second hand, the third hand and the fourth hand, to obtain a gesture space position of the second hand. For each group of gesture space positions calculated, the average depth and re-projection error corresponding to the group of gesture space positions are calculated, to obtain the average depth and re-projection error of the hands in the images captured by each view of the multi-view camera. According to the average depth and re-projection error, two hands meeting the constraint condition are selected as the same hands in the multi-view camera, that is, if the average depth and re-projection error meet the constraint condition, one hand in the first image and one hand in the second image are selected as the same hands in the multi-view camera. Thus, the corresponding relationship of the same hands in the images captured by the multi-view camera is accurately and quickly determined through the spatial position and re-projection error, and the matching accuracy and efficiency of the same hands in the multi-view camera scene are improved.

[0080] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium provided by the present application and used in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM) and the like.

[0081] It should be noted that, in the present document, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a", "comprising", or "comprises" does not, without further restriction, exclude the existence of additional elements of the process, method, article, or apparatus that comprises the element.

[0082] The preferred embodiments of the present application have been described above with the specific details of the preferred embodiments to provide complete understanding of the application. However, it will be apparent to persons having ordinary skill in the technology that, without departing from the spirit and scope of the application, they can make equivalent changes or modifications to the preferred embodiments. Therefore, the patent protection scope of the present application should be subject to the protection scope of the patent claims.

Claims

1. A method for matching identical hands in a multi-view camera, characterized in that, include: Acquire the first image captured by the first view of the multi-view camera, the first image containing the first hand and the second hand; Acquire a second image captured by the second eye of a multi-view camera, the second image containing the third and fourth hands; Based on the triangulation calculation of key points of the first hand, the third hand, and the fourth hand, the spatial position of the gesture of the first hand is obtained; Based on the triangulation calculation of key points of the second hand, the third hand, and the fourth hand, the spatial position of the gesture of the second hand is obtained; The average depth and reprojection error are calculated based on the spatial position of the gestures of the first hand and the second hand. Based on the average depth and reprojection error, two hands that meet the constraints are selected as the same hand in the multi-view camera. The step of calculating the average depth and reprojection error based on the spatial position of the gestures of the first hand and the second hand includes: Obtain the imaging model of the multi-view camera; Obtain the intrinsic and extrinsic parameters of the multi-view camera; The phase plane position of the image is calculated based on the imaging model and the intrinsic and extrinsic parameters; The reprojection error is calculated based on the phase plane position and the gesture spatial position of the first hand and the gesture spatial position of the second hand, respectively. The reprojection error is the error of reprojecting the 3D spatial point back to the phase plane of the left and right eyes of the multi-view camera. The step of performing triangulation calculations on key points based on the first hand, the third hand, and the fourth hand to obtain the gesture spatial position of the first hand includes: Obtain the coordinates of the key hand points corresponding to the first hand, the key hand points corresponding to the third hand, and the key hand points corresponding to the fourth hand; Corresponding the key points of the first hand, the third hand, and the fourth hand; Based on the coordinates of the corresponding key points of the hands, triangulation calculations are performed on the key points of the first hand, the third hand, and the fourth hand to obtain the spatial position of the gesture of the first hand.

2. The method for matching identical hands in a multi-view camera according to claim 1, characterized in that, The acquisition of the first image captured by the first view of the multi-view camera, wherein the first image contains a first hand and a second hand, includes: Acquire the first image captured by the first view of the multi-view camera; Identify the number of hands contained in the first image; If the number is greater than a preset value, hand detection is performed on the first image to merge the extra hands until the number of hands contained in the first image is less than or equal to the preset value.

3. The method for matching identical hands in a multi-view camera according to claim 1, characterized in that, Before selecting two hands that meet the constraints based on the average depth and reprojection error as the same hand in the multi-view camera, the process also includes: Obtain a natural interaction distance for gestures; The average depth constraint is configured based on the interaction distance.

4. The method for matching identical hands in a multi-view camera according to claim 3, characterized in that, The step of selecting two hands that meet the constraints based on the average depth and reprojection error as the same hand in the multi-view camera includes: Select contestants whose average depth falls within the range of the natural interaction distance of the gesture; Two hands with reprojection errors less than a preset error are selected from the candidates and used as the same hands in the multi-view camera.

5. The method for matching identical hands in a multi-view camera according to claim 4, characterized in that, The step of selecting two hands from the candidates whose reprojection error is less than a preset error as the same hands in the multi-view camera includes: If the number of hands with a reprojection error less than the preset error selected from the candidates is greater than two; The two hands with the smallest reprojection error from the candidates are selected as the same hands in the multi-view camera.

6. A matching device for the same hand in a multi-view camera, used to implement the method according to any one of claims 1-5, characterized in that, include: The first image module is used to acquire a first image captured by the first eye of the multi-view camera, the first image containing a first hand and a second hand; The second image module is used to acquire a second image captured by the second eye of the multi-view camera, the second image containing the third and fourth hands; The first conversion module is used to perform triangulation calculations on key points based on the first hand, the third hand, and the fourth hand to obtain the gesture spatial position of the first hand; The second conversion module is used to perform triangulation calculations on key points based on the second hand, the third hand, and the fourth hand to obtain the gesture spatial position of the second hand; The depth projection module is used to calculate the average depth and reprojection error based on the gesture spatial position of the first hand and the gesture spatial position of the second hand. The constraint module is used to select two hands that meet the constraint conditions as the same hand in the multi-view camera based on the average depth and reprojection error.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the matching method for the same hand in a multi-view camera according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the matching method for the same hand in the multi-view camera according to any one of claims 1 to 5.

Images