Handle, Image Processing System, Method, Device, and Readable Storage Medium

Abstract

The present application discloses a handle, an image processing system, a method, a device, and a readable storage medium. The handle includes a body and a camera module. The camera module includes a first camera and a second camera; the body includes a holding portion; the first camera is disposed on the body, at the top of the holding portion and on the back side of the body, and is capable of collecting a first image; the second camera is disposed on the body, at the bottom of the holding portion and on the back side of the body, and is capable of collecting a second image; wherein, the first image and the second image can be used for positioning the handle, and the second image can be used for obtaining the pose information of the human body holding the handle.

Description

Technical Field

[0001] This application belongs to the technical field of image processing systems, and particularly relates to a handle, an image processing system, a control method of the image processing system, a control device of the image processing system, and a readable storage medium. Background Art

[0002] In related technologies, a VR device includes a head-mounted display and a handle. The head-mounted display is provided with a camera, and the handle is provided with an aperture. The camera on the head-mounted display locates the handle by capturing the aperture on the handle. However, during the use of the VR device, the camera of the head-mounted display rotates with the movement of the human head. When the camera of the head-mounted display is not facing the handle, the position information of the handle cannot be collected, resulting in a large blind area in the positioning of the handle. Summary of the Invention

[0003] This application aims to provide a handle, an image processing system, a control method, a control device, and a readable storage medium, which can at least solve one of the problems that when the camera of the head-mounted display is not facing the handle, the position information of the handle cannot be collected, resulting in a large blind area in the positioning of the handle.

[0004] To solve the above technical problems, this application is implemented as follows:

[0005] In a first aspect, an embodiment of this application provides a handle, which includes a body and a camera module. The camera module includes a first camera and a second camera; the body includes a holding portion; the first camera is disposed on the body, at the top of the holding portion and on the back side of the body, and is capable of collecting a first image; the second camera is disposed on the body, at the bottom of the holding portion and on the back side of the body, and is capable of collecting a second image; wherein, the first image and the second image can be used for positioning the handle, and the second image can be used for obtaining the pose information of the human body holding the handle.

[0006] In a second aspect, an embodiment of this application provides an image processing system, which includes the handle according to any of the above technical solutions and a head-mounted display module, and the head-mounted display module is communicatively connected to the handle.

[0007] In a third aspect, an embodiment of this application provides a control method of an image processing system. The control method is used for the image processing system according to any of the above technical solutions. The control method includes: obtaining the first image collected by the first camera; obtaining the second image collected by the second camera; obtaining the position information of the handle according to the first image and the second image, and obtaining the pose information of the human body holding the handle according to the second image.

[0008] Fourth aspect, an embodiment of the present application proposes a control device for an image processing system. The control device is used for the image processing system of any of the above technical solutions. The control device includes a first image acquisition unit, a second image acquisition unit, and an operation unit. The first image acquisition unit is used to acquire a first image captured by a first camera. The second image acquisition unit is used to acquire a second image captured by a second camera. The operation unit is used to obtain the position information of the handle according to the first image and the second image, and obtain the pose information of the human body holding the handle according to the second image.

[0009] Fifth aspect, an embodiment of the present application proposes an image processing system, including a processor and a memory. The memory stores a program or instruction that can run on the processor. When the program or instruction is executed by the processor, it implements the steps of the control method of the image processing system of any of the above technical solutions.

[0010] Sixth aspect, an embodiment of the present application proposes a readable storage medium. A program or instruction is stored on the readable storage medium. When the program or instruction is executed by the processor, it implements the steps of the control method of the image processing system of any of the above technical solutions.

[0011] In the embodiment of the present application, the handle includes a body. A hand-held part is provided on the body. A first camera and a second camera are respectively provided on both sides of the hand-held part, realizing the installation and positioning of the first camera and the second camera. And since both the first camera and the second camera are provided on the body, the first camera and the second camera can move along with the movement of the handle, so that the first camera and the second camera can take images according to the position where the handle is located. Since the first image and the second image can be used to position the handle, and the first image is captured by the first camera provided on the handle, and the second image is captured by the second camera provided on the handle, the positioning of the handle is no longer restricted by the relative position between the handle and the head-mounted display, thereby reducing the blind area of the handle positioning and improving the positioning accuracy of the handle. While positioning through the first camera and the second camera provided on the handle, the pose information of the human body holding the handle can also be obtained through the second image, realizing the function multiplexing of the second camera, thereby simplifying the structure of the handle, reducing the power consumption of the handle, and further reducing the manufacturing cost and use cost of the handle.

[0012] The image processing system includes a handle of any of the above technical solutions. Therefore, the image processing system has all the beneficial effects of the handle of any of the above technical solutions.

[0013] The control method of the image processing system is used for the image processing system of any of the above technical solutions. Therefore, the control method of the image processing system has all the beneficial effects of the image processing system of any of the above technical solutions.

[0014] The control device of the image processing system is used for the image processing system of any of the above technical solutions. Therefore, the control device of the image processing system has all the beneficial effects of the image processing system of any of the above technical solutions.

[0015] The image processing system includes a processor and a memory. The memory stores programs or instructions that can run on the processor. When the programs or instructions are executed by the processor, the steps of the control method of the image processing system of any of the above technical solutions are implemented. Therefore, the image processing system has all the beneficial effects of the control method of the image processing system of any of the above technical solutions.

[0016] Programs or instructions are stored on a readable storage medium. When the programs or instructions are executed by the processor, the steps of the control method of the image processing system of any of the above technical solutions are implemented. Therefore, the readable storage medium has all the beneficial effects of the control method of the image processing system of any of the above technical solutions.

[0017] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present application. Description of the Drawings

[0018] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0019] Figure 1 is one of the schematic diagrams of the handle according to the embodiment of the present application;

[0020] Figure 2 is another schematic diagram of the handle according to the embodiment of the present application;

[0021] Figure 3 is the schematic structural diagram of the image processing system according to the embodiment of the present application;

[0022] Figure 4 is one of the flowcharts of the control method of the image processing system according to the embodiment of the present application;

[0023] Figure 5 is another flowchart of the control method of the image processing system according to the embodiment of the present application;

[0024] Figure 6 is the third flowchart of the control method of the image processing system according to the embodiment of the present application;

[0025] Figure 7 is the block diagram of the control device of the image processing system according to the embodiment of the present application;

[0026] Figure 8 is one of the block diagrams of the arithmetic unit according to the embodiment of the present application;

[0027] Figure 9 It is the second block diagram of the arithmetic unit according to an embodiment of the present application;

[0028] Figure 10 It is the third block diagram of the arithmetic unit according to an embodiment of the present application.

[0029] Reference numerals:

[0030] 100 Image processing system, 110 Handle, 111 Body, 112 First camera, 113 Second camera, 114 Control key, 115 Holding part, 120 Head-mounted display module, 122 Third camera module, 124 Processor, 130 Camera module. Detailed implementation manners

[0031] Embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary only for explaining the present application and should not be construed as limiting the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts fall within the scope of protection of the present application.

[0032] The terms "first" and "second" in the description and claims of the present application may explicitly or implicitly include one or more of such features. In the description of the present application, unless otherwise specified, the meaning of "a plurality" is two or more. In addition, "and / or" in the description and claims means at least one of the connected objects. The character " / " generally means that the related objects before and after are in an "or" relationship.

[0033] In the description of the present application, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present application.

[0034] In the description of the present application, it should be noted that, unless otherwise clearly defined and limited, the terms "installation", "connection", and "coupling" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be a direct connection or an indirect connection through an intermediate medium, and it may be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific circumstances.

[0035] The following will be described in conjunction with Figures 1 to 10 a handle 110, an image processing system 100, a control method, a control device, and a readable storage medium according to embodiments of the present application.

[0036] As Figure 1 and Figure 2 shown, according to some embodiments of the present application, the handle 110 includes a body 111 and an imaging module 130. The imaging module 130 includes a first camera 112 and a second camera 113. The body 111 includes a grip portion 115. The first camera 112 is disposed on the body 111, at the top of the grip portion 115 and on the back side of the body 111, and is capable of collecting a first image. The second camera 113 is disposed on the body 111, at the bottom of the grip portion 115 and on the back side of the body 111, and is capable of collecting a second image. Among them, the first image and the second image can be used to locate the handle 110, and the second image can be used to obtain the pose information of the human body holding the handle 110.

[0037] In this embodiment, the handle 110 includes a body 111. A hand-held portion is provided on the body 111, and a first camera 112 and a second camera 113 are respectively provided on both sides of the hand-held portion, realizing the installation and positioning of the first camera 112 and the second camera 113. And since both the first camera 112 and the second camera 113 are provided on the body 111, the first camera 112 and the second camera 113 can move along with the movement of the handle 110, so that the first camera 112 and the second camera 113 can take images according to the position where the handle 110 is located. Since the first image and the second image can be used to position the handle 110, and the first image is taken by the first camera 112 provided on the handle 110, and the second image is taken by the second camera 113 provided on the handle 110, the positioning of the handle 110 is no longer restricted by the relative position between the handle 110 and the head-mounted display, thereby reducing the blind area of the handle 110 positioning and improving the positioning accuracy of the handle 110. While positioning through the first camera 112 and the second camera 113 provided on the handle 110, the pose information of the human body holding the handle 110 can also be obtained through the second image, realizing the function multiplexing of the second camera 113, thereby simplifying the structure of the handle 110, reducing the power consumption of the handle 110, and further reducing the manufacturing cost and use cost of the handle 110.

[0038] Specifically, when the human body holds the handle, the first camera 112 is located at the top of the holding portion 115, which means the first camera 112 is above the holding portion 115.

[0039] When the human body holds the handle, the first camera 112 is located on the back side of the body 111, which means the first camera 112 is on the side of the body 111 facing the human body.

[0040] When the human body holds the handle, the second camera 113 is located at the bottom of the holding portion 115, which means the second camera 113 is below the holding portion 115.

[0041] When the human body holds the handle, the second camera 113 is located on the back side of the body 111, which means the second camera 113 is on the side of the body 111 facing the human body.

[0042] In an implementation manner, two camera modules, namely a first camera 112 and a second camera 113, are provided on the handle 110. When the human body holds the handle 110, the first camera 112 is located at the top of the body 111 of the handle 110 and on the back side of the body 111, so that the first camera 112 can face the human body, and thus the first image can be taken through the first camera 112.

[0043] When a human body holds the handle 110, the second camera 113 is located at the bottom of the main body 111 of the handle 110 and on the back side of the main body 111. The second camera 113 faces the human body, and thus a second image can be captured through the second camera 113.

[0044] When positioning the handle 110, after the second image is framed, some frames of the second image are combined with the first image, and based on the depth information of some frames of the second image and the depth information of the first image, the positioning of the handle 110 can be achieved.

[0045] After the second image is framed, another part of the frames of the second image can be combined with the images collected by the head-mounted display to determine the position and posture of the human body. Thus, through the second image collected by the second camera 113, the positioning of the handle 110 can be achieved, and the position and posture of the human body can also be determined.

[0046] Furthermore, the VR device can collect the position and posture of the lower body of the human body through the second camera 113, collect the position and posture of the upper body of the human body through the camera module of the head-mounted display, and then synthesize the second image and the images collected by the camera module of the head-mounted display, thereby achieving the collection of the position and posture of the human body.

[0047] In an implementation manner, for the handle 110 provided in the present application, since positioning is performed through the first camera 112 and the second camera 113 provided on the handle 110, compared with the solution of setting a halo on the handle 110 for positioning, even if the two handles 110 cross each other or move into the blind area of the head-mounted display camera, it will not affect the positioning progress of the handle 110.

[0048] For the handle 110 provided in the present application, since positioning is performed through the first camera 112 and the second camera 113 provided on the handle 110, that is, the handle 110 provided in the present application can achieve the positioning of the handle 110 through two camera modules. Compared with the handle 110 that uses three camera modules for positioning, the system efficiency consumed is relatively small, thereby optimizing power consumption and cost.

[0049] In the handle 110 provided in the present application, the second camera 113 is framed to realize the reuse of the same camera device for positioning and image capture in six degrees of freedom (Degree Of Freedom, DOF), so as to use two cameras to complete the work of six degrees of freedom positioning and lower body image capture simultaneously. Since the camera is installed on the handle 110 and can move arbitrarily with the handle 110, dead-angle-free positioning of the handle 110 is achieved, and it will not be lost during use.

[0050] In an implementation manner, the first camera 112 and the second camera 113 are depth cameras.

[0051] According to some embodiments of the present application, as Figure 1 shown, the angle α between the axial direction of the first camera 112 and the axial direction of the second camera 113 is greater than or equal to 60 degrees and less than or equal to 120 degrees.

[0052] In this embodiment, the angle α between the axial direction of the first camera 112 and the axial direction of the second camera 113 is 60 degrees to 120 degrees, which can further increase the acquisition range of the first camera 112 and the second camera 113, reduce the positioning dead angle of the handle 110, and thus improve the performance of the handle 110.

[0053] Furthermore, the angle α between the axial direction of the first camera 112 and the axial direction of the second camera 113 can be 60 degrees.

[0054] The angle α between the axial direction of the first camera 112 and the axial direction of the second camera 113 can also be 90 degrees.

[0055] The angle α between the axial direction of the first camera 112 and the axial direction of the second camera 113 can also be 120 degrees.

[0056] According to some embodiments of the present application, as Figure 2 shown, the distance B between the first camera 112 and the second camera 113 is greater than or equal to 90 millimeters and less than or equal to 130 millimeters.

[0057] In this embodiment, the distance B between the first camera 112 and the second camera 113 is 90 millimeters to 130 millimeters, which can further increase the acquisition range of the first camera 112 and the second camera 113, reduce the positioning dead angle of the handle 110, and thus improve the performance of the handle 110.

[0058] Furthermore, the distance B between the first camera 112 and the second camera 113 can be 90 millimeters.

[0059] The distance B between the first camera 112 and the second camera 113 can also be 110 millimeters.

[0060] The distance B between the first camera 112 and the second camera 113 can also be 130 millimeters.

[0061] According to some embodiments of the present application, as Figure 1 shown, the handle 110 further includes a control key 114, and the control key 114 is arranged on the front side of the body 111; the first camera 112 and / or the second camera 113 are arranged on the back side of the body 111.

[0062] In this embodiment, the control keys 114 are arranged on the front side of the main body 111; the first camera 112 and the second camera 113 are arranged on the back side of the main body 111, and the back side of the main body 111 and the front side of the main body 111 are two opposite sides of the main body 111, so that the first camera 112 and the second camera 113 are arranged on the back side of the handle 110, facilitating the collection of the human body through the first camera 112 and / or the second camera 113, and further facilitating the positioning of the handle 110 through the first image and / or the second image, as well as the collection of the position and posture of the human body, improving the convenience of the handle 110 during use.

[0063] The control keys 114 include a joystick and touch function keys.

[0064] Specifically, when the human body holds the handle, the back side of the main body 111 is the side of the main body 111 away from the human body.

[0065] The front side of the main body 111 is the side opposite to the back side of the main body 111, that is, when the human body holds the handle, the front side of the main body 111 is the side of the main body 111 facing the human body.

[0066] According to some embodiments of the present application, the frame rate of the second camera 113 is greater than or equal to the frame rate of the first camera 112.

[0067] In this embodiment, after the handle 110 collects images through the first camera 112 and the second camera 113, the second image needs to be frame-divided. Some frames of the second image are used to cooperate with the first image to achieve the positioning of the handle 110, and another part of the frames of the second image needs to be used for the collection of the position and posture of the human body. Since the frame rate of the second camera 113 is greater than the frame rate of the first camera 112, the number of frames of the frame-divided second image can match the number of frames of the first image, thereby facilitating the processing of the first image and the second image, and further improving the accuracy of positioning the handle 110.

[0068] The frame rate of the second camera 113 can also be equal to the frame rate of the first camera 112.

[0069] According to the image processing system 100 of some embodiments of the present application, as Figure 3 shown, the image processing system 100 includes a handle 110 and a head-mounted display module 120 as described in any of the above embodiments, and the head-mounted display module 120 is communicatively connected to the handle 110. Since the image processing system 100 includes the handle 110 as described in any of the above embodiments, the image processing system 100 has all the beneficial effects of the handle 110 in any of the above embodiments.

[0070] The image processing system 100 can be a virtual reality (VR) device, or a mobile phone or a tablet computer.

[0071] According to some embodiments of the present application, as Figure 3 shown, the image processing system 100 further includes a head-mounted display module 120. The head-mounted display module 120 includes a third camera module 122 and a processor 124. The third camera module 122 is capable of collecting a third image; the processor 124 is electrically connected to the third camera module 122 and communicatively connected to the handle 110, and is capable of obtaining the pose information of the human body according to the second image and the third image.

[0072] In this embodiment, the head-mounted display module 120 includes a third camera module 122. The third camera module 122 is capable of collecting a third image, and thus the image of the upper body of the human body can be collected through the third camera module 122. The processor 124 communicates with the handle 110, and thus the second image can be obtained. After synthesizing the second image and the third image, the position and posture of the whole body of the human body can be determined, further improving the accuracy of determining the pose information of the human body. And the synthesis of the second image and the third image is realized by the processor 124 provided in the head-mounted display module 120, further simplifying the structure of the handle 110, and thus reducing the power consumption of the handle 110.

[0073] Specifically, the image processing system 100 realizes the synthesis of the second image and the third image by setting the processor 124 on the head-mounted display module 120, makes full use of the third camera module 122 of the head-mounted display module 120 to track the upper body and the computing power advantage of the head-mounted display module 120 to perform full-body image synthesis processing, saves 1 / 3 of the cost of the camera device of the handle 110, and improves the power consumption efficiency of the system.

[0074] Specifically, the second camera 113 arranged below the handle 110 is framed to perform the image of the lower body and the dynamic tracking of the image synthesis, and the data is transmitted to the head-mounted display module 120, and the head-mounted display module 120 completes the upper body tracking and image synthesis, so as to realize the three-dimensional reconstruction, dynamic tracking and full-body digital human dynamic synthesis of the whole body while completing the 6-degree-of-freedom positioning.

[0075] Specifically, the third camera module 122 is a camera. The processor 124 is the processor of the head-mounted module 120.

[0076] According to some embodiments of the present application, the frame rate of the third camera module 122 is the same as the frame rate of the second camera 113.

[0077] In this embodiment, the frame rate of the third camera module 122 is the same as that of the second camera 113. When synthesizing the third image and the second image, there is no need for frame interpolation, which makes the synthesis of the second image and the third image more accurate, thereby improving the accuracy of obtaining the pose information of the human body. Moreover, since the frame rate of the third camera module 122 is the same as that of the second camera 113, the algorithm required for image synthesis is simpler, thereby reducing the difficulty of image synthesis.

[0078] The frame rate of the third camera module 122 can also be different from that of the second camera 113. In the case where the frame rate of the third camera module 122 is different from that of the second camera 113, image synthesis is achieved through timestamp coordination. Moreover, the difference in the frame rate between the third camera module 122 and the second camera 113 can reduce the cost of the electronic device.

[0079] According to the control method of the image processing system in some embodiments of the present application, the control method is used for the image processing system in any of the above embodiments, as Figure 4 shown, the control method includes:

[0080] Step 402, obtaining a first image collected by a first camera;

[0081] Step 404, obtaining a second image collected by a second camera;

[0082] Step 406, obtaining the position information of the handle according to the first image and the second image, and obtaining the pose information of the human body holding the handle according to the second image.

[0083] In this embodiment, the image processing system separately controls the first camera and the second camera disposed on the handle to collect the first image and the second image, and while obtaining the position information of the handle according to the first image and the second image, obtains the pose information of the human body holding the handle according to the second image. By using the first camera and the second camera simultaneously, the positioning of the handle and the acquisition of the human body pose information are realized, thereby realizing the function multiplexing of the second camera, simplifying the structure of the handle, reducing the power consumption of the handle, and reducing the manufacturing cost and usage cost of the handle. Since the first image is captured by the first camera disposed on the handle and the second image is captured by the second camera disposed on the handle, the positioning of the handle is no longer restricted by the relative position between the handle and the head-mounted display, thereby reducing the blind area of the handle positioning and improving the positioning accuracy of the handle.

[0084] Specifically, the depth image information formed by the first camera and the second camera is used to perform the 6-degree-of-freedom positioning of the handle itself.

[0085] The image frames obtained by the second camera are distributed according to different exposure durations. A part of them is used for the 6-degree-of-freedom positioning and pose information acquisition of the handle itself in the simulation software SLAM calculation, and the other part is used for dynamic image tracking and synthesis, so as to realize the two-way task execution function on the same component.

[0086] According to some embodiments of the present application, obtaining the position information of the handle based on the first image and the second image includes: splitting the second image into frames to obtain a first sub-image and a second sub-image; obtaining the position information of the handle based on the first image and the first sub-image.

[0087] In this embodiment, the second image is split into frames to obtain a first sub-image and a second sub-image, and the position information of the handle is obtained based on the first image and the first sub-image.

[0088] According to some embodiments of the present application, obtaining the pose information of the human body holding the handle based on the second image includes: obtaining the pose information of the lower body of the human body holding the handle based on the second sub-image; or, obtaining a third image collected by a third camera module of the head-mounted display module; synthesizing the second sub-image and the third image; obtaining the pose information of the human body based on the synthesized second sub-image and third image.

[0089] In this embodiment, while obtaining the position information of the handle, the pose information of the lower body of the human body holding the handle can also be obtained based on the second sub-image, and the third camera module of the head-mounted display module can also be controlled to collect a third image; the processor of the head-mounted display module is controlled to synthesize the second sub-image and the third image, and the pose information of the human body is obtained based on the synthesized second sub-image and third image, so as to realize the function multiplexing of the second camera, simplify the structure of the handle, reduce the power consumption of the handle, and reduce the manufacturing cost and usage cost of the handle.

[0090] When splitting the second image into frames, the proportional relationship of the exposure frames is allocated according to different exposure requirements. For example, the time required for the exposure frames used for image tracking and synthesis is longer than that used for simulation. A part of the image frames with a certain duration is used for the self-positioning of the 6 degrees of freedom of the handle, and another part of the exposure frames with a certain duration is used for image tracking and synthesis.

[0091] As Figure 5 shown, the control method of the image processing system includes:

[0092] Step 502, obtaining a first image collected by a first camera;

[0093] Step 504, obtaining a second image collected by a second camera;

[0094] Step 506, splitting the second image into frames to obtain a first sub-image and a second sub-image;

[0095] Step 508, obtain the position information of the handle according to the first image and the first sub-image;

[0096] Step 510, obtain a third image captured by a third camera module of the headset module;

[0097] Step 512, synthesize the second sub-image and the third image;

[0098] Step 514, obtain the pose information of the human body according to the synthesized second sub-image and third image.

[0099] As Figure 6 shown, the control method of the image processing system includes:

[0100] Step 802, obtain a first image captured by a first camera;

[0101] Step 804, obtain a second image captured by a second camera;

[0102] Step 806, perform frame division on the second image to obtain a first sub-image and a second sub-image;

[0103] Step 808, obtain the position information of the handle according to the first image and the first sub-image;

[0104] Step 810, obtain the pose information of the lower body of the human body holding the handle according to the second sub-image.

[0105] The control device 600 of the image processing system according to some embodiments of the present application, the control device 600 is used for the image processing system of any of the above embodiments, as Figure 7 shown, the control device 600 includes a first image acquisition unit 610, a second image acquisition unit 620 and an operation unit 630; the first image acquisition unit 610 is used to obtain a first image captured by a first camera; the second image acquisition unit 620 is used to obtain a second image captured by a second camera; the operation unit 630 is used to obtain the position information of the handle according to the first image and the second image, and obtain the pose information of the human body holding the handle according to the second image.

[0106] In this embodiment, the image processing system controls the first camera and the second camera disposed on the handle to collect the first image and the second image respectively. While obtaining the position information of the handle based on the first image and the second image, it obtains the pose information of the human body holding the handle according to the second image. By using the first camera and the second camera simultaneously, the positioning of the handle and the acquisition of the human body pose information are achieved, thereby realizing the function multiplexing of the second camera, simplifying the structure of the handle, reducing the power consumption of the handle, and reducing the manufacturing cost and usage cost of the handle. Since the first image is captured by the first camera disposed on the handle and the second image is captured by the second camera disposed on the handle, the positioning of the handle is no longer restricted by the relative position between the handle and the head-mounted display, thereby reducing the blind area of the handle positioning and improving the positioning accuracy of the handle.

[0107] According to some embodiments of the present application, as Figure 8 shown, the operation unit 630 includes a frame division unit 631 and a positioning unit 632; the frame division unit 631 is configured to divide the second image into frames to obtain a first sub-image and a second sub-image; the positioning unit 632 is configured to obtain the position information of the handle according to the first image and the first sub-image.

[0108] In this embodiment, the second image is divided into frames to obtain a first sub-image and a second sub-image, and the position information of the handle is obtained according to the first image and the first sub-image.

[0109] According to some embodiments of the present application, as Figure 9 shown, the operation unit 630 may include an operation sub-unit 636, and the operation sub-unit 636 is configured to obtain the pose information of the lower body of the human body holding the handle according to the second sub-image.

[0110] As Figure 10 shown, the operation unit 630 may also include a third image acquisition unit 633, a synthesis unit 634, and an acquisition unit 635. The third image acquisition unit 633 is configured to acquire a third image acquired by the third camera module of the head-mounted display module; the synthesis unit 634 is configured to synthesize the second sub-image and the third image; the acquisition unit 635 is configured to obtain the pose information of the human body according to the synthesized second sub-image and third image.

[0111] In this embodiment, while obtaining the handle position information, the pose information of the lower body of the human body holding the handle can also be obtained according to the second sub-image, and the third camera module of the head-mounted display module can also be controlled to acquire a third image; control the processor of the head-mounted display module to synthesize the second sub-image and the third image, and obtain the pose information of the human body according to the synthesized second sub-image and third image, thereby realizing the function multiplexing of the second camera, simplifying the structure of the handle, reducing the power consumption of the handle, and reducing the manufacturing cost and usage cost of the handle.

[0112] When performing frame processing on the second image, the proportional relationship of the exposure frames is allocated according to different time requirements of the exposure. For example, the time required for the exposure frames used for image tracking and synthesis is longer than that used for simulation. A part of the image frames with a certain duration is used for the self-positioning of the 6 degrees of freedom of the handle, and another part of the exposure frames with a certain duration is used for image tracking and synthesis.

[0113] According to some embodiments of the present application, an image processing system includes a processor and a memory. The memory stores programs or instructions that can be run on the processor. When the programs or instructions are executed by the processor, the steps of the control method of the image processing system in any of the above embodiments are implemented.

[0114] In this embodiment, the image processing system includes a processor and a memory. The memory stores programs or instructions that can be run on the processor. When the programs or instructions are executed by the processor, the steps of the control method of the image processing system in any of the above embodiments are implemented. Therefore, the image processing system has all the beneficial effects of the control method of the image processing system in any of the above embodiments.

[0115] According to some embodiments of the present application, a readable storage medium stores programs or instructions. When the programs or instructions are executed by a processor, the steps of the control method of the image processing system in any of the above embodiments are implemented.

[0116] In this embodiment, the readable storage medium stores programs or instructions. When the programs or instructions are executed by a processor, the steps of the control method of the image processing system in any of the above embodiments are implemented. Therefore, the readable storage medium has all the beneficial effects of the control method of the image processing system in any of the above embodiments.

[0117] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "schematic embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0118] Although the embodiments of the present application have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and purposes of the present application. The scope of the present application is defined by the claims and their equivalents.

Claims

1. A handle, characterized in that, Comprising: A body, the body including a holding portion; An imaging module, the imaging module consisting of a first camera and a second camera; The first camera is disposed on the body, at the top of the holding portion and on the back side of the body, and is capable of collecting a first image; The second camera is disposed on the body, at the bottom of the holding portion and on the back side of the body, and is capable of collecting a second image; Wherein, the first image and the second image can be used for positioning the handle, and the second image can be used for obtaining the pose information of the human body holding the handle; When the human body holds the handle, the first camera faces the human body, and the second camera faces the human body; The first camera and the second camera are depth cameras.

2. The handle according to claim 1, wherein The angle between the axis of the first camera and the axis of the second camera is greater than or equal to 60 degrees and less than or equal to 120 degrees.

3. The handle according to claim 1, characterized in that, The distance between the first camera and the second camera is greater than or equal to 90 millimeters and less than or equal to 130 millimeters.

4. The handle according to claim 1, wherein Further comprising: Control keys, the control keys being disposed on the front side of the body; The first camera and / or the second camera are disposed on the back side of the body.

5. The handle according to any one of claims 1 to 4, characterized in that The frame rate of the second camera is greater than or equal to the frame rate of the first camera.

6. An image processing system, characterized in that, Comprising: The handle according to any one of claims 1 to 5; A head-mounted display module, the head-mounted display module being communicatively connected to the handle.

7. The image processing system according to claim 6, characterized in that, The head-mounted display module includes: A third imaging module, the third imaging module being capable of collecting a third image; A processor, the processor being electrically connected to the third imaging module and communicatively connected to the handle, and being capable of obtaining the pose information of the human body according to the second image and the third image.

8. The image processing system according to claim 7, wherein The frame rate of the third imaging module is the same as the frame rate of the second camera.

9. A control method for an image processing system, characterized in that, The control method is used for the image processing system according to any one of claims 6 to 8, and the control method includes: Obtaining the first image collected by the first camera; Obtaining the second image collected by the second camera; Obtaining the position information of the handle according to the first image and the second image, and obtaining the pose information of the human body holding the handle according to the second image.

10. The control method of the image processing system according to claim 9, characterized in that, The obtaining the position information of the handle according to the first image and the second image includes: Framing the second image to obtain a first sub-image and a second sub-image; Obtaining the position information of the handle according to the first image and the first sub-image.

11. The control method of the image processing system according to claim 10, wherein The obtaining the pose information of the human body holding the handle according to the second image includes: Obtaining the pose information of the lower body of the human body holding the handle according to the second sub-image; or Obtaining the third image collected by the third imaging module of the head-mounted display module; Synthesizing the second sub-image and the third image; Obtaining the pose information of the whole body of the human body holding the handle according to the synthesized second sub-image and third image.

12. A control device for an image processing system, characterized in that, The control device is used for the image processing system according to any one of claims 6 to 8, and the control device includes: A first image acquisition unit for controlling the first camera to collect a first image; A second image acquisition unit, configured to control the second camera to acquire a second image; An operation unit, configured to obtain the position information of the handle according to the first image and the second image, and obtain the pose information of the human body holding the handle according to the second image.

13. The control device of the image processing system according to claim 12, characterized in that, The operation unit includes: A frame division unit, configured to divide the second image into frames to obtain a first sub-image and a second sub-image; A positioning unit, configured to obtain the information of the handle according to the first image and the first sub-image.

14. The control device of the image processing system according to claim 13, characterized in that, The operation unit includes: An operation sub-unit, configured to obtain the pose information of the lower body of the human body holding the handle according to the second sub-image; or A third image acquisition unit, configured to obtain a third image acquired by a third camera module of the head-mounted display module; A synthesis unit, configured to synthesize the second sub-image and the third image; An acquisition unit, configured to obtain the pose information of the human body according to the synthesized second sub-image and third image.

15. An image processing system, characterized in that, Comprising a processor and a memory, the memory stores a program or instruction that can run on the processor, and when the program or instruction is executed by the processor, the steps of the control method of the image processing system according to any one of claims 9 to 11 are implemented.

16. A readable storage medium, characterized in that, A program or instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, the steps of the control method of the image processing system according to any one of claims 9 to 11 are implemented.

Images