Internal pitching interaction system based on fisheye infrared camera and internal pitching interaction system calibration method

Abstract

The invention discloses an internal pitching interaction system based on a fisheye infrared camera and an internal pitching interaction system calibration method, belongs to the technical field of internal pitching interaction, and solves the problems that the accuracy error of angular points of a checkerboard identified by the fisheye camera is relatively large in an internal pitching projection display environment, the calibration position of the internal pitching interaction system cannot be manually and finely adjusted, and the accuracy of the internal pitching interaction system is relatively high. And the problem of deviation between the projection picture and the position of the video identification module is solved. The system comprises an internal pitching ball, a projector, a fisheye infrared camera, an annular infrared light supplementing device and an internal pitching ball mounting frame, the inner throwing ball is arranged on the inner throwing ball mounting frame; the annular infrared light supplementing device is arranged under the opening of the inner throwing ball, and a notch is formed in the inner ring of the annular infrared light supplementing device and used for fixing the fisheye infrared camera; and the projector is arranged in the inner pitching ball mounting frame and projects into the inner pitching ball through the high-definition fisheye lens. The method is suitable for installation and calibration scenes of the internal pitching interaction system.

Description

Technical Field

[0001] The invention belongs to the technical field of internal ball pitching interaction, and in particular relates to an internal ball pitching interaction system based on a fisheye infrared camera. Background Art

[0002] The internal projection ball interactive system is generally composed of a ball screen, a projector with a fisheye lens, an infrared camera with a fisheye lens, a circular infrared fill light device, a computer host and a system structure framework. The experiencer can touch the surface of the ball screen with his fingers, and the interactive software can detect the finger touch position to achieve real-time interaction with the internal projection ball display effect.

[0003] Chinese patent publication number CN116055708A discloses "a method and system for perceiving visual interactive spherical screen three-dimensional stereoscopic imaging", which realizes the basic functions of displaying and interacting with the video content of the inner pitch ball. At present, in the inner pitch ball interactive system, the above patent adopts the scheme of obtaining the parameter information of the spherical screen device to realize the conversion of the original video data to the three-dimensional spherical screen projection data, and obtaining the finger touch information through the projection chessboard and color-coded structured light and obtaining the action data through the infrared sensing module and the video recognition module. Such inner pitch ball interactive schemes generally need to analyze the spherical screen parameters and calibrate the interactive position through the chessboard and other methods. In the projection display environment of the inner pitch ball, the marking methods such as the chessboard have serious distortion. There is a problem of large precision error for the fisheye camera to identify the corner points of the chessboard, and it is impossible to manually fine-tune its calibration position, resulting in a deviation between the position of the projection picture and the video recognition module, affecting the experience of real-time touch interaction. Summary of the invention

[0004] The present invention proposes an intra-ball interactive system based on a fisheye infrared camera, the purpose of which is to solve the problem that in the projection display environment of the intra-ball, marking methods such as checkerboards have serious distortion, there is a large accuracy error in the fisheye camera's recognition of the corner points of the checkerboard, and its calibration position cannot be manually fine-tuned, resulting in a deviation between the position of the projection image and the video recognition module, affecting the experience of real-time touch interaction.

[0005] An internal ball throwing interactive system based on a fisheye infrared camera, the system comprising:

[0006] Inner ball, projector, fisheye infrared camera, annular infrared fill light device and inner ball mounting frame;

[0007] The inner pitch ball is arranged on the inner pitch ball mounting frame, and an opening is arranged at the bottom of the inner pitch ball;

[0008] The annular infrared fill light device is arranged directly below the opening of the inner ball, and is used to fill light inside the inner ball. The inner ring of the annular infrared fill light device is provided with a notch, and is used to fix the fisheye infrared camera.

[0009] The projector is arranged inside the inner projection ball installation frame, the lens of the projector is a high-definition fisheye lens, and the projector is used to project into the inner projection ball;

[0010] The fisheye infrared camera is used to capture images inside the inner pitching ball.

[0011] Furthermore, a preferred solution is provided: a plurality of infrared lamp beads are provided on the annular infrared fill light device.

[0012] Furthermore, a preferred solution is provided: the plurality of infrared lamp beads are distributed on the housing of the annular infrared fill light device.

[0013] Furthermore, a preferred solution is provided: the plurality of infrared lamp beads are arranged in a ring, and the ring is at least one circle.

[0014] Furthermore, a preferred solution is provided: the circular ring is at least 2 circles, the spacing between the multiple infrared lamp beads along the tangent direction of the circular ring is 10 mm, and the spacing between the multiple infrared lamp beads along the radial direction is 10 mm.

[0015] Furthermore, a preferred solution is provided: the opening for the inner ball pitching is circular.

[0016] Furthermore, a preferred solution is provided: the radius of the circle is not less than the outer diameter of the annular infrared fill light device.

[0017] Furthermore, a preferred solution is provided: the interface of the fisheye infrared camera is an M12 interface.

[0018] Furthermore, a preferred solution is provided: an 850nm narrowband filter is installed at the rear of the lens of the fisheye infrared camera.

[0019] A method for calibrating an internal pitching interactive system, wherein the internal pitching interactive system is an internal pitching interactive system based on a fisheye infrared camera as described in any combination of the above schemes, and the system also includes a computer host. The method is implemented using a multi-concentric circle multi-reference point position calibration algorithm, and the method includes:

[0020] A projector is used to project the calibration image onto the inner projection ball through a high-definition fisheye lens to obtain multiple crosshair coordinates;

[0021] The calibration screen includes a plurality of crosshairs, and the plurality of crosshairs are evenly arranged;

[0022] Touch the crosshairs in sequence, and use the fisheye infrared camera to identify the coordinates of the center point of the contour touched by the finger each time to obtain multiple mapping coordinates;

[0023] When there is inconsistency between the crosshair coordinates and the corresponding mapping coordinates, the display screen of the computer host and the projection screen of the system are set to a copy screen relationship, and the coordinates of the center point of the contour touched by the finger are adjusted on the display screen of the computer host.

[0024] Compared with the prior art, the advantages of the present invention are:

[0025] The invention discloses an internal projection ball interactive system based on a fisheye infrared camera. The system senses the touch image of a finger on the outer surface of a ball screen through a fisheye infrared camera and a circular infrared fill light device, and combines a multi-concentric circle multi-reference point position calibration algorithm between the fisheye infrared camera and the fisheye projection lens. The system can realize precise alignment of the interactive position and the feedback position through automatic fast calibration and manual position calibration, thereby achieving the purpose of precise multi-point interaction with the content displayed on the internal projection ball screen.

[0026] The present invention is applicable to the installation and calibration scenarios of the internal ball pitching interactive system. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1 The overall structure diagram of an internal ball interactive system based on a fisheye infrared camera according to a specific embodiment of the present invention, wherein 1 is an internal ball, 2 is a projector, 3 is a fisheye infrared camera, 4 is a ring-shaped infrared fill light device, 5 is an internal ball installation frame, 6 is a computer host, and 7 is a high-definition fisheye lens;

[0029] Figure 2 This is a schematic structural diagram of the annular infrared fill light device according to a specific embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram of a picture taken by a fisheye infrared camera according to a first specific embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram of multiple concentric circles and multiple reference points according to the second specific implementation mode of the present invention;

[0032] Figure 5Schematic diagram of multi-concentric circle multi-reference point projection display according to the second specific embodiment of the present invention

[0033] Figure 6 This is a schematic diagram of finger recognition sensing according to the second specific implementation mode of the present invention;

[0034] Figure 7 This is a schematic diagram of the fisheye camera image calibration position according to the second specific implementation mode of the present invention;

[0035] Figure 8 This is a calibration workflow diagram of an internal ball-throwing interactive system based on a fisheye infrared camera as described in the second specific implementation mode of the present invention. DETAILED DESCRIPTION

[0036] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0037] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0038] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise.

[0039] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present application. Therefore, the present application is not limited to the specific implementation methods disclosed below.

[0041] Implementation method 1:

[0042] Reference Figure 1 , Figure 2 , Figure 3 This embodiment will be described.

[0043] An internal ball throwing interactive system based on a fisheye infrared camera, the system comprising:

[0044] Inner projection ball 1, projector 2, fisheye infrared camera 3, annular infrared fill light device 4 and inner projection ball installation frame 5;

[0045] The inner pitch ball 1 is arranged on the inner pitch ball mounting frame 5, and an opening is arranged at the bottom of the inner pitch ball 1;

[0046] The annular infrared fill light device 4 is arranged just below the opening of the inner ball 1, and is used to fill light inside the inner ball 1. The inner ring of the annular infrared fill light device 4 is provided with a notch, and is used to fix the fisheye infrared camera 3;

[0047] The projector 2 is arranged inside the inner projection ball installation frame 5, the lens of the projector 2 is a high-definition fisheye lens 7, and the projector 2 is used to project into the inner projection ball 1;

[0048] The fisheye infrared camera 3 is used to capture images inside the inner pitching ball 1 .

[0049] Furthermore, the annular infrared fill light device 4 is provided with a plurality of infrared lamp beads; the plurality of infrared lamp beads are distributed on the outer shell of the annular infrared fill light device 4; the plurality of infrared lamp beads are arranged in a circular ring, the circular ring is at least one circle; the circular ring is at least 2 circles, the spacing of the plurality of infrared lamp beads along the tangent direction of the circular ring is 10 mm, and the spacing of the plurality of infrared lamp beads along the radial direction is 10 mm;

[0050] The annular infrared fill light device 4 described in this embodiment is as follows Figure 2 As shown, there are 106 infrared lamp beads, each of which has a light emission wavelength of 850nm and an emission angle of 90 degrees, so as to ensure that the interior of the entire inner ball is illuminated by the infrared lamp beads. At the same time, the driving power supply of the annular fill light device has an adjustable working voltage to ensure that the fill light brightness can be controlled within the optimal range;

[0051] Furthermore, the opening of the inner ball 1 is circular; the radius of the circle is not less than the outer diameter of the annular infrared fill light device 4;

[0052] Furthermore, the fisheye infrared camera 3 is arranged at the notch of the inner ring of the annular infrared fill light device 4. The interface of the fisheye infrared camera 3 is an M12 interface, the focal length is 1.19 mm, and the field of view is 185°x185°x185°; an 850nm narrow-band filter is installed at the rear of the lens of the fisheye infrared camera 3 to filter out ambient light of other bands, and the clearest internal picture of the inner ball can be captured by manual focusing. The captured image is as follows Figure 3 shown.

[0053] The fisheye infrared camera-based internal pitching interactive system described in this embodiment can realize functions such as clicking and dragging the interactive display effect and zooming and rotating the displayed image during multi-point simultaneous operation by identifying the interactive recognition position. In addition, the recognition system can recognize continuous touch action types through the pre-trained AI gesture recognition model to realize the internal pitching interactive function based on specific gestures.

[0054] Implementation method 2:

[0055] Reference Figure 4 , Figure 5 , Figure 6 , Figure 7 , Figure 8 This embodiment will be described.

[0056] A method for calibrating an internal pitching interactive system, wherein the internal pitching interactive system is an internal pitching interactive system based on a fisheye infrared camera as described in Embodiment 1, wherein the system further comprises a computer host 6, wherein the method is implemented by using a multi-concentric circle multi-reference point position calibration algorithm, wherein the arrangement of the calibration reference points on the plane is as follows: Figure 4 As shown, the projector projects the reference point image onto the inner projection ball through a high-definition fisheye lens, and its arrangement is as follows Figure 5 As shown, the fisheye infrared camera then identifies the coordinates of the center point of the contour touched by the finger at the corresponding position, such as Figure 6 As shown, the center point coordinate is used as a mapping coordinate between the camera image and the projected image, and the recognition of all finger touch points is completed in the specified order, such as Figure 7 As shown, a mapping relationship of all coordinate points between the projection screen and the camera screen is obtained, realizing an accurate one-to-one correspondence between the actual interaction position and the effect feedback position in the projection screen.

[0057] Specifically, the method comprises:

[0058] Turn on the projector 2, project the calibration image onto the inner projection ball 1 through the high-definition fisheye lens 7, and obtain multiple crosshair coordinates;

[0059] like Figure 4As shown, the calibration screen includes a plurality of crosshairs, and the plurality of crosshairs are evenly arranged in a concentric circle structure, and the interval between adjacent crosshairs on the same circle is the ratio of the circumference of the circle to the number of crosshairs on the circle, and the interval between two adjacent concentric circles does not exceed the ratio of half the width of the projection screen to the number of concentric circles;

[0060] Touch the crosshairs in sequence, and use the fisheye infrared camera 3 to identify the coordinates of the center point of the contour touched by the finger each time, and obtain multiple mapping coordinates;

[0061] When there is inconsistency between the crosshair coordinates and the corresponding mapping coordinates, the display screen of the computer host 6 and the projection screen of the system are set to a copy screen relationship, and the coordinates of the center point of the contour touched by the finger are adjusted on the display screen of the computer host 6.

[0062] Specifically, Figure 8 As shown, the calibration method described in this embodiment is specifically as follows:

[0063] S1: using the projector 2 to project the calibration image onto the inner pitch ball (1) through the high-definition fisheye lens 7, and the fisheye infrared camera 3 collects the real-time image of the inner pitch ball 1;

[0064] S2: Determine whether the projection image and the camera image need to be calibrated:

[0065] S2.1: If necessary, project multiple concentric circles of multiple reference points using projector 2, and highlight the current calibration point, then determine whether it is in automatic calibration mode. If so, stably identify the finger touch position for more than 3 seconds, automatically save the finger touch position and switch to the next calibration point until the position calibration of all points is completed; if not, observe the finger touch position through the fisheye infrared camera 3 screen, move the center point of the interactive position to the finger touch position; save all calibration points, and perform coordinate mapping of the projection screen and the fisheye infrared camera 3 screen, and save until there is no need to calibrate the projection screen and the fisheye infrared camera 3 screen, and proceed to S2.2;

[0066] S2.2: If not required, the coordinate transformation data of the projection image and the camera image are loaded, and then the finger touch position is identified by the fisheye infrared camera 3, and the output position is obtained by coordinate transformation;

[0067] S3: Send the coordinates of the output position and gesture actions to the interactive effect software to complete the calibration.

[0068] The embodiment described in this invention is an interactive system for an inner pitching ball based on a fisheye infrared camera. The multiple concentric circles and multiple reference points projected onto the inner pitching ball can move the position coordinates of the center points of the multiple concentric circles by keyboard so that they are projected at the center position of the top of the inner pitching ball. The radius of the concentric circles can be scaled and controlled so that the reference points of the outermost circle are displayed at the appropriate position of the inner pitching ball so that they can be clearly captured by the fisheye infrared camera.

[0069] At the same time, the number of rings and the number of points in each ring in the multi-concentric circle multi-reference point type can be increased or decreased. When more rings and more points in each ring are used, the projection screen feedback position and the actual interactive position will be more accurate, but the time of the calibration process is also increased. The specific number can be reasonably set according to the size of the inner pitching sphere to balance the fast position calibration and the smaller coordinate error between the interactive position and the projection feedback position.

[0070] The method described in this embodiment can obtain a clear image of the inside of the inner pitched ball by setting the camera parameter values ​​such as brightness, contrast, gain, etc. of the fisheye infrared camera before calibrating the position of the multi-concentric circle type multi-reference points. When a finger touches the outer surface of the inner pitched ball, the image of the position where the finger touches will be displayed in the camera screen due to the irradiation of infrared light, and then the center point of the finger touch position is determined by the recognition sensing software of the inner pitched ball interactive system, which specifically includes the implementation of algorithms such as image morphological transformation, pixel brightness thresholding, and contour center point coordinate position search, such as Figure 6 shown.

[0071] When touching the outer surface of the inner pitching ball with a finger, nearby fingers may be recognized at the same time and multiple recognition points may appear, resulting in calibration failure, or a single finger touch posture may be abnormal, resulting in a deviation between the recognition center point coordinates and the set position, resulting in inconsistency between the interactive position and the feedback position. In this case, the host monitor can be connected to the computer host, and the projection screen and the host display screen can be set to a copy screen relationship. Through the visual camera screen, the recognition center point position coordinates in the camera screen can be directly fine-tuned to ensure a high degree of consistency between the interactive position and the feedback position.

[0072] The multi-concentric circle multi-reference point position calibration algorithm adopted in this embodiment is an extended application of the triangulation positioning algorithm. Specifically, the entire display area and the corresponding camera screen are decomposed into triangular areas between multiple nearest points through multi-concentric circle multi-reference points. Since each projection reference point has a corresponding camera screen recognition point, the triangular area within the camera screen corresponding to each triangle in the internal area of ​​each projection screen can be obtained. When the camera recognizes the center point position of the interactive finger, it can first determine which triangular area within the camera screen this center point position belongs to, and based on the ratio of the distance between the center point coordinates and the three vertices of the triangle, it can be inferred that the actual position within the corresponding triangle in the projection screen can be deduced, and the projection interaction feedback position information corresponding to the actual interaction position can be obtained.

[0073] Those skilled in the art will appreciate that the above are only preferred embodiments of the present invention, and the various embodiments of the present disclosure and / or the features described in the claims may be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. It is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art may still modify the technical solutions described in the aforementioned embodiments, or perform equivalent substitutions on some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

[0074] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0075] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0076] Finally, it should be noted that the above implementation modes are only used to illustrate the technical solutions of the present disclosure rather than to limit its protection scope. Although the present disclosure has been described in detail with reference to the above implementation modes, ordinary technicians in the relevant field should understand that after reading the present disclosure, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation modes of the invention, but these changes, modifications or equivalent substitutions are all within the protection scope of the disclosed claims to be approved.

Claims

1. An internal ball throwing interactive system based on a fisheye infrared camera, characterized in that: The system comprises: An inner projection ball (1), a projector (2), a fisheye infrared camera (3), a ring-shaped infrared fill light device (4) and an inner projection ball installation frame (5); The inner pitch ball (1) is arranged on the inner pitch ball mounting frame (5), and an opening is arranged at the bottom of the inner pitch ball (1); The annular infrared fill light device (4) is arranged directly below the opening of the inner ball (1) and is used to fill light inside the inner ball (1). The inner ring of the annular infrared fill light device (4) is provided with a notch for fixing the fisheye infrared camera (3); The projector (2) is arranged inside the inner projection ball installation frame (5), the lens of the projector (2) is a high-definition fisheye lens (7), and the projector (2) is used to project into the inner projection ball (1); The fisheye infrared camera (3) is used to capture images inside the inner pitching ball (1).

2. The fisheye infrared camera-based internal ball-throwing interactive system according to claim 1, characterized in that: The annular infrared fill light device (4) is provided with a plurality of infrared lamp beads.

3. According to the fisheye infrared camera-based internal ball-throwing interactive system according to claim 2, it is characterized in that: The plurality of infrared lamp beads are distributed on the outer shell of the annular infrared fill light device (4).

4. The fisheye infrared camera-based internal ball-throwing interactive system according to claim 3, characterized in that: The plurality of infrared lamp beads are arranged in a ring, and the ring is at least one circle.

5. The fisheye infrared camera-based internal ball-throwing interactive system according to claim 4, characterized in that: The circular ring has at least 2 circles, the spacing between the multiple infrared lamp beads along the tangent direction of the circular ring is 10 mm, and the spacing between the multiple infrared lamp beads along the radial direction is 10 mm.

6. The fisheye infrared camera-based internal ball-throwing interactive system according to claim 1, characterized in that: The opening of the inner pitching ball (1) is circular.

7. The fisheye infrared camera-based internal ball-throwing interactive system according to claim 6, characterized in that: The radius of the circle is not less than the outer diameter of the annular infrared fill light device (4).

8. The fisheye infrared camera-based internal ball-throwing interactive system according to claim 1, characterized in that: The interface of the fisheye infrared camera (3) is an M12 interface.

9. The fisheye infrared camera-based internal ball-throwing interactive system according to claim 1, characterized in that: An 850nm narrow-band filter is installed at the rear of the lens of the fisheye infrared camera (3).

10. The calibration method of the internal pitching interactive system is characterized in that: The internal ball pitching interactive system is an internal ball pitching interactive system based on a fisheye infrared camera as claimed in any one of claims 1 to 7, the system further comprising a computer host (6), the method is implemented by adopting a multi-concentric circle multi-reference point position calibration algorithm, and the method comprises: A projector (2) is used to project a calibration image onto an inner projection ball (1) through a high-definition fisheye lens (7) to obtain a plurality of crosshair coordinates; The calibration screen includes a plurality of crosshairs, and the plurality of crosshairs are evenly arranged; Touch the crosshairs in sequence, and use the fisheye infrared camera (3) to identify the coordinates of the center point of the contour touched by the finger each time, and obtain multiple mapping coordinates; When there is inconsistency between the crosshair coordinates and the corresponding mapping coordinates, the display screen of the computer host (6) and the projection screen of the system are set to a copy screen relationship, and the coordinates of the center point of the contour touched by the finger are adjusted on the display screen of the computer host (6).

Images