Method for eliminating mirror reflection interference in VR positioning, hardware and application

Abstract

The invention relates to a method for eliminating mirror reflection interference in VR positioning, hardware and application, and the method comprises the steps: detecting and matching a highlight area in a binocular camera image, calculating a corresponding 3D position through triangulation, carrying out the consistency analysis of the parallax and depth of the highlight area, recognizing inconsistent points as mirror reflection points, and removing the mirror reflection points. The mirror surface reflection interference is eliminated; hardware is realized by the method, and the method is applied to a VR environment with an illumination intensity gradient greater than a preset value. The method detects and matches the highlight area in the binocular camera image, calculates the 3D position, analyzes the consistency of the parallax and depth of the highlight area, effectively recognizes and rejects the specular reflection points, reduces the SLAM positioning instability, improves the VR positioning precision, improves the mapping quality, accelerates the convergence speed of algorithm calculation while eliminating the external point interference, and accelerates the calculation efficiency. The calculation consumption of the system is reduced; the method is implemented only on the basis of original configuration, hardware is simple, and implementation cost is low.

Description

Technical Field

[0001] The present invention relates to the technical field of calculation, extrapolation or counting, and in particular to a method, hardware and application for eliminating mirror reflection interference in VR positioning. Background Art

[0002] With the rapid development of the leisure and entertainment industry, numerous terminal simulators for virtual reality environments and augmented reality environments have emerged to meet users' needs for a stronger sense of control and immersion.

[0003] SLAM (Simultaneous Localization and Mapping) positioning refers to the process in which a robot can simultaneously locate itself and build an environmental map through sensor data when moving from any position in an unknown environment. The technology allows the robot to continuously update its position information during movement and build an incremental map based on this information, thereby achieving autonomous navigation and positioning. SLAM positioning is used in virtual reality devices, which use binocular cameras or multi-camera cameras to simultaneously obtain 3D information and depth information of the surrounding environment, thereby achieving precise positioning.

[0004] However, in actual applications, high-brightness interference objects such as mirror reflections will generate false 3D points in the image, resulting in positioning errors, which in turn affects the stability and accuracy of the SLAM system. It is difficult to distinguish the self-luminous area of ​​a real object from the high-brightness area caused by mirror reflections by relying solely on traditional feature matching and depth calculation methods, which will affect the positioning accuracy and ultimately affect the immersion effect in a virtual reality environment.

[0005] In the prior art, some technical solutions attempt to solve the above problems by installing additional sensors such as lidar and structured light, or by filtering polarized light through lens coating, but these solutions all have problems such as complex hardware configuration and poor imaging effect. Summary of the invention

[0006] The present invention solves the problems existing in the prior art and provides a method, hardware and application for eliminating mirror reflection interference in VR positioning, thereby eliminating the interference of high-brightness reflection spots of mirror reflection on the positioning of the SLAM system of the virtual reality device and improving the positioning accuracy.

[0007] The technical solution adopted by the present invention is a method for eliminating mirror reflection interference in VR positioning. The method detects and matches highlight areas in a binocular camera image, calculates the corresponding 3D position using triangulation, performs consistency analysis on the parallax and depth of the highlight area, identifies inconsistent points as mirror reflection points, removes the mirror reflection points, and eliminates mirror reflection interference.

[0008] Preferably, the method comprises the following steps: S1 obtains the feature points of the binocular camera and obtains its corresponding 3D position; S2 analyzes the consistency of the 3D position of each set of feature points; S3 identifies and removes specular reflection points based on consistency analysis; S4 outputs the filtered 3D coordinate points.

[0009] Preferably, S1 comprises the following steps: S1.1 Detect and match feature points in the left camera and right camera images of the binocular camera respectively; S1.2 Calculate the 3D position of each matching point by triangulation based on the external and internal parameters of the stereo camera.

[0010] Preferably, the depth Z = f·T / (x L -x R ), where f is the focal length of the camera, T is the baseline distance of the binocular camera, and x L and x R are the horizontal positions of the matching points in the left and right images, respectively.

[0011] Preferably, in S2, analyzing the consistency of the 3D position of each group of feature points includes analyzing depth consistency and calculating parallax consistency.

[0012] Preferably, the 3D position information is calculated by acquiring images of multiple angles in the same scene. If the standard deviation of the depth is less than a preset value, there is no mirror reflection interference. Otherwise, outliers are extracted as mirror reflection points.

[0013] Preferably, the parallax of matching points of images of multiple angles in the same scene is obtained, and if the parallax variation is greater than a preset value, then there is a mirror reflection point in this pair of matching points.

[0014] A computer-readable storage medium stores a program for eliminating mirror reflection interference in VR positioning. When the program is executed by a processor, the method for eliminating mirror reflection interference in VR positioning is implemented.

[0015] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for eliminating mirror reflection interference in VR positioning is implemented.

[0016] An application of the method for eliminating mirror reflection interference in VR positioning is applied to a VR environment where the light intensity gradient is greater than a preset value.

[0017] The present invention relates to a method, hardware and application for eliminating mirror reflection interference in VR positioning. The method detects and matches highlight areas in a binocular camera image, calculates the corresponding 3D position by triangulation, performs consistency analysis on the parallax and depth of the highlight area, identifies inconsistent points as mirror reflection points, removes the mirror reflection points, and eliminates mirror reflection interference. The method is used to implement hardware and is applied to a VR environment where the light intensity gradient is greater than a preset value.

[0018] The beneficial effects of the present invention are that by detecting and matching highlight areas in the binocular camera image, calculating the 3D position using triangulation, and performing consistency analysis on the parallax and depth of the highlight area, the mirror reflection points can be effectively identified and eliminated, the instability of SLAM positioning can be reduced, the accuracy of VR positioning can be improved, the quality of map construction can be improved, the convergence speed of the algorithm calculation can be accelerated while eliminating external point interference, the computing efficiency can be increased, and the computing consumption of the system can be reduced; there is no need to rely on additional sensors, and it is only implemented on the original configuration, the hardware is simple, and the implementation cost is low. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0020] The present invention is further described in detail below in conjunction with embodiments, but the protection scope of the present invention is not limited thereto.

[0021] The present invention relates to a method for eliminating mirror reflection interference in VR positioning. The method detects and matches highlight areas in a binocular camera image, calculates the corresponding 3D position by triangulation, performs consistency analysis on the parallax and depth of the highlight area, identifies inconsistent points as mirror reflection points, removes the mirror reflection points, and eliminates mirror reflection interference.

[0022] Regarding the "highlight area", the present invention provides two definition methods: (1) In a given neighborhood, if the brightness value of a pixel is higher than that of its surrounding pixels, then this pixel and its vicinity can be defined as part of the highlight area; specifically, (1-1) Define the neighborhood Select an appropriate neighborhood size, such as 3x3, 5x5, etc. The neighborhood can be square, circular or any shape, depending on the needs of the specific application; in this embodiment, a rectangular window around the center pixel is used; (1-2) Compare brightness values For each pixel, check whether it is a local maximum in its neighborhood, that is, whether the brightness value of the pixel is strictly greater than the brightness values ​​of all other pixels in its neighborhood; (1-3) Determine the highlighted area If a pixel is a local maximum in its neighborhood, it can be considered as part of the highlight area; usually, not only the pixel itself is considered to be highlighted, but also its nearby pixels can be included to form a continuous highlight area, such as its adjacent corner points; (2) Using numerical definitions In a grayscale image, highlight areas can be simply defined as those areas whose pixel values ​​exceed a certain threshold. For example, in an 8-bit grayscale image, if the threshold is set to 200, then all areas with pixel values ​​greater than or equal to 200 can be considered highlight areas.

[0023] The above two definitions of highlight areas are both desirable, and the specific application is adjusted based on the actual needs of the technicians.

[0024] The method is described below in conjunction with specific steps, and the method comprises the following steps: S1 obtains the feature points of the binocular camera and obtains its corresponding 3D position; S1 includes the following steps: S1.1 Detect and match feature points in the left and right camera images of the binocular camera; the corresponding 2D points X and X' are expressed as X = (x L , y L ) and X'=( x R , y R ); S1.2 Based on the external and internal parameters of the binocular camera, the external parameters include rotation and translation matrices, and the internal parameters include focal length, optical center, etc., and the 3D position of each matching point is calculated by triangulation.

[0025] Depth Z = f·T / (x L -x R ), where f is the focal length of the camera, T is the baseline distance of the binocular camera, and x L and x R are the horizontal positions of the matching points in the left and right images respectively; In the present invention, x is defined as L -x R Parallax refers to the difference in position of the same point in the same scene in the left and right images. More specifically, for a point in the real world, it is projected to a pixel position in a certain row in the left image, and in the right image the point will be projected to a different pixel position in the same row. The horizontal distance between these two pixel positions is the parallax.

[0026] Considering that the 3D position of the specular reflection point depends on the angle of the reflected light and the change of the viewing angle, its position in the 3D space is not consistent. S2 analyzes the consistency of the 3D position of each set of feature points; Further, analyzing the consistency of the 3D position of each set of feature points includes analyzing the depth consistency and calculating the disparity consistency, and S3 identifies and removes specular reflection points based on consistency analysis; When analyzing depth consistency, the same scene photographed by the binocular camera is captured from multiple angles, and the 3D depth of the highlight point is calculated. If the point is a self-luminous source on a real object, the 3D depth should be basically consistent under multiple perspectives, but if it is a mirror reflection, its depth may fluctuate dramatically with the change of perspective; by setting a threshold for depth change, points with large depth changes can be eliminated or the weight of these points in the optimizer can be reduced; in the specific implementation process, 3D position information is calculated by obtaining images of multiple angles of the same scene. If the standard deviation of the depth is less than the preset value, it means that there are no outliers in the current part of the points, and they are evenly distributed in a certain space without mirror reflection interference. Otherwise, the outliers are extracted as mirror reflection points; the extraction of outliers is content that is easy for technical personnel in this field to understand.

[0027] When analyzing and calculating parallax consistency, the parallax of matching points is calculated in multiple perspectives, that is, the horizontal distance difference between the left and right cameras. The parallax of real objects should remain relatively consistent in multiple perspectives, while the parallax of mirror reflection points is unstable, and can be judged in the following way: Parallax change = |parallax angle 1 - parallax angle 2| The parallax perspective 1 and parallax perspective 2 here actually refer to the parallax under perspective 1 and perspective 2 respectively; if the parallax change exceeds a certain set threshold, it can be considered that the point may be a mirror reflection point.

[0028] S4 outputs the filtered 3D coordinate points.

[0029] The present invention also relates to a computer-readable storage medium, on which is stored a program for eliminating mirror reflection interference in VR positioning. When the program is executed by a processor, the method for eliminating mirror reflection interference in VR positioning is implemented.

[0030] The present invention also relates to a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for eliminating mirror reflection interference in VR positioning is implemented.

[0031] The present invention also relates to an application of the method for eliminating mirror reflection interference in VR positioning, which is applied to a VR environment in which the light intensity gradient is greater than a preset value.

[0032] In the present invention, the illumination intensity gradient refers to the average nonlinear change from one area to another, which is used to describe the difference in illumination intensity between an area and its adjacent area. If the illumination intensity is stronger than that of the adjacent area, it is positive, otherwise it is negative. When the illumination intensity gradient is greater than a preset value, it means that the illumination intensity of at least one area is large enough to correspond to reflection, specular reflection, etc.

[0033] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0034] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0035] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0036] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0037] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0038] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for eliminating mirror reflection interference in VR positioning, characterized in that: The method detects and matches highlight areas in a binocular camera image, calculates their corresponding 3D positions using triangulation, performs consistency analysis on the parallax and depth of the highlight areas, identifies inconsistent points as mirror reflection points, removes the mirror reflection points, and eliminates mirror reflection interference.

2. A method for eliminating mirror reflection interference in VR positioning according to claim 1, characterized in that: The method comprises the following steps: S1 obtains the feature points of the binocular camera and obtains its corresponding 3D position; S2 analyzes the consistency of the 3D position of each set of feature points; S3 identifies and removes specular reflection points based on consistency analysis; S4 outputs the filtered 3D coordinate points.

3. A method for eliminating mirror reflection interference in VR positioning according to claim 2, characterized in that: S1 includes the following steps: S1.1 Detect and match feature points in the left camera and right camera images of the binocular camera respectively; S1.2 Calculate the 3D position of each matching point by triangulation based on the external and internal parameters of the stereo camera.

4. A method for eliminating mirror reflection interference in VR positioning according to claim 3, characterized in that: Depth Z = f·T / (x L -x R ), where f is the focal length of the camera, T is the baseline distance of the binocular camera, and x L and x R are the horizontal positions of the matching points in the left and right images, respectively.

5. A method for eliminating mirror reflection interference in VR positioning according to claim 2, characterized in that: In S2, analyzing the consistency of the 3D position of each set of feature points includes analyzing the depth consistency and calculating the disparity consistency.

6. A method for eliminating mirror reflection interference in VR positioning according to claim 5, characterized in that: The 3D position information is calculated by obtaining images from multiple angles of the same scene. If the standard deviation of the depth is less than the preset value, there is no mirror reflection interference. Otherwise, the outlier points are extracted as mirror reflection points.

7. A method for eliminating mirror reflection interference in VR positioning according to claim 5, characterized in that: The parallax of the matching points of images from multiple angles in the same scene is obtained. If the parallax change is greater than a preset value, there is a mirror reflection point in this pair of matching points.

8. A computer-readable storage medium, characterized in that: A program for eliminating mirror reflection interference in VR positioning is stored thereon, and when the program is executed by a processor, the method for eliminating mirror reflection interference in VR positioning as described in one of claims 1 to 7 is implemented.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements the method for eliminating mirror reflection interference in VR positioning as described in one of claims 1 to 7.

10. An application of the method for eliminating mirror reflection interference in VR positioning according to any one of claims 1 to 7, characterized in that: Applicable to VR environments where the light intensity gradient is greater than the preset value.

Images