Interactive space target positioning method and device for multi-vr helmet

Abstract

The application relates to the technical field of high-precision positioning, in particular to an interactive space target positioning method and device for a multi-VR helmet, which comprises the following steps: acquiring a virtual scene image; dividing the virtual scene image into a reference frame and a work frame; acquiring corner points in each virtual scene image; acquiring corner point feature intensity of each corner point, comparing the corner point feature intensity with a first preset threshold, and acquiring a significant feature point; acquiring a significant feature point pair between the reference frame and each work frame; acquiring feature point offset of each work frame; acquiring a matching abnormal factor of each work frame; acquiring an optical flow field confusion degree of each work frame; acquiring a noise interference value of each work frame; acquiring a positioning accuracy factor of each work frame, comparing the positioning accuracy factor with a second preset threshold, acquiring a key frame in switching operation, and completing target positioning of the interactive space. The application filters out the key frame by analyzing the image quality of the work frame, thereby improving the precision of target positioning.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of high-precision positioning technology, in particular to an interactive space target positioning method and device for a multi-VR helmet. BACKGROUND

[0002] The interactive space target positioning technology of the multi-VR helmet is an innovative technology combining virtual reality (VR) and high-precision space positioning, aiming to realize motion tracking in a multi-user collaborative operation environment through multi-sensor fusion, and the core is to realize real-time interaction and natural interaction of multiple users in a virtual scene, ensuring that the user's actions and positions can be accurately perceived and fed back by the system. In high-risk operations such as switching operation training in the power industry, through the interactive space target positioning of the multi-VR helmet, trainees can practice and complete the switching operation process based on the experience of approaching the real scene, and improve their operation skills and emergency handling capabilities.

[0003] When using the multi-VR helmet for switching operation training, the traditional VR interactive positioning interaction requires the deployment of motion recognition devices and control backends, has high requirements for the site, and has the disadvantages of long deployment time and immobility. The Inside-out technology does not require additional spatial positioning devices, and the sensors of the VR device can be used for environmental perception and actual position calculation. However, the Inside-out technology has low tracking accuracy and is greatly affected by environmental lighting. When using the Inside-out technology combined with the multi-VR helmet to realize virtual switching operation practice, feature points may be lost due to complex external lighting and trainee movements, resulting in inaccurate key frame acquisition, target positioning deviation in the interactive space, and low interactive space target positioning accuracy, which prevents trainees from normally positioning targets in the interactive space and affects the normal progress of training. SUMMARY

[0004] To solve the above technical problems, the purpose of the present application is to provide an interactive space target positioning method and device for a multi-VR helmet, and the technical solution adopted is as follows:

[0005] In a first aspect, the present application provides an interactive space target positioning method for a multi-VR helmet, which includes the following steps:

[0006] Obtaining a virtual scene image in the switching operation process; selecting a frame from the virtual scene image when the switch is controlled in the switching operation process as a reference frame, and selecting the remaining frames in the switching operation process except the reference frame as operation frames;

[0007] obtaining a corner point feature intensity of each corner point according to a dispersion degree of all run lengths of all gray levels in a neighborhood window of each corner point and a difference between a gray value of each corner point and all gray values in the corresponding neighborhood window, and comparing the corner point feature intensity with a first preset threshold to obtain a significant feature point;

[0008] obtaining a significant feature point pair between the reference frame and each work frame, obtaining a feature point displacement of each work frame according to a proportion of the significant feature point pair between the reference frame and each work frame and a distance of each significant feature point pair, obtaining a matching abnormal factor of each work frame in combination with a difference in the corner point feature intensity of each significant feature point pair between the reference frame and each work frame, obtaining a light flow field disorder degree of each work frame according to a motion trajectory disorder degree of a light flow vector field between each work frame and a next work frame, and obtaining a noise interference value of each work frame in combination with an energy difference of high frequency components in a neighborhood window of all significant feature points in each work frame;

[0009] obtaining a positioning accuracy factor of each work frame according to the abnormal matching factor and the noise interference value of each work frame, comparing the positioning accuracy factor with a second preset threshold to obtain a key frame in the switching operation, and completing the target positioning of the interactive space.

[0010] Preferably, a calculation formula of the corner point feature intensity of each corner point is as follows: ; in the formula, is the corner point feature intensity of the i-th corner point; is an accumulated sum of absolute values of gray value differences between the i-th corner point and all other pixel points in the corresponding neighborhood window of the i-th corner point; is a variance of all run lengths of all gray levels in the corresponding neighborhood window of the i-th corner point.

[0011] Preferably, a specific process of obtaining the significant feature point is as follows: a corner point with a corner point feature intensity greater than or equal to the first preset threshold is recorded as a significant feature point.

[0012] Preferably, a calculation formula of the feature point displacement of each work frame is as follows: ; in the formula, is the feature point displacement of the t-th work frame, are respectively a total number of matching successful significant feature points and a total number of significant feature points between the reference frame and the t-th work frame, is an accumulated sum of Euclidean distances of all significant feature point pairs between the reference frame and the t-th work frame.

[0013] Preferably, a calculation formula of the matching abnormal factor of each work frame is as follows: ; in the formula, is the matching abnormal factor of the t-th work frame; an accumulated result of absolute values of differences between corner feature intensities of all significant feature point pairs between the reference frame and the t-th operation frame; a feature point offset of the t-th operation frame.

[0014] Preferably, the process of obtaining the optical flow field chaos degree comprises: obtaining an optical flow vector field of each operation frame; taking the optical flow vector field of a single operation frame and the optical flow vector field of the next operation frame thereof as inputs of an Anosim inter-group difference analysis algorithm to obtain a global R value and a significance level value q between the optical flow vector fields corresponding to the single operation frame and the next operation frame thereof, and taking a ratio of the global R value and the significance level value q as the optical flow field chaos degree of the single operation frame.

[0015] Preferably, the calculation formula of the noise interference value of each operation frame is: ; in the formula, the noise interference value of the t-th operation frame, an accumulated sum of absolute values of energy differences of all significant feature point corresponding all high-frequency component combinations in the t-th operation frame, the optical flow field chaos degree of the t-th operation frame; wherein the all significant feature point corresponding all high-frequency component combinations means that all high-frequency components of each significant feature point are combined two by two to obtain all high-frequency component combinations of each significant feature point.

[0016] Preferably, the calculation formula of the positioning accuracy factor of each operation frame is: ; in the formula, the positioning accuracy factor of the t-th operation frame; the matching abnormality factor of the t-th operation frame; the noise interference value of the t-th operation frame; norm() is a normalization function.

[0017] Preferably, the specific process of obtaining the key frame in the switching operation and completing the target positioning of the interactive space comprises:

[0018] taking an operation frame with a positioning accuracy factor greater than or equal to a second preset threshold value as a key frame, and taking an operation frame with a positioning accuracy factor less than the second preset threshold value as a non-key frame;

[0019] In the switching operation process of each trainee each time, all non-key frames are cut or removed, the remaining key frames are spliced into a continuous video stream in a time sequence order, and the video stream is transmitted in real time to a corresponding VR helmet display module through a transmission network for display.

[0020] In a second aspect, the embodiments of the present application further provide an interactive space target positioning device for a multi-VR helmet, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the steps of the interactive space target positioning method for the multi-VR helmet according to any one of the above embodiments when executing the computer program.

[0021] The present application has at least the following beneficial effects:

[0022] 1. The present application provides a way to obtain significant feature points by the gray level difference in the corner neighborhood range and the gray value continuity change difference condition, which can more accurately screen out feature points in the VR interactive space virtual scene that provide rich local information in the target matching positioning process, improve the robustness of the interactive space target feature points, and facilitate more accurate subsequent key frame screening, thereby improving the target positioning accuracy of the interactive space;

[0023] 2. The present application obtains a positioning accuracy factor according to the matching abnormal factor and the noise interference value, and screens the key frame to complete the VR interactive space target positioning, effectively avoiding the problem of being unable to accurately obtain the key frame caused by the loss and deviation of feature points due to complex external light and trainee movement, and more accurately dividing the key frame of the VR interactive space virtual scene of the trainee in the switching operation process, thereby reducing the risk of target positioning deviation of the trainee in the interactive space and improving the target positioning accuracy of the interactive space. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0025] Figure 1 A step flow chart of the interactive space target positioning method for the multi-VR helmet provided by an embodiment of the present application is provided.

[0026] Figure 2 An acquisition flow chart of the positioning accuracy factor of each operation frame provided by an embodiment of the present application is provided. DETAILED DESCRIPTION

[0027] For further elaboration of the technical means and effects taken by the present application to achieve the predetermined object of the application, the specific embodiments, structures, features and effects of the interactive space target positioning method and device for multiple VR headsets according to the present application are described in detail as follows in combination with the drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0029] The specific scheme of the interactive space target positioning method and device for multiple VR headsets provided by the present application is described in detail below in combination with the drawings.

[0030] Please refer to Figure 1 which shows the step flowchart of the interactive space target positioning method for multiple VR headsets provided by one embodiment of the present application, which includes the following steps:

[0031] Step one: acquire virtual scene images in the switching operation process; select any frame from the virtual scene images in the switching operation process when the switch is controlled as a reference frame, and the remaining frames in the switching operation process except the reference frame as operation frames.

[0032] The switching operation site environment of the substation is scanned by a laser radar, a laser scanner (FARO S330) or a depth camera, a high-precision three-dimensional point cloud model is constructed, the original three-dimensional point cloud data collected is denoised or smoothed, the quality of the point cloud data is improved, and a virtual scene is generated by importing Unity / Unreal engine. In another implementation of the present application, BIM modeling can also be performed on the denoised or smoothed three-dimensional point cloud data to obtain a 1:1 high-precision internal device model and an external structure restoration model.

[0033] The VR headset integrated with Inside-out tracking function is worn by the switching operation personnel to be trained, the environment is captured in real time through the binocular camera and IMU sensor on the headset, the six-degree-of-freedom (4DoF) pose is calculated by combining the SLAM algorithm, the positioning data of the multiple VR headsets is uploaded to the central server through WiFi or 5G network, and the interactive space of the trainees is synchronized through timestamp alignment and coordinate conversion.

[0034] In the obtained virtual scene, the coordinates of the trainee's switching device (switch, circuit breaker) are marked, the visual recognition of the trainee's VR headset is aligned with the VR system coordinate system in the virtual scene, and the accuracy of the device position is ensured. Multiple users operate the virtual device through handle actions and complete the switching operation training in the VR interactive space. In the above process, edge computing can also be used to reduce data transmission delay. Thus, the virtual environment of the switching operation site of the power transformer can be obtained by the above method.

[0035] All virtual scene images obtained in the switching operation are subjected to grayscale processing. In order to prevent external environmental noise interference from seriously affecting the image quality, histogram equalization is used to enhance all virtual scene images obtained in the switching operation. Grayscale processing and histogram equalization are known technologies, and the specific acquisition process will not be described in detail.

[0036] In each switching operation training process in the VR interactive space, when the trainee controls the electrical device, the trainee remains stationary for more than 1s. From all virtual scene images corresponding to the interactive space in each stationary state, any one frame is selected as a reference frame for each switching operation. All virtual scene images except the reference frame in each switching operation are recorded as operation frames for each switching operation. That is, in one switching operation process, there is one reference frame and multiple operation frames.

[0037] Step 2: Obtain the corner points in each virtual scene image; according to the dispersion degree of all run lengths of all gray levels in the neighborhood window of each corner point, and the difference between the gray value of each corner point and all gray values in the corresponding neighborhood window, obtain the corner point feature strength of each corner point, and compare it with the first preset threshold to obtain the significant feature points.

[0038] During the switching operation of the trainee by wearing the VR headset, strong light, reflection and glare caused by complex external lighting environment may cause the camera of the VR headset to fail to accurately identify the feature points. When the user is stationary, the jitter and positioning drift of the object edges in the interactive space virtual scene are obvious, which makes the trainee unable to accurately position the target in the interactive space virtual scene, affecting the normal training.

[0039] Specifically, the more serious the influence of the complex external lighting environment on the images obtained by the camera of the VR headset, the worse the positioning effect of the trainee on the target in the interactive space virtual scene, and the more serious the loss of feature points in each virtual scene image corresponding to the interactive space obtained by the VR headset. Further, when the user is stationary, the more serious the displacement of the feature points in the virtual scene image corresponding to the interactive space, and the more obvious the difference between the same feature point and the surrounding environment.

[0040] Each frame of virtual scene image obtained by the binocular camera of the VR headset is taken as an input, and all the corner point information in each frame of virtual scene image is obtained by using the Harris corner point detection algorithm. The Harris corner point detection algorithm is a known technology, and the specific process will not be described again. Taking any frame of virtual scene image as an example for analysis. Taking a window of N*N size as the neighborhood window of each corner point, N is an odd number greater than 1, the greater N is, the more accurate the feature strength evaluation result of the corner point representing in the neighborhood range of the corresponding virtual scene image in the interactive space is, and the greater the calculation amount is. In this embodiment, N is 5; a gray run matrix is obtained based on the neighborhood window of each corner point, and each run length of each gray level in the neighborhood window of each corner point is obtained through the gray run matrix. The calculation method of the gray run matrix is a known technology, and the specific process will not be described again.

[0041] As a preferred embodiment, the corner point feature strength of each corner point is obtained according to the discrete degree of all the run lengths of all the gray levels in the neighborhood window of each corner point, and the difference between the gray value of each corner point and all the gray values in the corresponding neighborhood window, for representing the uniqueness of the neighborhood range of the position of the corner point in the virtual scene of the interactive space.

[0042] In this embodiment, the corner point feature strength of the i-th corner point is denoted as , and the specific expression is: ; in the formula, is the corner point feature strength of the i-th corner point; is the cumulative sum of the absolute values of the gray value differences between the i-th corner point and all the other pixel points in the corresponding neighborhood window of the i-th corner point; is the variance of all the run lengths of all the gray levels in the corresponding neighborhood window of the i-th corner point.

[0043] The corner point feature strength reflects the difference between the corner point and the neighborhood range in the virtual scene image, the stronger the corner point feature strength is, the more distinctive the position of the corner point in the virtual scene of the interactive space can provide, and the more accurate the virtual scene target positioning accuracy can be provided in the feature point matching; represents the difference degree of the continuous change of the gray value of the i-th corner point in the neighborhood range. In the virtual scene target positioning process of the VR interactive space, the greater the difference degree is, the more local information the corner point can provide in the target matching positioning, and the more helpful it is to the target positioning and tracking in the virtual scene.

[0044] The corner point feature intensity of all corner points in all virtual scene images is taken as input, and a segmentation threshold is obtained by using the OTSU method, which is denoted as a first preset threshold. The corner point with a corner point feature intensity greater than or equal to the first preset threshold is denoted as a significant feature point. The OTSU method is a known technology, and the specific process is not described again. The beneficial effect of the present application in obtaining the significant feature point is that the corner points in the virtual scene corresponding to the VR interactive space that can provide rich local information can be better screened out, thereby improving the target positioning accuracy, and on the basis of improving the target positioning accuracy, the calculation amount is reduced, and the target positioning speed of the VR interactive space is improved.

[0045] Step three: obtaining the significant feature point pairs between the reference frame and each operation frame; obtaining the feature point offset of each operation frame according to the proportion of the significant feature point pairs between the reference frame and each operation frame and the distance of each significant feature point pair; and obtaining the matching abnormal factor of each operation frame in combination with the difference in the corner point feature intensity of each significant feature point pair between the reference frame and each operation frame; obtaining the optical flow field chaos degree of each operation frame according to the motion trajectory chaos degree of the optical flow vector field of each operation frame and the next operation frame; and obtaining the noise interference value of each operation frame in combination with the energy difference of the high-frequency components in the neighborhood window of all significant feature points in each operation frame.

[0046] The more serious the influence of the complex external light environment on the images obtained by the VR headset camera, the more serious the feature point loss phenomenon in the reference frame and the operation frame when the trainee performs the switching operation in the interactive space virtual scene, and the more obvious the feature point position offset phenomenon in the interactive space.

[0047] The significant feature points in the reference frame and the operation frame in each switching operation process of the trainee can be obtained in the above manner. Taking any switching operation process as an example, the reference frame and the operation frame in the switching operation process are analyzed. All significant feature points in the reference frame and the operation frame obtained in the switching operation process are taken as input, and the corner point feature intensity of each significant feature point is taken as a descriptor. The FLANN Matcher (Fast Library for Approximate Nearest Neighbors) feature point matching algorithm is used to obtain all significant feature point pairs between the reference frame and each operation frame. The FLANN Matcher feature point matching algorithm is a known technology, and the specific process is not described again.

[0048] Further, as a preferred implementation, the feature point offset of each operation frame is obtained according to the proportion of the significant feature point pairs between the reference frame and each operation frame and the distance of each significant feature point pair, and the matching abnormal factor of each operation frame is obtained in combination with the difference in the corner point feature intensity of each significant feature point pair between the reference frame and each operation frame, which is used to represent the matching abnormal degree of the feature points in the reference frame and each operation frame.

[0049] In this embodiment, the matching abnormality factor of the t-th job frame is denoted as , and the specific expression is: , wherein, is the matching abnormality factor of the t-th job frame; is the cumulative result of the absolute value of the difference between the corner feature intensities of all significant feature point pairs between the reference frame and the t-th job frame; is the feature point displacement of the t-th job frame. The calculation process of the feature point displacement is: , wherein, is the feature point displacement of the t-th job frame, are the total number of matching successful significant feature points and the total number of significant feature points between the reference frame and the t-th job frame, respectively, is the cumulative sum of the Euclidean distances of all significant feature point pairs between the reference frame and the t-th job frame.

[0050] The matching abnormality factor represents the degree of abnormality in the matching of feature points in the reference frame and the job frame due to the influence of complex external light during the switching operation. The feature point displacement reflects the position displacement of the significant feature points between the reference frame and the job frame. When the target positioning in the interactive space virtual scene is more affected by external light, the local information difference in the neighborhood range of the matching successful feature points is greater, i.e., the value of the index is greater. The matching success rate of the significant feature points in the reference frame and the job frame is smaller, and the displacement distance of the matching successful feature points is greater, i.e., the value of the index is greater.

[0051] There are still some drawbacks in screening the key frame of target positioning in the interactive space only by the matching abnormality factor between the reference frame and each job frame obtained in the above-mentioned electrical equipment switching operation process. That is, complex external light and the motion of the trainee in the interactive space virtual scene will produce random noise points, which may be misjudged as significant feature points, and may cause the actual target positioning accuracy in the interactive space virtual scene to be high while the matching abnormality factor between the reference frame and the job frame is large, affecting the target positioning accuracy in the interactive space.

[0052] Specifically, when the job frame image is more seriously disturbed by random noise points caused by complex external light environment and the motion of the trainee during target positioning, the high-frequency energy difference condition in the neighborhood range of the significant feature points in the job frame is more obvious, and the optical flow field estimation motion trajectory between adjacent frames is more chaotic.

[0053] Further, all pixel gray scale values in the neighborhood window of each salient feature point in each work frame are arranged in a sequence in the order from left to right and from top to bottom, all the sequences are taken as inputs of FFT fast Fourier transform respectively, frequency components of pixel gray scale values in the neighborhood range of each salient feature point in each work frame are obtained, frequency components of pixel gray scale values in the neighborhood range of each salient feature point are arranged in descending order, and the first 5% of the frequency components are selected as high frequency components of each salient feature point. All high frequency components of each salient feature point are combined in pairs to obtain all high frequency component combinations of each salient feature point. Each work frame image is taken as an input of the Farneback dense optical flow field algorithm to obtain an optical flow vector field of each work frame. Since the FFT fast Fourier transform and the Farneback dense optical flow field algorithm are both known technologies, the specific obtaining process is not described in detail.

[0054] Further, as a preferred embodiment, the noise point interference value of each work frame is obtained according to the motion trajectory confusion degree of the optical flow vector field of each work frame and the next work frame, and the energy difference value of the high frequency components in the neighborhood window of all salient feature points in each work frame is combined to obtain the noise point interference value of each work frame, which is used to represent the interference degree of the salient feature point matching of the interactive space virtual scene image of the trainee in each switching operation process by random noise points.

[0055] In this embodiment, the noise point interference value of the tthwork frame is denoted as , and the specific expression is: ; in the formula, is the noise point interference value of the tthwork frame, is the cumulative sum of the absolute values of the energy difference values of all high frequency component combinations corresponding to all salient feature points in the tthwork frame, is the optical flow field confusion degree of the tthwork frame, which is used to represent the confusion degree of the motion trajectory of the optical flow vector field of the tthwork frame and the next work frame. Specifically, in this embodiment, the optical flow field confusion degree is obtained by taking the optical flow vector field of the tthwork frame and the t+1thwork frame as inputs of the Anosim intergroup difference analysis algorithm, obtaining the global R value and the significance level value q between the optical flow vector fields corresponding to the tthwork frame and the t+1thwork frame, and taking the ratio of the global R value and the significance level value q as the optical flow field confusion degree of the tthwork frame, denoted as . When is larger, it means that the difference between the optical flow vector fields corresponding to the tthwork frame and the t+1thwork frame is larger, and the motion trajectory of the optical flow vector field is more chaotic. The Anosim intergroup difference analysis algorithm is a known technology, and the specific process is not described in detail. It should be noted that when there is no work frame after the tthwork frame, the noise point interference value of the tthwork frame is not calculated.

[0056] Noise point interference value The greater the value of the matching abnormality factor of the t-th frame of the job frame, the greater the degree of interference of the matching of the feature points in the t-th frame of the job frame by random noise. The greater the value of the high-frequency component chaos degree of the t-th frame of the job frame, the greater the energy difference between the high-frequency components in the neighborhood of the salient feature points in the job frame, and the more serious the interference of the job frame by random noise in the target positioning process; and the greater the value of the optical flow field chaos degree of the t-th frame of the job frame, the greater the difference between the optical flow vector fields of the image of the interactive space virtual scene corresponding to the t-th frame of the job frame and the next frame of the job frame, and the more serious the interference of the job frame by random noise in the target positioning process. The greater the value of the high-frequency component chaos degree of the t-th frame of the job frame, the greater the energy difference between the high-frequency components in the neighborhood of the salient feature points in the job frame, and the more serious the interference of the job frame by random noise in the target positioning process; and the greater the value of the optical flow field chaos degree of the t-th frame of the job frame, the greater the difference between the optical flow vector fields of the image of the interactive space virtual scene corresponding to the t-th frame of the job frame and the next frame of the job frame, and the more serious the interference of the job frame by random noise in the target positioning process.

[0057] Step four: obtaining a positioning accuracy factor of each frame of the job frame according to the abnormal matching factor and the noise interference value of each frame of the job frame, and comparing the positioning accuracy factor with a second preset threshold value to obtain a key frame in the switching job and complete the target positioning of the interactive space.

[0058] Further, when the matching abnormality factor of the job frame is smaller and the noise interference value is lower, it indicates that the target positioning in the VR interactive space virtual scene corresponding to the job frame is more accurate, and the job frame should not be used as a key frame for target positioning.

[0059] As a preferred embodiment, a positioning accuracy factor of each frame of the job frame is constructed according to the matching abnormality factor and the noise interference value of each frame of the job frame, and is used to represent the positioning accuracy of each frame of the job frame as a key frame for target positioning of the interactive space virtual scene. The flow chart for obtaining the positioning accuracy factor of each frame of the job frame is shown in FIG. 4. Figure 2

[0060] In this embodiment, the positioning accuracy factor of the t-th frame of the job frame is denoted as , and the specific expression is as follows: ; in the formula, is the positioning accuracy factor of the t-th frame of the job frame; is the matching abnormality factor of the t-th frame of the job frame; is the noise interference value of the t-th frame of the job frame; and norm() is a normalization function, so that the value of is in the range of [0, 1].

[0061] When is greater, it indicates that the matching of the feature points between the t-th frame of the job frame and the reference frame is better, the interference by random noise is lighter, the target positioning in the interactive space virtual scene is more accurate, and therefore the t-th frame of the job frame should be used as a key frame for positioning the target in the interactive space virtual scene.

[0062] ​A second preset threshold Y is set, and during each switching operation of the trainee, an operation frame with a positioning accuracy factor greater than or equal to the second preset threshold is taken as a key frame, and an operation frame with a positioning accuracy factor less than the second preset threshold is taken as a non-key frame. In this embodiment, the second preset threshold Y is 0.6, and the implementer can set the size of the second preset threshold according to the actual situation.

[0063] During each switching operation of the trainee, all non-key frames are cut or removed, or key frame features are propagated using a TSDF model to generate interpolation frames to replace the original non-key frames. In this embodiment, all non-key frames are removed using the former, and the remaining key frames are spliced into a continuous video stream in chronological order, which is transmitted in real time to all VR headsets worn by the trainees through a WiFi or 5G network for display.

[0064] At this point, the interactive space target positioning method for multiple VR headsets can be realized in the above manner.

[0065] Based on the same inventive concept as the above method, the embodiments of the present application also provide an interactive space target positioning device for multiple VR headsets, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the steps of the interactive space target positioning method for multiple VR headsets in any one of the above methods when executing the computer program.

[0066] It should be noted that the above sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. The above describes specific embodiments of the present specification. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are also possible or can be advantageous.

[0067] Each embodiment in the specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other, and each embodiment mainly describes the differences from other embodiments.

[0068] The above is only the preferred embodiment of the present application, and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for interactive space target positioning for multi-VR headsets, characterized in that, The method comprises the following steps: Obtaining virtual scene images in the switching operation process; selecting a frame from the virtual scene images in the switching operation process as a reference frame when the switch is controlled, and taking the remaining frames in the switching operation process except the reference frame as operation frames; Obtaining corner points in each virtual scene image; obtaining the corner point feature strength of each corner point according to the dispersion degree of all run lengths of all gray levels in the neighborhood window of each corner point and the difference between the gray value of each corner point and all gray values in the corresponding neighborhood window, and comparing the corner point feature strength with a first preset threshold to obtain a significant feature point; Obtaining the significant feature point pairs between the reference frame and each operation frame; obtaining the feature point displacement of each operation frame according to the proportion of the significant feature point pairs between the reference frame and each operation frame and the distance of each significant feature point pair; obtaining the matching abnormality factor of each operation frame in combination with the difference in the corner point feature strength of each significant feature point pair between the reference frame and each operation frame; obtaining the optical flow field chaos degree of each operation frame according to the motion trajectory chaos degree of the optical flow vector field of each operation frame and the next operation frame; and obtaining the noise interference value of each operation frame in combination with the energy difference of the high-frequency components in the neighborhood window of all significant feature points in each operation frame; Obtaining the positioning accuracy factor of each operation frame according to the abnormal matching factor and the noise interference value of each operation frame, comparing the positioning accuracy factor with a second preset threshold to obtain the key frame in the switching operation, and completing the target positioning of the interactive space.

2. The interactive space target positioning method for multi-VR headsets of claim 1, wherein, The calculation formula of the corner point feature strength of each corner point is: ; wherein, is the corner point feature strength of the i th corner point; is the absolute value of the gray value difference between the i th corner point and all other pixel points in the corresponding neighborhood window of the i th corner point; is the variance of all run lengths of all gray levels in the corresponding neighborhood window of the i th corner point.

3. The interaction space target positioning method for multi-VR headsets of claim 1, wherein, The specific process of obtaining the significant feature point is that the corner point with a corner point feature strength greater than or equal to the first preset threshold is recorded as a significant feature point.

4. The interaction space target positioning method for multi-VR headsets of claim 1, wherein, The calculation formula of the feature point offset of each operation frame is: ; in the formula, is the feature point offset of the tth operation frame, is the total number of matching successful significant feature points between the reference frame and the tth operation frame, and is the total number of significant feature points, is the cumulative sum of the Euclidean distances of all significant feature point pairs between the reference frame and the tth operation frame.

5. The interaction space target positioning method for multi-VR headsets of claim 1, wherein, The calculation formula of the matching abnormality factor of each operation frame is: ; in the formula, is the matching abnormality factor of the tth operation frame; is the cumulative result of the absolute value of the difference between the corner feature intensities of all the significant feature point pairs between the reference frame and the tth operation frame; is the feature point offset of the tth operation frame.

6. The interaction space target positioning method for multi-VR headsets of claim 1, wherein, The process of obtaining the optical flow field chaos degree is that the optical flow vector field of each operation frame is obtained; the optical flow vector fields of a single operation frame and the next operation frame are taken as the input of the Anosim inter-group difference analysis algorithm to obtain the global R value and the significance level value q between the corresponding optical flow vector fields of the single operation frame and the next operation frame, and the ratio of the global R value and the significance level value q is taken as the optical flow field chaos degree of the single operation frame.

7. The interaction space target positioning method for multi-VR headsets of claim 1, wherein, The calculation formula of the noise interference value of each operation frame is: ; wherein, is the noise interference value of the tth operation frame, is the absolute value of the energy difference of all significant feature point corresponding to all high frequency component combinations in the tth operation frame, is the optical flow field confusion degree of the tth operation frame; wherein, the all significant feature point corresponding to all high frequency component combinations means that all high frequency components of each significant feature point are combined two by two to obtain all high frequency component combinations of each significant feature point.

8. The interaction space target positioning method for multi-VR headsets of claim 1, wherein, The calculation formula of the positioning accuracy factor of each job frame is: ; wherein, is the positioning accuracy factor of the tth job frame; is the matching abnormality factor of the tth job frame; is the noise interference value of the tth job frame; and norm() is a normalization function.

9. The interaction space target positioning method for multi-VR headsets of claim 1, wherein, The specific process of obtaining the key frame in the switching operation and completing the target positioning of the interactive space is that: The operation frame with a positioning accuracy factor greater than or equal to the second preset threshold is taken as a key frame, and the operation frame with a positioning accuracy factor less than the second preset threshold is taken as a non-key frame; In the switching operation process of each trainee, all non-key frames are cut or removed, the remaining key frames are spliced into a continuous video stream in chronological order, and the video stream is transmitted in real time to the corresponding VR helmet display module through a transmission network for display.

10. An interactive space targeting device for a multi-VR headset, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, The processor executes the computer program to realize the steps of the interactive space target positioning method for multiple VR helmets according to any one of claims 1-9.

Images