Interactive space target positioning method and device for multiple VR helmets

Abstract

The invention relates to the technical field of high-precision positioning, in particular to an interactive space target positioning method and device for multiple VR helmets, and the method comprises the steps: obtaining a virtual scene image; dividing the virtual scene image into a reference frame and an operation frame; obtaining angular points in each virtual scene image; obtaining the corner feature intensity of each corner, and comparing the corner feature intensity with a first preset threshold value to obtain a significant feature point; obtaining a significant feature point pair between the reference frame and each operation frame; obtaining feature point offset of each operation frame; obtaining a matching abnormal factor of each operation frame; obtaining the optical flow field confusion degree of each operation frame; obtaining a noise interference value of each operation frame; and obtaining a positioning accuracy factor of each operation frame, comparing the positioning accuracy factor with a second preset threshold value, obtaining a key frame in switching operation, and completing target positioning of the interaction space. According to the method, the key frame is screened out by analyzing the image quality of the operation frame, so that the target positioning precision is improved.

Description

Technical Field

[0001] This application relates to the field of high-precision positioning technology, and particularly to an interactive space target positioning method and device for multiple VR headsets. Background Art

[0002] The interactive space target positioning technology for multiple VR headsets is an innovative technology that combines virtual reality (VR) and high-precision spatial positioning. Its aim is to achieve action tracking in a multi-person collaborative operation environment through multi-sensor fusion. The core lies in enabling real-time interaction and natural interaction among multiple users in a virtual scene, ensuring that the actions and positions of users can be accurately sensed and fed back by the system. In high-risk operations in the power industry such as switching operation training, through the interactive space target positioning of multiple VR headsets, trainees can practice and complete the entire switching operation process on the basis of experiencing a near-real scenario, thereby improving their operation skills and emergency handling capabilities.

[0003] When using multiple VR headsets for switching operation training, traditional VR interactive positioning requires the deployment of action recognition devices and control backends, which have high requirements for the venue and have drawbacks such as long deployment time and immobility. The Inside-out technology does not require additional spatial positioning devices, and the sensors of the VR devices themselves can be used to perceive the environment and calculate the actual position. However, the Inside-out technology has low tracking accuracy and is greatly affected by environmental light. When using the Inside-out technology in combination with multiple VR headsets to achieve virtual switching operation training, due to complex external light and the movement of trainees, feature points are lost, resulting in the inability to accurately obtain key frames, causing target positioning deviation in the interactive space and the drawback of low accuracy of interactive space target positioning, which makes it impossible for trainees to normally locate targets in the interactive space and affects the normal progress of training. Summary of the Invention

[0004] To solve the above technical problems, the purpose of this application is to provide an interactive space target positioning method and device for multiple VR headsets, and the specific technical solutions adopted are as follows: In the first aspect, an embodiment of this application provides an interactive space target positioning method for multiple VR headsets, and the method includes the following steps: Obtain virtual scene images during the switching operation process; select any frame from the virtual scene images when controlling the switch during the switching operation process as the reference frame, and use the remaining frames other than the reference frame during the switching operation process as the operation frames; Obtain the corner points in each virtual scene image; according to the degree of dispersion of all run lengths of all gray levels within the neighborhood window of each corner point, and the difference between the gray value of each corner point and all gray values within the corresponding neighborhood window, obtain the corner feature intensity of each corner point, compare it with the first preset threshold, and obtain the significant feature points; Obtain the pairs of significant feature points between the reference frame and each working frame; according to the proportion of the pairs of significant feature points between the reference frame and each working frame and the distances of each pair of significant feature points, obtain the feature point offset of each working frame; and combine the difference in the corner feature intensity of each pair of significant feature points between the reference frame and each working frame to obtain the matching anomaly factor of each working frame; according to the degree of chaos of the motion trajectories of the optical flow vector fields between each working frame and the next working frame, obtain the optical flow field chaos of each working frame; and combine the energy difference of the high-frequency components within the neighborhood windows of all significant feature points in each working frame to obtain the noise interference value of each working frame; According to the anomaly matching factor and the noise interference value of each working frame, obtain the positioning accuracy factor of each working frame, compare it with the second preset threshold, obtain the key frames in the switching operation, and complete the target positioning of the interaction space.

[0005] Preferably, the calculation formula for the corner feature intensity of each corner point is: ; where is the corner feature intensity of the i-th corner point; is the cumulative sum of the absolute values of the differences in gray values between the i-th corner point and all other pixel points in its corresponding neighborhood window; is the variance of all run lengths of all gray levels in the neighborhood window corresponding to the i-th corner point.

[0006] Preferably, the specific process of obtaining the significant feature points is: Denote the corner points with corner feature intensity greater than or equal to the first preset threshold as significant feature points.

[0007] Preferably, the calculation formula for the feature point offset of each working frame is: ; where is the feature point offset of the t-th working frame, are respectively the total number of successfully matched significant feature points and the total number of significant feature points between the reference frame and the t-th working frame, is the cumulative sum of the Euclidean distances of all pairs of significant feature points between the reference frame and the t-th working frame.

[0008] Preferably, the calculation formula for the matching anomaly factor of each working frame is: ; where is the matching anomaly factor of the t-th working frame; is the cumulative result of the absolute values of the differences between the corner feature intensities of all pairs of significant feature points between the reference frame and the t-th working frame; is the feature point offset of the job frame at the t-th frame.

[0009] Preferably, the process of obtaining the optical flow field chaos degree is as follows: obtaining the optical flow vector field of each job frame; using the optical flow vector fields of a single job frame and its next job frame as the input of the Anosim between-group difference analysis algorithm, obtaining the global R value and the significance level value q between the corresponding optical flow vector fields of the single job frame and its next job frame, and taking the ratio of the global R value to the significance level value q as the optical flow field chaos degree of the single job frame.

[0010] Preferably, the calculation formula for the noise interference value of each job frame is: ; where is the noise interference value of the job frame at the t-th frame, is the sum of the absolute values of the energy differences of all high-frequency component combinations corresponding to all significant feature points in the job frame at the t-th frame, is the optical flow field chaos degree of the job frame at the t-th frame; where, all high-frequency component combinations corresponding to all significant feature points refer to all high-frequency component combinations obtained by pairwise combining all high-frequency components of each significant feature point.

[0011] Preferably, the calculation formula for the positioning accuracy factor of each job frame is: ; where is the positioning accuracy factor of the job frame at the t-th frame; is the matching anomaly factor of the job frame at the t-th frame; is the noise interference value of the job frame at the t-th frame; norm( ) is the normalization function.

[0012] Preferably, the specific process of obtaining the key frame in the switching operation and completing the target positioning of the interaction space is as follows: Taking the job frame with the positioning accuracy factor greater than or equal to the second preset threshold as the key frame, and taking the job frame with the positioning accuracy factor less than the second preset threshold as the non-key frame; During each switching operation of each trainee, cutting or removing all non-key frames, splicing the remaining key frames into a continuous video stream in chronological order, and transmitting the video stream to the corresponding VR helmet display module for display in real time through the transmission network.

[0013] In a second aspect, the embodiments of the present application further provide an interactive space target positioning device for multiple VR helmets, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of the above-mentioned interactive space target positioning method for multiple VR helmets are implemented.

[0014] The present application has at least the following beneficial effects: 1. The present application provides a method for obtaining significant feature points through the gray - scale difference in the corner neighborhood range and the change difference in gray - scale value continuity, which can more accurately screen out the feature points that provide rich local information in the virtual scene of the VR interaction space during the target matching and positioning process, improve the robustness of the target feature points in the interaction space, and is beneficial to more accurately screening key frames subsequently, thereby improving the target positioning accuracy in the interaction space; 2. The present application obtains a positioning accuracy factor based on the matching anomaly factor and the noise interference value, and uses this to screen key frames to complete the target positioning in the VR interaction space, effectively avoiding the problem of inaccurate acquisition of key frames caused by the loss and deviation of feature points due to complex external lighting and the movement of the trainee. It can more accurately divide the key frames of the virtual scene in the VR interaction space during the switching operation process of the trainee, reduce the risk of target positioning deviation of the trainee in the interaction space, and improve the target positioning accuracy in the interaction space. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following - described drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0016] Figure 1 It is a flowchart of the steps of a method for target positioning in the interaction space for multiple VR helmets provided in an embodiment of the present application; Figure 2 It is a flowchart for obtaining the positioning accuracy factor of each operation frame provided in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] In order to further elaborate on the technical means and effects adopted by the present application to achieve the intended invention purpose, the following, in combination with the drawings and preferred embodiments, details the specific implementation manners, structures, features, and effects of the method and device for target positioning in the interaction space for multiple VR helmets proposed according to the present application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which this application belongs.

[0019] The specific scheme of the interactive space target positioning method and device for multiple VR helmets provided by the present application is described in detail below with reference to the accompanying drawings.

[0020] See also Figure 1 , which shows a flowchart of a method for interactive space target positioning for multiple VR helmets provided by an embodiment of the present application, the method comprising the following steps: Step 1: Acquire a virtual scene image during the switching operation; select any one frame from the virtual scene image when the switch is controlled during the switching operation as a reference frame, and use the remaining frames except the reference frame during the switching operation as operation frames.

[0021] Scan the substation switching operation site environment through lidar, laser scanner (FARO S330) or depth camera, build a high-precision 3D point cloud model, denoise or smooth the collected original 3D point cloud data, improve the point cloud data quality, and import it into Unity / Unreal engine to generate a virtual scene. In another implementation of this application, BIM modeling can also be performed through the denoised or smoothed 3D point cloud data to obtain a 1:1 high-precision internal equipment model and external structure restoration model.

[0022] The switching operators to be trained are equipped with VR helmets with integrated inside-out tracking functions. The binocular camera and IMU sensor on the helmet can capture the environment in real time, and the six-degree-of-freedom (4DoF) posture is calculated using the SLAM algorithm. The positioning data of multiple VR helmets are uploaded to the central server via WiFi or 5G networks, and the interactive space synchronization of the trainees is achieved through timestamp alignment and coordinate transformation.

[0023] Mark the coordinates of the trainee's switching equipment (switches, circuit breakers) in the virtual scene obtained above, align the visual recognition of the trainee's VR helmet with the VR system coordinate system in the virtual scene to ensure the accuracy of the equipment position. Multiple users operate virtual devices through handle actions and complete switching operation training in the VR interactive space. In the above processing process, edge computing can also be used to reduce data transmission delay. At this point, the virtual environment of the substation switching operation site can be obtained through the above method.

[0024] All virtual scene images obtained in the switching operation are grayed; in order to prevent the external environment noise interference from seriously affecting the image quality, all virtual scene images obtained in the switching operation are enhanced by using histogram equalization. Graying and histogram equalization are both well-known technologies, and the specific acquisition process will not be described in detail.

[0025] During each switching operation training process in the VR interaction space, when the trainee performs the switching control operation on the electrical equipment and maintains a static state for more than 1 s, any one frame is randomly selected from all the virtual scene images corresponding to the interaction space in each static state as the reference frame for each switching operation, and all the virtual scene images except the reference frame in each switching operation are recorded as the operation frames of each switching operation. That is, during one switching operation process, it includes one reference frame and multiple operation frames.

[0026] Step 2: Obtain the corner points in each virtual scene image; according to the dispersion degree of all run lengths of all gray levels in the neighborhood window of each corner point, and the difference between the gray value of each corner point and all gray values in the corresponding neighborhood window, obtain the corner feature intensity of each corner point, and compare it with the first preset threshold to obtain the significant feature points.

[0027] During the process of the trainee performing the switching operation by wearing the VR helmet, phenomena such as strong light, reflection, and glare caused by the complex external lighting environment may cause the camera of the VR helmet to fail to accurately identify the feature points. When the user is stationary, the jitter and positioning drift of the object edges in the virtual scene of the interaction space are obvious, making it impossible for the trainee to accurately locate the target when performing the switching operation in the virtual scene of the interaction space, affecting the normal progress of the training.

[0028] Specifically, when the influence of the complex external lighting environment on the images obtained by the VR helmet camera is more serious and the positioning effect of the trainee on the target in the virtual scene of the interaction space is worse, the phenomenon of feature point loss in each frame of the virtual scene image corresponding to the interaction space obtained by the VR helmet is more serious; furthermore, when the user is in a stationary state, the position offset of the feature points in the virtual scene image corresponding to the interaction space is more serious, and the difference between the same feature point and the surrounding environment is more obvious.

[0029] Taking each frame of the virtual scene image obtained by the binocular camera of the VR helmet as the input respectively, use the Harris corner detection algorithm to obtain all the corner point information in each frame of the virtual scene image respectively. The Harris corner detection algorithm is a well-known technology, and the specific process will not be elaborated here. The following takes any one frame of the virtual scene image as an example for analysis. Taking each corner point as the center, a window with a size of N×N is taken as the neighborhood window of each corner point. N is an odd number greater than 1. The larger N is, the more accurate the evaluation result of the feature intensity represented by the corner point in the neighborhood range of the virtual scene image corresponding to the interaction space, and at the same time, the larger the calculation amount. In this embodiment, N is taken as 5; based on the neighborhood window of each corner point, obtain the gray run matrix, and obtain the run lengths of each corner point for each gray level in the neighborhood window through the gray run matrix. The calculation method of the gray run matrix is a well-known technology, and the specific process will not be elaborated here.

[0030] As a preferred embodiment, according to the dispersion degree of all run lengths of all gray levels within the neighborhood window of each corner point, and the difference between the gray value of each corner point and all gray values within the corresponding neighborhood window, the corner feature intensity of each corner point is obtained, which is used to characterize the uniqueness degree within the neighborhood range of the position where the corner point is located in the virtual scene of the interaction space.

[0031] In this embodiment, the corner feature intensity of the i-th corner point is denoted as , and its specific expression is: ; in the formula, is the corner feature intensity of the i-th corner point; is the sum of the absolute values of the differences in gray values between the i-th corner point and all the remaining pixel points in its corresponding neighborhood window; is the variance of all run lengths of all gray levels in the neighborhood window corresponding to the i-th corner point.

[0032] The corner feature intensity reflects the difference between the corner point and the neighborhood range in the virtual scene image. The stronger the corner feature intensity, the more distinct and unique representation can be provided at the position where the corner point is located in the virtual scene of the interaction space, and the more accurate the virtual scene target positioning accuracy can be provided during feature point matching; represents the degree of difference in the continuous change of gray values of the i-th corner point within the neighborhood range. During the virtual scene target positioning process in the VR interaction space, the larger it is, the more rich local information the corner point can provide during target matching and positioning, and the more helpful it is for target positioning and tracking in the virtual scene.

[0033] Taking the corner feature intensities of all corner points in all virtual scene images as input, the OTSU method is used to obtain the segmentation threshold, which is denoted as the first preset threshold. The corner points with corner feature intensities greater than or equal to the first preset threshold are denoted as significant feature points. The OTSU method is a well-known technology, and the specific process will not be elaborated here. The beneficial effect of obtaining significant feature points in this application is that it can better screen out the corner points in the virtual scene corresponding to the VR interaction space that can provide rich local information, thereby improving the target positioning accuracy. On the basis of improving the target positioning accuracy, it reduces the calculation amount and improves the target positioning speed in the VR interaction space.

[0034] Step 3: Obtain the pairs of significant feature points between the reference frame and each operation frame; according to the proportion of the pairs of significant feature points between the reference frame and each operation frame and the distances of each pair of significant feature points, obtain the feature point offset of each operation frame; and combine the differences in the corner feature intensities of each pair of significant feature points between the reference frame and each operation frame to obtain the matching anomaly factor of each operation frame; according to the degree of chaos of the motion trajectories of the optical flow vector fields between each operation frame and the next operation frame, obtain the optical flow field chaos of each operation frame; and combine the energy differences of the high-frequency components within the neighborhood windows of all significant feature points in each operation frame to obtain the noise interference value of each operation frame.

[0035] The more serious the influence of the external complex illumination environment on the images obtained by the VR helmet camera, the more serious the loss of feature points in the reference frame and the operation frames when the trainee performs the switching operation in the interactive space virtual scene, and the more obvious the phenomenon of the offset of the feature point positions in the interactive space.

[0036] In the above manner, the significant feature points in the reference frame and the operation frame images during each switching operation of the trainee can be obtained. Taking any switching operation process as an example, the reference frame and the operation frame during the switching operation are analyzed. All the significant feature points in the reference frame and the operation frame obtained during the switching operation are used as inputs, and the corner feature intensity of each significant feature point is used as a descriptor. The FLANN Matcher (Fast Library for Approximate Nearest Neighbors) feature point matching algorithm is used to obtain all pairs of significant feature points between the reference frame and each operation frame. The FLANN Matcher feature point matching algorithm is a well-known technology, and the specific process will not be elaborated.

[0037] Furthermore, as a preferred implementation manner, according to the proportion of the pairs of significant feature points between the reference frame and each operation frame and the distances of each pair of significant feature points, obtain the feature point offset of each operation frame, and combine the differences in the corner feature intensities of each pair of significant feature points between the reference frame and each operation frame to obtain the matching anomaly factor of each operation frame, which is used to characterize the degree of abnormal feature point matching between the reference frame and each operation frame.

[0038] In this embodiment, the matching anomaly factor of the t-th operation frame is denoted as , and its specific expression is: ; in the formula, is the matching anomaly factor of the t-th operation frame; is the cumulative result of the absolute values of the differences between the corner feature intensities of all pairs of significant feature points between the reference frame and the t-th operation frame; is the feature point offset of the t-th operation frame. Among them, the calculation process of the feature point offset is: , in the formula, is the feature point offset of the t-th frame operation frame, are respectively the total number of successfully matched significant feature points and the total number of significant feature points between the reference frame and the t-th frame operation frame, is the sum of the Euclidean distances of all significant feature point pairs between the reference frame and the t-th frame operation frame.

[0039] The matching anomaly factor represents the degree of abnormal matching of feature points in the reference frame and the operation frame caused by the complex external illumination during the switching operation process; the feature point offset reflects the position offset of significant feature points between the reference frame and the operation frame; when the target positioning in the interactive space virtual scene is more affected by the external illumination, the greater the difference in local information within the neighborhood range of the successfully matched feature points, that is, the index has a larger value; the smaller the matching success rate of significant feature points in the reference frame and the operation frame, and the larger the offset distance of the successfully matched feature points, that is, the index has a larger value.

[0040] Only screening the key frames of target positioning in the interactive space based on the matching anomaly factor between the reference frame and each operation frame obtained above still has certain drawbacks, that is, the complex external illumination and the movement of the trainee in the interactive space virtual scene will generate random noise, and these random noises may be misjudged as significant feature points, which may cause the actual target positioning accuracy in the interactive space virtual scene to be relatively high while the matching anomaly factor between the reference frame and the operation frame is relatively large, affecting the target positioning accuracy in the interactive space.

[0041] Specifically, when the operation frame image is more severely interfered by the complex external illumination environment and random noise generated by the movement of the trainee during the target positioning process, the difference in high-frequency energy generated within the neighborhood range of significant feature points in the operation frame is more obvious, and the estimated motion trajectory of the optical flow field between adjacent frames is more chaotic.

[0042] Furthermore, the gray values of all pixel points within the neighborhood window of each significant feature point in each operation frame are arranged in a sequence from left to right and from top to bottom. All the sequences are respectively used as the input of the FFT (Fast Fourier Transform) to obtain the frequency components of the pixel gray values within the neighborhood range of each significant feature point in each operation frame. The frequency components of the pixel gray values within the corresponding neighborhood range of each significant feature point are arranged in descending order, and the top 5% of the frequency components are selected as the high-frequency components of each significant feature point. All the high-frequency components of each significant feature point are combined pairwise to obtain all the high-frequency component combinations of each significant feature point. Each operation frame image is used as the input of the Farneback dense optical flow field algorithm to obtain the optical flow vector field of each operation frame. Since both the FFT and the Farneback dense optical flow field algorithm are well-known technologies, the specific acquisition process will not be elaborated too much.

[0043] Further, as a preferred embodiment, according to the degree of chaos of the motion trajectories of the optical flow vector fields of each operation frame and the next operation frame, the optical flow field chaos degree of each operation frame is obtained, and in combination with the energy difference of the high-frequency components within the neighborhood windows of all significant feature points in each operation frame, the noise interference value of each operation frame is obtained, which is used to characterize the degree of interference of the significant feature point matching of the interactive spatial virtual scene image by random noise during each switching operation of the trainee.

[0044] In this embodiment, the noise interference value of the t-th operation frame is denoted as , and its specific expression is: ; in the formula, is the noise interference value of the t-th operation frame, is the sum of the absolute values of the energy differences of all high-frequency component combinations corresponding to all significant feature points in the t-th operation frame, is the optical flow field chaos degree of the t-th operation frame, which is used to characterize the degree of chaos of the motion trajectories of the optical flow vector fields of the t-th operation frame and the next operation frame. Specifically, in this embodiment, the process of obtaining the optical flow field chaos degree is as follows: taking the optical flow vector fields of the t-th and (t + 1)-th operation frames as the input of the Anosim between-group difference analysis algorithm, obtaining the global R value and the significance level value q between the optical flow vector fields corresponding to the t-th and (t + 1)-th operation frames, and taking the ratio of the global R value to the significance level value q as the optical flow field chaos degree of the t-th operation frame, denoted as . When is larger, it indicates that the difference between the optical flow vector fields corresponding to the t-th and (t + 1)-th operation frames is larger, and the motion trajectories of the optical flow vector fields are more chaotic. The Anosim between-group difference analysis algorithm is a well-known technology, and the specific process will not be elaborated here. It should be noted that when there is no operation frame after the t-th operation frame, the noise interference value of the t-th frame is not calculated.

[0045] The larger the noise interference value , the greater the degree of interference of the feature point matching in the t-th operation frame by random noise; The larger the value of , the greater the energy difference between the high-frequency components within the neighborhood range of the significant feature points in the operation frame, and then the more severely the operation frame is interfered by random noise during the target positioning process; the larger the value of the optical flow field chaos degree

[0046] Step 4: According to the abnormal matching factors and noise interference values of each operation frame, obtain the positioning accuracy factors of each operation frame, compare them with the second preset threshold, obtain the key frames in the switching operation, and complete the target positioning of the interactive space.

[0047] Further, when the matching anomaly factor of the job frame is smaller and the noise interference value is lower, it indicates that the target positioning in the virtual scene of the VR interaction space corresponding to the job frame is more accurate, and the job frame should be less likely to be used as a key frame for target positioning.

[0048] As a preferred implementation, according to the matching anomaly factors and noise interference values of each job frame, a positioning accuracy factor for each job frame is constructed to characterize the positioning accuracy of each job frame as a key frame for target positioning in the virtual scene of the interaction space. The flowchart for obtaining the positioning accuracy factor of each job frame is as Figure 2 shown.

[0049] In this embodiment, the positioning accuracy factor of the t-th job frame is denoted as , and its specific expression is: ; where is the positioning accuracy factor of the t-th job frame; is the matching anomaly factor of the t-th job frame; is the noise interference value of the t-th job frame; norm( ) is a normalization function such that has a value range within [0, 1].

[0050] When is larger, it indicates that the feature point matching condition of the t-th job frame and the reference frame is better, and it is less affected by random noise interference. For accurate target positioning in the virtual scene of the interaction space, then the job frame should be more likely to be used as a key frame to perform target positioning in the virtual scene of the interaction space.

[0051] Set a second preset threshold Y. During each switching operation of the trainee, the job frames with a positioning accuracy factor greater than or equal to the second preset threshold are used as key frames; the job frames with a positioning accuracy factor less than the second preset threshold are used as non-key frames. In this embodiment, the second preset threshold Y is taken as 0.6, and the implementer can set the size of the second preset threshold according to the actual situation.

[0052] During each switching operation of the trainee, all non-key frames are cut or removed, or the key frame feature propagation is performed using the TSDF model to generate interpolation frames to replace the original non-key frames. In this embodiment, the former method is adopted to eliminate all non-key frames, and the remaining key frames are spliced into a continuous video stream in chronological order and transmitted in real time to the VR helmets worn by all trainees through WiFi or 5G network for display.

[0053] Thus, the method for target positioning in the interaction space for multiple VR helmets can be implemented through the above method.

[0054] Based on the same inventive concept as the above method, an embodiment of the present application further provides an interactive space target positioning device for multiple VR helmets, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of any one of the above methods for interactive space target positioning for multiple VR helmets are implemented.

[0055] It should be noted that the above sequence of embodiments of the present application is only for description and does not represent the superiority or inferiority of the embodiments. And the above specific embodiments of this specification have been described. Additionally, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0056] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and the differences between each embodiment and other embodiments are emphasized.

[0057] The above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present application shall be included within the protection scope of the present application.

Claims

1. A method for positioning a target in an interactive space for multiple VR helmets, characterized in that: The method comprises the following steps: Acquire a virtual scene image during the switching operation; select one frame from the virtual scene image when the switch is controlled during the switching operation as a reference frame, and use the remaining frames in the switching operation except the reference frame as operation frames; Obtain corner points in each virtual scene image; obtain corner point feature strength of each corner point according to the discrete degree of all run lengths of all gray levels in the neighborhood window of each corner point, and the difference between the gray value of each corner point and all gray values ​​in the corresponding neighborhood window, and compare with the first preset threshold to obtain significant feature points; Obtain the significant feature point pairs between the reference frame and each operating frame; obtain the feature point offset of each operating frame according to the proportion of significant feature point pairs between the reference frame and each operating frame and the distance of each significant feature point pair; and obtain the matching anomaly factor of each operating frame in combination with the difference in the corner feature strength of each significant feature point pair between the reference frame and each operating frame; obtain the optical flow field disorder of each operating frame according to the degree of disorder of the motion trajectory of the optical flow vector field of each operating frame and the next operating frame; and obtain the noise interference value of each operating frame in combination with the energy difference of the high-frequency components in the neighborhood window of all significant feature points in each operating frame; According to the abnormal matching factor and noise interference value of each operation frame, the positioning accuracy factor of each operation frame is obtained, and compared with the second preset threshold, the key frame in the switching operation is obtained to complete the target positioning of the interactive space.

2. The interactive space target positioning method for multiple VR helmets according to claim 1, characterized in that: The calculation formula of the corner point feature strength of each corner point is: ; In the formula, is the corner feature strength of the i-th corner point; It is the cumulative sum of the absolute values ​​of the grayscale value differences between the i-th corner point and all other pixels in its corresponding neighborhood window; is the variance of all run lengths of all gray levels in the neighborhood window corresponding to the i-th corner point.

3. The interactive space target positioning method for multiple VR helmets according to claim 1, characterized in that: The specific process of obtaining the significant feature points is: recording the corner points whose feature strength is greater than or equal to the first preset threshold as significant feature points.

4. The interactive space target positioning method for multiple VR helmets according to claim 1, characterized in that: The calculation formula of the feature point offset of each operation frame is: ; In the formula, is the feature point offset of the t-th frame, are the total number of successfully matched salient feature points and the total number of salient feature points between the reference frame and the t-th working frame, respectively. It is the cumulative sum of the Euclidean distances of all pairs of salient feature points between the reference frame and the t-th working frame.

5. The interactive space target positioning method for multiple VR helmets according to claim 1, characterized in that: The calculation formula of the matching anomaly factor of each operation frame is: ; In the formula, is the matching anomaly factor of the t-th job frame; It is the cumulative result of the absolute value of the difference between the corner feature intensities of all significant feature point pairs between the reference frame and the t-th working frame; is the feature point offset of the t-th operating frame.

6. The interactive space target positioning method for multiple VR helmets according to claim 1, characterized in that: The process of obtaining the optical flow field chaos degree is as follows: obtaining the optical flow vector field of each operating frame; using the optical flow vector field of a single operating frame and its next operating frame as the input of the Anosim inter-group difference analysis algorithm, obtaining the global R value and the significance level value q between the corresponding optical flow vector fields of the single operating frame and its next operating frame, and using the ratio of the global R value to the significance level value q as the optical flow field chaos degree of the single operating frame.

7. The interactive space target positioning method for multiple VR helmets according to claim 1, characterized in that: The calculation formula of the noise interference value of each operation frame is: ; In the formula, is the noise interference value of the t-th frame operation, is the cumulative sum of the absolute values ​​of the energy differences of all high-frequency component combinations corresponding to all significant feature points in the t-th frame. is the optical flow field disorder degree of the t-th frame operation frame; wherein, the combination of all high-frequency components corresponding to all the significant feature points refers to all high-frequency component combinations of each significant feature point obtained by combining all the high-frequency components of each significant feature point in pairs.

8. The interactive space target positioning method for multiple VR helmets according to claim 1, characterized in that: The calculation formula of the positioning accuracy factor of each operation frame is: ; In the formula, is the positioning accuracy factor of the t-th operating frame; is the matching anomaly factor of the t-th job frame; is the noise interference value of the t-th frame; norm() is the normalization function.

9. The interactive space target positioning method for multiple VR helmets according to claim 1, characterized in that: The specific process of obtaining the key frames in the switching operation and completing the target positioning of the interactive space is as follows: The operation frames whose positioning accuracy factor is greater than or equal to the second preset threshold are regarded as key frames, and the operation frames whose positioning accuracy factor is less than the second preset threshold are regarded as non-key frames; During each switching operation of each trainee, all non-key frames are cut or removed, and the remaining key frames are spliced ​​into a continuous video stream in chronological order, which is then transmitted in real time through the transmission network to the corresponding VR helmet display module for display.

10. An interactive space target positioning device for multiple VR helmets, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the interactive space target positioning method for multiple VR helmets are implemented as described in any one of claims 1 to 9.

Images