An augmented reality-based presentation method and apparatus

Abstract

The application discloses an augmented reality-based display method and device, and the method comprises the following steps: acquiring real scene image information; performing calculation processing on the real scene image information to obtain target scene information; and performing analysis processing on the target scene information to obtain target display information. It can be seen that, by superimposing virtual search and rescue information on a real terrain sand table environment, the real world and virtual information can be simultaneously observed, and the cooperation efficiency and response speed of a search and rescue emergency scheme are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of augmented reality, in particular to an augmented reality-based display method and device. BACKGROUND

[0002] With the rapid development of information technology and digital media, the form of search and rescue scene display is also constantly innovating. Traditional exhibition display mainly relies on static physical exhibits, text and picture explanations and video playing. This way is intuitive, but there are certain limitations in information transmission and audience interaction. In recent years, virtual reality (VR) and augmented reality (AR) technologies have been gradually applied to the field of exhibition display to improve the audience's sense of participation and experience.

[0003] Augmented reality (AR) technology provides a rich interactive experience for the audience by superimposing virtual information on the real scene. For example, the audience can view the three-dimensional model, animation demonstration or detailed information of the exhibit through special AR glasses, which not only enriches the display content, but also makes the display effect more lively and interesting.

[0004] Currently on the market, there are PC and VR (virtual reality) based display methods for search and rescue emergency plan display, which have the following characteristics:

[0005] PC-based display relies on static physical exhibits, text and picture explanations and video playing. This way is intuitive, but the information presentation is relatively single, the interactivity is insufficient, and it cannot provide an immersive experience, making it difficult for the audience to fully understand the complex search and rescue emergency plan process.

[0006] Virtual reality (VR) display provides an immersive experience for the audience through a completely immersive virtual environment. The audience can interact in the virtual environment to gain a deep understanding of the details of the search and rescue emergency operation. SUMMARY

[0007] The technical problem to be solved by the present application is to provide an augmented reality-based display method and device, which can superimpose virtual search and rescue information on the real terrain sand table environment, allowing the audience to see both the real world and virtual information, thereby improving the collaboration efficiency and response speed of the search and rescue emergency plan.

[0008] To solve the above technical problems, the first aspect of the embodiment of the present application discloses an augmented reality-based display method, which comprises:

[0009] S1, acquiring real scene image information;

[0010] S2, performing calculation processing on the real scene image information to obtain target scene information;

[0011] S3, performing analysis processing on the real scene image information and the target scene information to obtain target display information.

[0012] As an optional implementation, in the first aspect of the embodiment of the present application, the computing processing on the real scene image information to obtain the target scene information comprises:

[0013] S21, performing preprocessing on the real scene image information to obtain preprocessed image information;

[0014] S22, performing analysis processing on the preprocessed image information to obtain first angle information;

[0015] S23, performing computing processing on the first angle information to obtain rotation angle information and offset information;

[0016] S24, performing conversion processing on the rotation angle information and the offset information to obtain the target scene information.

[0017] As an optional implementation, in the first aspect of the embodiment of the present application, the analysis processing on the preprocessed image information to obtain the first angle information comprises:

[0018] S221, performing contour detection processing on the preprocessed image information to obtain contour information;

[0019] S222, performing screening processing on the contour information to obtain target contour information;

[0020] S223, performing computing processing on the preprocessed image information and the target contour information to obtain the first angle information.

[0021] As an optional implementation, in the first aspect of the embodiment of the present application, the computing processing on the first angle information to obtain the rotation angle information and the offset information comprises:

[0022] S231, obtaining second angle information and scene angle training data set;

[0023] S232, performing training processing on the scene angle training data set to obtain rotation angle offset information and offset amount offset information;

[0024] S233, using a scene angle calculation model to perform computing processing on the first angle information and the second angle information to obtain scene matrix information;

[0025] Wherein, the scene angle calculation model is:

[0026]

[0027] In the formula, H represents the scene matrix information. i For the i-th scene matrix in the scene matrix information, the H i It is a 3x3 matrix, (JDYX) i,j JDYY i,j ) represents the j-th angular coordinate information corresponding to the i-th angular information in the first angular information, (JDEX i,j ,JDEYi i,j ) represents the j-th angular coordinate information corresponding to the second angular information, and N represents the number of angular information in the first angular information.

[0028] S234, calculate and process the rotation angle offset information, the offset amount offset information, and the scene matrix information to obtain the rotation angle information and the offset amount information.

[0029] As an optional implementation, in the first aspect of the present invention, the calculation and processing of the rotation angle offset information, the offset amount offset information, and the scene matrix information to obtain the rotation angle information and the offset amount information includes:

[0030] S2341, Using the viewpoint offset calculation model, the scene matrix information is calculated and processed to obtain preprocessed rotation angle information and preprocessed offset information;

[0031] The calculation model for the viewpoint offset is as follows:

[0032] (θx i1 ,θy i1 )=(δ1·arctan(H i1,32 ),δ2·arctan(H i1,31 ));

[0033]

[0034] In the formula, (θx,θy) represents the preprocessed rotation angle information, (tx,ty) represents the preprocessed offset information, and (θx) represents the preprocessed offset information. i1 ,θy i1 ) represents the i1th rotation angle coordinate information in the preprocessed rotation angle information, (tx i1 ,ty i1 ) represents the i1th offset coordinate information in the preprocessed offset information, and H represents the scene matrix information. i1,23 H i1,31 H i1,32 and H i1,33wherein, δ1, δ2, δ3 and δ4 are respectively a first weight parameter, a second weight parameter, a third weight parameter and a fourth weight parameter, 1≤i1≤N, N is a number of scene matrices in the scene matrix information;

[0035] S2342, offsetting the rotation angle offset information, the offset amount offset information, the preprocessed rotation angle information and the preprocessed offset amount information by using the perspective offset amount offset calculation model to obtain rotation angle information and offset amount information;

[0036] wherein, the perspective offset amount offset calculation model is:

[0037] θ′=δ5·θ+Δθ;

[0038] t′=δ6·t+Δt;

[0039] wherein, θ′ is the rotation angle information, t′ is the offset amount information, θ and t are respectively the preprocessed rotation angle information and the preprocessed offset amount information, Δθ and Δt are respectively the rotation angle offset information and the offset amount offset information, and δ5 and δ6 are respectively a fifth weight parameter and a sixth weight parameter.

[0040] As an optional implementation, in the first aspect of the embodiment of the present application, the conversion processing of the rotation angle information and the offset amount information to obtain target scene information comprises:

[0041] S241, obtaining target coordinate information, target resolution information and target size information;

[0042] S242, converting the target coordinate information, the target resolution information, the target size information, the rotation angle information and the offset amount information by using a scene conversion model to obtain target scene information;

[0043] wherein, the scene conversion model is:

[0044]

[0045] Z=YS;

[0046] PYJ=θ′+t′;

[0047] wherein, (X, Y, Z) is corresponding coordinate information in the target scene information, PYJ is corresponding plane offset angle information in the target scene information, X k1 and Y k2X′ k1 and Y′ k2 Xk1and Yk2are the k1th X coordinate value and the k2th Y coordinate value in the target coordinate information, Ys is the Z coordinate value, XH and YH are the horizontal resolution and the vertical resolution in the target resolution information, L and D are the horizontal size information and the vertical size information in the target size information, M1 and M2 are the number of X coordinate values and the number of Y coordinate values in the target coordinate information, θ′ and t′ are the rotation angle information and the offset information.

[0048] As an optional implementation, in the first aspect of the embodiment of the present application, the analyzing and processing the real scene image information and the target scene information to obtain target display information comprises:

[0049] S31, obtaining virtual scene three-dimensional model information;

[0050] S32, traversing the virtual scene three-dimensional model information to obtain position information, direction information and scale information of the virtual scene three-dimensional model information;

[0051] S33, projecting the virtual scene three-dimensional model information into the real scene image information and the target scene information according to the position information, the direction information and the scale information to obtain target display information.

[0052] The second aspect of the embodiment of the present application discloses a display device based on augmented reality, which comprises:

[0053] a model obtaining unit configured to obtain real scene image information;

[0054] a first calculation module configured to calculate and process the real scene image information to obtain target scene information;

[0055] a second calculation module configured to analyze and process the target scene information to obtain target display information.

[0056] The third aspect of the embodiment of the present application discloses another display device based on augmented reality, which comprises:

[0057] a processor;

[0058] a memory coupled to the processor and storing executable program codes;

[0059] the processor invokes the executable program codes stored in the memory to execute part or all steps of the display method based on augmented reality disclosed in the first aspect of the embodiment of the present application.

[0060] The fourth aspect of the embodiment of the present application discloses a computer readable storage medium, which stores computer instructions, and the computer instructions are used to execute part or all steps of the augmented reality-based display method disclosed in the first aspect of the embodiment of the present application when called.

[0061] Compared with the prior art, the embodiment of the present application has the following beneficial effects:

[0062] In the embodiment of the present application, real scene image information is acquired, the real scene image information is calculated and processed to obtain target scene information, and the target scene information is analyzed and processed to obtain target display information. It can be seen that, by superimposing virtual search and rescue information on a real terrain sand table environment, the real world and virtual information can be seen at the same time, which is beneficial to improving the cooperation efficiency and response speed of search and rescue emergency plans. BRIEF DESCRIPTION OF DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0064] Figure 1 The flowchart of the augmented reality-based display method disclosed in the embodiment of the present application is shown in the figure.

[0065] Figure 2 The structure diagram of the augmented reality-based display device disclosed in the embodiment of the present application is shown in the figure.

[0066] Figure 3 The structure diagram of another augmented reality-based display device disclosed in the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0067] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0068] The terms "first", "second", and the like in the description and in the claims of the present application and above-described drawings are used to distinguish different objects, and are not used to describe a particular order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product, or equipment including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product, or equipment.

[0069] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment can be included in at least one embodiment of the application. The appearance of the phrase in various places in the specification does not necessarily all refer to the same embodiment, nor is it necessarily mutually exclusive of other embodiments. It is explicitly and implicitly understood that the embodiments described herein can be combined with other embodiments.

[0070] The application discloses an augmented reality-based display method and device, which can simultaneously see the real world and virtual information by superimposing virtual search and rescue information on a real terrain sand table environment, and is beneficial to improving the cooperation efficiency and response speed of search and rescue emergency plans. The following will be described in detail.

[0071] Embodiment one

[0072] Please refer to Figure 1 , Figure 1 is a flowchart of an augmented reality-based display method disclosed by the embodiment of the application. Wherein, Figure 1 The augmented reality-based display method described is applied to an augmented reality-based display device, such as a local server or a cloud server for augmented reality-based display optimization management, and the embodiment of the application is not limited. As shown in Figure 1 The augmented reality-based display method can include the following operations:

[0073] S1, acquiring real scene image information;

[0074] It should be noted that the real scene image information is captured by a front camera of AR glasses, which captures the environment image in the user's field of view, acquires a real-time video stream, and arranges each frame in the real-time video stream in time order from small to large to obtain. The real scene image information includes a plurality of acquisition image information, i.e. a plurality of continuous frames in the real-time video stream.

[0075] S2, performing calculation processing on the real scene image information to obtain target scene information;

[0076] S3, analyzing and processing the real scene image information and the target scene information to obtain target display information.

[0077] It can be seen that the augmented reality-based display method described in the embodiments of the present application can simultaneously see the real world and virtual information by superimposing virtual search and rescue information on the real terrain sand table environment, which is conducive to improving the cooperation efficiency and response speed of search and rescue emergency plans.

[0078] In an optional embodiment, the calculating and processing the real scene image information to obtain target scene information comprises:

[0079] S21, pre-processing the real scene image information to obtain pre-processed image information;

[0080] S22, analyzing and processing the pre-processed image information to obtain first angle information;

[0081] S23, calculating and processing the first angle information to obtain rotation angle information and offset information;

[0082] S24, converting and processing the rotation angle information and the offset information to obtain target scene information.

[0083] It can be seen that the augmented reality-based display method described in the embodiments of the present application can simultaneously see the real world and virtual information by superimposing virtual search and rescue information on the real terrain sand table environment, which is conducive to improving the cooperation efficiency and response speed of search and rescue emergency plans.

[0084] In another optional embodiment, the pre-processing the real scene image information to obtain pre-processed image information comprises:

[0085] S211, thresholding processing the real scene image information to obtain thresholded scene image information;

[0086] It should be noted that the thresholding processing described above is obtained by a thresholding calculation model;

[0087] The thresholding calculation model is:

[0088] YZ s1,s2 = δ7·U1 s1,s2 + δ8·U2 s1,s2 + δ9·U3 s1,s2 1≤s1≤N,1≤s2≤L;

[0089] δ7+ δ8+ δ9= 1;

[0090] 0< δ7< 1, 0< v8< 1, 0< δ9< 1;

[0091] In the formula, YZ is the thresholded scene image information, YZ s1,s2 is the s2 pixel value of the s1 thresholded image information in the thresholded scene image information, U1 s1,s2 , U2 s1,s2 and U3 s1,s2 are respectively the first channel value, the second channel value and the third channel value of the s2 pixel of the s1 acquisition image information in the real scene image information, s1 is the number of the acquisition image information in the real scene image information, L is the total number of pixels of the s1 acquisition image information in the real scene image information, and δ7, δ8 and δ9 are the seventh weight parameter, the eighth weight parameter and the ninth weight parameter.

[0092] It should be noted that the first channel value, the second channel value and the third channel value represent one of the red channel value, the green channel value and the blue channel value, for example, the first channel value is the red channel value, the second channel value is the blue channel value, and the third channel value is the green channel value, and the specific setting is set by the user, and the embodiments of the present application are not limited specifically.

[0093] It should be noted that the thresholding calculation model improves the accuracy and effect of thresholding by reasonable weighting of the seventh weight parameter, the eighth weight parameter and the ninth weight parameter, which helps to keep important features while removing noise, enhances the usable information of the image, and can automatically adjust the weight parameter according to the actual demand, automatically optimize the image processing flow and improve the overall performance.

[0094] S212, the thresholded scene image information is denoised to obtain denoised image information;

[0095] It should be noted that the above denoising processing is performed by a Gaussian filter or other algorithms for denoising images, and the embodiments of the present application are not limited specifically.

[0096] S213, the denoised image information is edge enhanced to obtain preprocessed image information.

[0097] It should be noted that the above edge enhancement processing is performed by a Sobel operator or other algorithms capable of edge enhancement processing, and the embodiments of the present application are not limited specifically.

[0098] It should be noted that, in the embodiments of the present invention, edge enhancement and contour detection are performed on the electronic sand table in the denoised image information. When AR glasses are used, the electronic sand table captured by the camera in the AR glasses is converted into a three-dimensional spatial scene, and the search and rescue plan model information (virtual scene three-dimensional model information) is projected into the three-dimensional spatial scene. Then, the map data, topographic map and environmental features of the search and rescue area are loaded and displayed on the electronic sand table in real time, thereby realizing the superposition of the virtual information of the search and rescue plan and the real scene information of the electronic sand table.

[0099] It should be noted that the electronic sand table integrates a Geographic Information System (GIS), which can load and display map data, topographic maps and environmental features of the search and rescue area in real time. At the same time, it synchronizes the 3D model information of the virtual scene participating in the search and rescue to the display software on the AR glasses (for displaying the display results information), providing data support for it.

[0100] It should be noted that the electronic sand table is a device that can display map data, topographic maps and environmental features of the search and rescue area. It is rectangular in shape, and the specific device type or information is not specifically limited in this embodiment of the invention.

[0101] As can be seen, by implementing the augmented reality-based display method described in the embodiments of the present invention, virtual search and rescue information can be superimposed on the real terrain sand table environment, allowing people to see both the real world and virtual information at the same time, which is beneficial to improving the collaborative efficiency and response speed of search and rescue emergency plans.

[0102] In another optional embodiment, the step of analyzing and processing the preprocessed image information to obtain the first angle information includes:

[0103] S221, Perform contour detection processing on the preprocessed image information to obtain contour information;

[0104] It should be noted that the above contour detection process uses the Canny edge detection algorithm to detect the edges of each preprocessed image information in the preprocessed image information. Other edge detection algorithms can also be used. Specifically, this embodiment of the invention does not limit the specific methods.

[0105] S222, The contour information is filtered to obtain the target contour information;

[0106] It should be noted that the above filtering process is performed by a user-defined algorithm or other algorithms that can filter contour information; this embodiment of the invention does not impose any specific limitations.

[0107] S223, the preprocessed image information and the target contour information are calculated and processed to obtain the first angle information.

[0108] It should be noted that the above calculation processing is calculated by polygon approximation algorithm recognition. For example, if the electronic sand table is a long rectangle, the polygon approximation algorithm recognition is used to identify the four vertices of the rectangle in the image, and the coordinate information of the four fixed points (including the first angle coordinate, the second angle coordinate, the third angle coordinate and the fourth angle coordinate) is recorded as the angle information. The first angle information includes calculating the electronic sand table contour information in the target contour information corresponding to all pre-processing collected image information and all pre-processing collected image information in the pre-processing image information.

[0109] It should be noted that the coordinate information in S223 and S231 is the coordinate in the two-dimensional coordinate system, that is, the left lower corner of the pre-processing image information is taken as the origin, the direction from the left lower corner to the right lower corner is taken as the positive direction of the X axis, and the direction from the left lower corner to the left upper corner is taken as the positive direction of the Y axis. Each pixel point in the image is in the first quadrant of the coordinate system, and each pixel point corresponds to a coordinate point.

[0110] It can be seen that the method for displaying based on augmented reality described in the embodiment of the application can superimpose virtual search and rescue information on the real terrain sand table environment, so that the real world and virtual information can be seen at the same time, which is beneficial to improving the cooperation efficiency and response speed of the search and rescue emergency plan.

[0111] In an optional embodiment, the calculation processing of the first angle information includes obtaining rotation angle information and offset information.

[0112] S231, obtaining second angle information and scene angle training data set;

[0113] It should be noted that the above second angle information is the theoretical angle information of the electronic sand table in the pre-processing image information. For example, the vertex coordinates of the left lower corner of the theoretical angle information of the electronic sand table in the pre-processing image information are (150, 250), and the resolution of the electronic sand table is 1920x1080. Therefore, the theoretical angle information is the left lower corner (150, 250), the right lower corner (150+1919, 250), the left upper corner (150, 250+1079) and the right upper corner (150+1919, 250+1079).

[0114] It should be noted that the scene angle training data set is a data set for training the rotation angle information and the offset information of the electronic sand table in the real scene image information. Finally, the rotation angle offset information and the offset information are obtained. The data set can be obtained from the past test data or obtained from the public database, and the embodiment of the application does not make specific limitation.

[0115] S232, training processing is performed on the scene angle training data set to obtain rotation angle offset information and offset amount offset information;

[0116] S233, using a scene angle calculation model, the first angle information and the second angle information are calculated to obtain scene matrix information;

[0117] The scene angle calculation model is:

[0118]

[0119] In the formula, H is the scene matrix information, H i is the i-th scene matrix in the scene matrix information, and the H i is a 3*3 matrix, (JDYX i,j ,JDYYi i,j ) is the j-th angle coordinate information corresponding to the i-th angle information in the first angle information, (JDEX i,j ,JDEYi i,j ) is the corresponding j-th angle coordinate information in the second angle information, and N is the number of angle information in the first angle information.

[0120] S234, the rotation angle offset information, the offset amount offset information, and the scene matrix information are calculated to obtain rotation angle information and offset amount information.

[0121] It can be seen that the method for displaying based on augmented reality described in the embodiment of the application can simultaneously show the real world and virtual information by superimposing virtual search and rescue information on the real terrain sand table environment, which is beneficial to improving the cooperation efficiency and response speed of search and rescue emergency plans.

[0122] In an optional embodiment, the training processing of the scene angle training data set obtains rotation angle offset information and offset amount offset information, including:

[0123] It should be noted that the training processing of the scene angle training data set to obtain rotation angle offset information and offset amount offset information can also be obtained by convolutional neural network (CNN) or recurrent neural network (RNN).

[0124] S2321, obtaining a rotation angle increment value and an offset amount increment value;

[0125] S2322, using a preset angle update model, the scene angle training data set is calculated to obtain a training rotation angle value and a training offset amount value;

[0126] It should be noted that the above calculation processing can be calculated by the user set calculation processing mode, and the embodiments of the present application are not limited.

[0127] It should be noted that the above-mentioned preset angle updating model can be a linear updating model, a rotation matrix model, a convolutional neural network (CNN) or a recurrent neural network (RNN), and the embodiments of the present application are not limited.

[0128] S2323, obtaining real rotation angle values and real offset values corresponding to the scene angle training data set;

[0129] S2324, using an error calculation model to calculate the rotation angle increment value, the offset increment value, the training rotation angle value, the training offset value, the real rotation angle value and the real offset value, and obtain error rotation angle value and error offset value;

[0130] The error calculation model is:

[0131] θ w = θ z - θ x - θ l ;

[0132] t w = t z - t x - t l ;

[0133] In the formula, θ w and t w are the error rotation angle value and the error offset value, θ z and t z are the real rotation angle value and the real offset value, θ x and t x are the training rotation angle value and the training offset value, θ l and t l are the rotation angle increment value and the offset increment value.

[0134] By calculating the error rotation angle value and the error offset value, the system can evaluate the difference between the calculation result and the actual rotation angle and the offset. This helps to optimize the system and adjust the calculation model to make it more accurate.

[0135] S2325, determining whether the error rotation angle value is less than a preset error rotation angle threshold to obtain a first error determination result;

[0136] When the first error determination result is yes, S2326 is executed;

[0137] When the first error judgment result is no, a first scene angle training dataset is obtained, the first scene angle training dataset is determined as the scene angle training dataset, and S2327 is executed;

[0138] S2326, whether the error offset value is less than a preset error offset threshold is judged, and a second error judgment result is obtained;

[0139] When the second error judgment result is no, a second scene angle training dataset is obtained, the second scene angle training dataset is determined as the scene angle training dataset, and S2327 is executed;

[0140] When the second error judgment result is yes, S2328 is executed;

[0141] The first scene angle training dataset contains more samples for rotation, and the second scene angle training dataset contains more samples for offset. By judging whether the error rotation angle and the error offset are within the corresponding threshold range, the system can determine which training dataset to use according to the different levels or sequences of errors (rotation angle error and offset error). This hierarchical processing helps to optimize the model for different problems and improves the effectiveness of training and adjustment. At the same time, the model can converge faster and more accurately.

[0142] It should be noted that the preset error rotation angle threshold and the preset error offset threshold can be set by the user or obtained according to historical data, and the embodiments of the present application are not limited.

[0143] For example, the preset error rotation angle threshold and the preset error offset threshold have a value range of [0.01, 0.05].

[0144] S2327, the parameter update model is used to update the rotation angle increment value and the offset increment value, to obtain an updated rotation angle increment value and an updated offset increment value, and to determine the updated rotation angle increment value as the rotation angle increment value and the updated offset increment value as the offset increment value, and execute S2322;

[0145] The parameter update model is:

[0146] θ′ l ←θ l +θθ;

[0147] t′ l ←t l +tt;

[0148] In the formula, θ′ land t' l are preset rotation angle update step values and preset offset update step values;

[0149] It should be noted that the preset rotation angle update step values and the preset offset update step values can be set by a user or obtained according to historical data, and embodiments of the present application are not limited thereto.

[0150] For example, the preset rotation angle update step values and the preset offset update step values are both 0.1.

[0151] S2328, determining the rotation angle increment value as rotation angle offset information and determining the offset increment value as offset offset information.

[0152] It can be seen that, by implementing the augmented reality-based display method described in the embodiments of the present application, virtual search and rescue information is superimposed on a real terrain sand table environment, so that the real world and virtual information can be seen at the same time, which is beneficial to improving the cooperation efficiency and response speed of search and rescue emergency plans.

[0153] In an optional embodiment, the calculation and processing of the rotation angle offset information, the offset offset information and the scene matrix information to obtain rotation angle information and offset information comprises:

[0154] S2341, calculating and processing the scene matrix information by using a view angle offset calculation model to obtain preprocessed rotation angle information and preprocessed offset information.

[0155] The view angle offset calculation model is:

[0156] (θx i1 ,θy i1 )=(δ1·arctan(H i1,32 ),δ2·arctan(H i1,31 ));

[0157]

[0158] In the formula, (θx, θy) is the preprocessed rotation angle information, (tx, ty) is the preprocessed offset information, (θx i1 ,θy i1 ) is the i1th rotation angle coordinate information in the preprocessed rotation angle information, (tx i1 ,ty i1 ) is the i1th offset coordinate information in the preprocessed offset information, H is the scene matrix information, and H i1,23 , Hi1,31 , H i1,32 and H i1,33 are a value of a 2nd row and a 3rd column, a value of a 3rd row and a 1st column, a value of a 3rd row and a 2nd column, and a value of a 3rd row and a 3rd column of an i1th scene matrix in the scene matrix information, δ1, δ2, δ3 and δ4 are respectively a first weight parameter, a second weight parameter, a third weight parameter and a fourth weight parameter, 1≤i1≤N, N is a quantity of scene matrices in the scene matrix information;

[0159] It should be noted that the first weight parameter, the second weight parameter, the third weight parameter and the fourth weight parameter can be set by a user, or can be obtained according to historical data, and embodiments of the present application do not make specific limitations.

[0160] It should be noted that the first weight parameter, the second weight parameter, the third weight parameter and the fourth weight parameter allow flexible adjustment of the rotation angle and the offset in different scenes. If the influence of the rotation angle is greater, the values of δ1 and δ2 can be increased, and if the influence of the offset is greater, the values of δ3 and δ4 can be increased.

[0161] Through the above view angle offset calculation model, more reliable geometric information can be extracted from the original scene data, thereby improving the reliability of the calculation and the fitting capability for complex view angle changes in a real scene, and meanwhile, the influence of the absolute scale in the scene matrix can be eliminated, and the robustness of the calculation is enhanced.

[0162] S2342, the rotation angle offset information, the offset offset information, the preprocessed rotation angle information and the preprocessed offset information are calculated and processed by using the view angle offset offset calculation model, to obtain rotation angle information and offset information;

[0163] The view angle offset offset calculation model is:

[0164] θ' = δ5·θ + Δθ;

[0165] t' = δ6·t + Δt;

[0166] In the formula, θ' is the rotation angle information, t' is the offset information, θ and t are respectively the preprocessed rotation angle information and the preprocessed offset information, Δθ and Δt are respectively the rotation angle offset information and the offset offset information, and δ5 and δ6 are respectively a fifth weight parameter and a sixth weight parameter.

[0167] It should be noted that the fifth weight parameter and the sixth weight parameter can be set by a user, or can be obtained according to historical data, and embodiments of the present application do not make specific limitations.

[0168] It should be noted that the combination of the fifth weight parameter, the sixth weight parameter, the rotation angle offset information and the offset amount offset information makes the model have stronger fault tolerance to abnormal values or errors,

[0169] It can be seen that the augmented reality-based display method described in the embodiments of the present application can simultaneously show the real world and virtual information by superimposing virtual search and rescue information on the real terrain sand table environment, which is beneficial to improving the cooperation efficiency and response speed of search and rescue emergency plans.

[0170] In an optional embodiment, the conversion processing of the rotation angle information and the offset amount information to obtain target scene information includes:

[0171] S241, obtaining target coordinate information, target resolution information and target size information;

[0172] It should be noted that the target coordinate information is the coordinate information of the electronic sand table in the real scene image information in the two-dimensional coordinate system, and the coordinate information herein is that the top point of the upper left corner of the electronic sand table in the real scene image information is the origin, the direction from the upper left corner to the upper right corner is the positive direction of the X axis, and the direction from the upper left corner to the lower left corner is the positive direction of the Y axis. Each pixel point in the image is in the first quadrant of the coordinate system, and each pixel point corresponds to a coordinate point.

[0173] It should be noted that the target resolution information and the target size information are the actual resolution information and size information of the electronic sand table.

[0174] S242, using a scene conversion model to convert the target coordinate information, the target resolution information, the target size information, the rotation angle information and the offset amount information to obtain target scene information;

[0175] The scene conversion model is:

[0176]

[0177] Z=YS;

[0178] PYJ=θ′+t′;

[0179] In the formula, (X, Y, Z) is the corresponding coordinate information in the target scene information, PYJ is the corresponding plane offset angle information in the target scene information, X k1 and Y k2 are the k1th X coordinate value and the k2th Y coordinate value in the corresponding coordinate information in the target scene information, X′ k1 and Y′ k2For the k1th X coordinate value and the k2th Y coordinate value in the target coordinate information, YS is the Z coordinate value, XH and YH are the horizontal resolution and the vertical resolution in the target resolution information, L and D are the horizontal size information and the vertical size information in the target size information, M1 and M2 are the number of X coordinate values and the number of Y coordinate values in the target coordinate information, and θ' and t' are the rotation angle information and the offset information.

[0180] It should be noted that the Z coordinate value is set by a user or obtained according to historical data, and the embodiment of the application is not limited.

[0181] It should be noted that the entire area of the electronic sand table is projected into a three-dimensional coordinate space according to the corresponding coordinate information and the plane offset angle information in the target scene information through a scene conversion model.

[0182] It should be noted that the size of the electronic sand table in the target scene information is completely consistent with that in the real scene image information.

[0183] It should be noted that after the target scene information is obtained, the edge of the target scene information can be adjusted to coincide with the edge of the electronic sand table in the real scene image information through a gesture in the AR glasses.

[0184] It can be seen that the method for displaying based on augmented reality described in the embodiment of the application can simultaneously show the real world and virtual information by superimposing virtual search and rescue information on a real terrain sand table environment, and is beneficial to improving the cooperation efficiency and response speed of a search and rescue emergency plan.

[0185] In an optional embodiment, the analyzing and processing the real scene image information and the target scene information to obtain target display information comprises:

[0186] S31, obtaining virtual scene three-dimensional model information;

[0187] It should be noted that the virtual scene three-dimensional model information is three-dimensional model information of a rescue device (such as a rescue plane, a ship, etc.) in a search and rescue plan and corresponding position information, direction information and scale information of the rescue device on an electronic sand table.

[0188] S32, traversing the virtual scene three-dimensional model information to obtain position information, direction information and scale information of the virtual scene three-dimensional model information;

[0189] It should be noted that the above traversal can be implemented by using Unity3D or other existing tools, and the embodiment of the application is not limited in detail.

[0190] S33, projecting the virtual scene three-dimensional model information into the real scene image information and the target scene information according to the position information, the direction information and the scale information, to obtain target display information.

[0191] It should be noted that the projection operation described above is obtained by using the display software on the AR glasses through the simulation software. Through the projection operation, the AR glasses can simultaneously see the map data, the topographic map and the environmental features of the rescue site displayed by the electronic sand table in the real scene image information, and can simultaneously see the state information of the virtual scene three-dimensional model information in the target scene information above the electronic sand table through the AR glasses by projecting the virtual scene three-dimensional model information into the target scene information, such as the current travel position and direction of the rescue aircraft.

[0192] It should be noted that the projection operation described above can be performed by the simulation software set by the user, or can be obtained based on other existing projection technologies, and the embodiments of the present application are not limited.

[0193] It should be noted that the real scene image information is the environmental image (video stream) arranged in time sequence based on the field of view of the user obtained by the AR glasses. After the embodiments of the present application, the target display information obtained is also arranged in time sequence. The target display information is output and displayed in time sequence by the video display device in the AR glasses, and finally a dynamic output video stream is obtained.

[0194] In the present application, the personnel participating in the simulation wear AR glasses, and can interact with the rescue unit through gesture operation and voice instruction after the target display information is output.

[0195] It can be seen that by implementing the display method based on augmented reality described in the embodiments of the present application, the virtual search and rescue information is superimposed in the real terrain sand table environment, and the real world and the virtual information can be seen at the same time, which is beneficial to improving the cooperation efficiency and response speed of the search and rescue emergency plan.

[0196] It should be noted that, in the present application, after obtaining the target display information, the target display information can be kept in the AR glasses through the scene keeping space persistence method, when the user leaves the original target display information, the virtual scene three-dimensional model information in the original target display information is kept in a specific position through the world anchor, when the user returns to the target display information, the virtual scene three-dimensional model information content of the original target display information can be accurately restored. When the AR glasses are running, the current target display information is saved in real time, when the user wears the glasses and returns to the target display information again, the AR glasses can match the current target display information. With the passage of time, the AR glasses can establish a spatial map of the environment it sees. The spatial anchor is used to fix the position of the virtual image on the spatial map.

[0197] Embodiment two

[0198] Please refer to Figure 2 , Figure 2 is a structural schematic diagram of a display device based on augmented reality disclosed by the embodiment of the present application. Among them, Figure 2 The display device based on augmented reality described is applied to an augmented reality-based display optimization system, such as a local server or a cloud server for augmented reality-based display, etc., which is not limited by the embodiment of the present application. As shown in Figure 2 The display device based on augmented reality comprises:

[0199] The acquisition model 201 is used to acquire real scene image information;

[0200] The first calculation module 202 is used to calculate and process the real scene image information to obtain target scene information;

[0201] The second calculation module 203 is used to analyze and process the target scene information to obtain target display information.

[0202] It can be seen that, by implementing the display device based on augmented reality described in the embodiment of the present application, the virtual search and rescue information is superimposed in the real terrain sand table environment, the real world and the virtual information can be seen at the same time, which is beneficial to improve the cooperation efficiency and response speed of the search and rescue emergency plan.

[0203] Embodiment three

[0204] Please refer to Figure 3 , Figure 3 is a structural schematic diagram of another display device based on augmented reality disclosed by the embodiment of the present application. Among them, Figure 3 The display device based on augmented reality described is applied to an augmented reality-based display optimization system, such as a local server or a cloud server for augmented reality-based display, etc., which is not limited by the embodiment of the present application. As shown in Figure 3As shown, the augmented reality-based display device comprises:

[0205] a processor 301;

[0206] a memory 302 coupled to the processor 301 and storing executable program codes;

[0207] The processor 301 invokes the executable program codes stored in the memory 302 to execute part or all of the steps of the augmented reality-based display method described in Embodiment One.

[0208] It can be seen that the augmented reality-based display device described in the embodiments of the present application can simultaneously show the real world and virtual information by superimposing virtual search and rescue information on the real terrain sand table environment, which is conducive to improving the cooperation efficiency and response speed of search and rescue emergency plans.

[0209] Embodiment Four

[0210] The embodiments of the present application disclose a computer readable storage medium, which stores computer instructions, and the computer instructions are used to execute part or all of the steps of the augmented reality-based display method described in Embodiment One when invoked.

[0211] Embodiment Five

[0212] The embodiments of the present application disclose a computer program product, which comprises a non-transitory computer readable storage medium storing a computer program, and the computer program is operable to make a computer execute part or all of the steps of the augmented reality-based display method described in Embodiment One.

[0213] The system embodiments described above are only schematic, wherein the modules illustrated as separate components can or can not be physically separated, and the components illustrated as modules can or can not be physical modules, i.e., can be located in one place or distributed on multiple network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment scheme. Those skilled in the art can understand and implement without creative labor.

[0214] Those skilled in the art can clearly understand the implementation of the various embodiments by means of software and necessary general hardware platforms through the above specific description of the embodiments, and of course, the embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that contributes to the present application can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, a magnetic disk storage, a magnetic tape storage, or any other medium that can be used to carry or store data in a computer readable manner.

[0215] Finally, it should be noted that: the method and device for displaying based on augmented reality disclosed by the embodiments of the present application are only the preferred embodiments of the present application, and are used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An augmented reality-based presentation method, characterized by, The method comprises: S1, acquiring real scene image information; S2, performing calculation processing on the real scene image information to obtain target scene information; S3, performing analysis processing on the real scene image information and the target scene information to obtain target display information; The calculation processing on the real scene image information to obtain target scene information comprises: S21, performing preprocessing on the real scene image information to obtain preprocessed image information; S22, performing analysis processing on the preprocessed image information to obtain first angle information; S23, performing calculation processing on the first angle information to obtain rotation angle information and offset information; S24, performing conversion processing on the rotation angle information and the offset information to obtain target scene information; The calculation processing on the first angle information to obtain rotation angle information and offset information comprises: S231, acquiring second angle information and scene angle training data set; S232, performing training processing on the scene angle training data set to obtain rotation angle offset information and offset information; S233, using a scene angle calculation model to perform calculation processing on the first angle information and the second angle information to obtain scene matrix information; The scene angle calculation model is: In the formula, H represents the scene matrix information, H i represents the i-th scene matrix in the scene matrix information, H i is a 3*3 matrix, (JDYX i,j , JDYYi i,j ) represents the j-th angle coordinate information corresponding to the i-th angle information in the first angle information, (JDEX i,j , JDEYi i,j ) represents the j-th angle coordinate information corresponding to the second angle information, and N represents the number of angle information in the first angle information. S234, performing calculation processing on the rotation angle offset information, the offset information, and the scene matrix information to obtain rotation angle information and offset information; The conversion processing on the rotation angle information and the offset information to obtain target scene information comprises: S241, acquiring target coordinate information, target resolution information, and target size information; S242, using a scene conversion model to perform conversion processing on the target coordinate information, the target resolution information, the target size information, the rotation angle information, and the offset information to obtain target scene information; The scene conversion model is: Z=YS; PYJ=θ′+t′; In the formula, (X, Y, Z) is corresponding coordinate information in the target scene information, PYJ is corresponding plane offset angle information in the target scene information, X k1 and Y k2 are the k1th X coordinate value and the k2th Y coordinate value in the corresponding coordinate information in the target scene information, X′ k1 and Y′ k2 are the k1th X coordinate value and the k2th Y coordinate value in the target coordinate information, YS is the Z coordinate value, XH and YH are the horizontal resolution and the vertical resolution in the target resolution information, L and D are the horizontal size information and the vertical size information in the target size information, M1 and M2 are the number of X coordinate values and the number of Y coordinate values in the target coordinate information, θ′ and t′ are the rotation angle information and the offset information.

2. The augmented reality-based presentation method according to claim 1, characterized by, The analysis processing on the preprocessed image information to obtain first angle information comprises: S221, performing contour detection processing on the preprocessed image information to obtain contour information; S222, performing screening processing on the contour information to obtain target contour information; S223, performing calculation processing on the preprocessed image information and the target contour information to obtain first angle information.

3. The augmented reality-based presentation method of claim 1, wherein, The calculation processing on the rotation angle offset information, the offset information, and the scene matrix information to obtain rotation angle information and offset information comprises: S2341, using a view angle offset calculation model to perform calculation processing on the scene matrix information to obtain preprocessed rotation angle information and preprocessed offset information; The view angle offset calculation model is: (θx i1 ,θy i1 ) = (δ1 · arctan(H i1,32 ), δ2 · arctan(H i1,31 )) ; where (θx, θy) is the pre-processing rotation angle information, (tx, ty) is the pre-processing offset information, (θx i1 , θy i1 ) is the i1th rotation angle coordinate information in the pre-processing rotation angle information, (tx i1 , ty i1 ) is the i1th offset coordinate information in the pre-processing offset information, H is the scene matrix information, H i1,23 , H i1,31 , H i1,32 , and H i1,33 are the value of the 2nd row and the 3rd column, the value of the 3rd row and the 1st column, the value of the 3rd row and the 2nd column, and the value of the 3rd row and the 3rd column of the i1th scene matrix in the scene matrix information, δ1, δ2, δ3, and δ4 are respectively a first weight parameter, a second weight parameter, a third weight parameter, and a fourth weight parameter, 1≤i1≤N, and N is the number of scene matrices in the scene matrix information. S2342, using a view angle offset calculation model to perform calculation processing on the rotation angle offset information, the offset information, the preprocessed rotation angle information, and the preprocessed offset information to obtain rotation angle information and offset information; The view angle offset amount offset calculation model is: θ′=δ5·θ+Δθ; t′=δ6·t+Δt; In the formula, θ′ is the rotation angle information, t′ is the offset amount information, θ and t are the preprocessed rotation angle information and the preprocessed offset amount information respectively, Δθ and Δt are the rotation angle offset information and the offset amount offset information respectively, and δ5 and δ6 are the fifth weight parameter and the sixth weight parameter respectively.

4. The augmented reality-based presentation method of claim 1, wherein, The analysis and processing of the real scene image information and the target scene information obtains target display information, and the analysis and processing includes: S31, obtaining virtual scene three-dimensional model information; S32, traversing the virtual scene three-dimensional model information to obtain position information, direction information and scale information of the virtual scene three-dimensional model information; S33, projecting the virtual scene three-dimensional model information into the real scene image information and the target scene information according to the position information, the direction information and the scale information to obtain target display information.

5. An augmented reality based presentation device, characterized by, The device includes: a processor; a memory coupled to the processor and storing executable program codes; the processor invokes the executable program codes stored in the memory to execute the augmented reality-based display method according to any one of claims 1-4.

6. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, and the computer instructions are used to execute the augmented reality-based display method according to any one of claims 1-4 when invoked.

Images