Virtual simulation training method and system based on AR technology

Abstract

The present application belongs to the technical field of virtual simulation, and provides a virtual simulation training method and system based on AR technology, which comprises the following steps: eliminating redundant prompt information in prompt information by combining training scene image data and interactive behavior data, retaining key prompt information, reducing image occlusion of AR scene training operation, judging whether key prompt information is affected by occlusion operation, if so, replacing the prompt area of key prompt information, which is conducive to the timely attention of users to key prompt information, analyzing the distance between the operation area in the training scene image and the corresponding prompt area of key prompt information, judging whether the distribution of the current key prompt information is reasonable, if not, optimizing the corresponding prompt area of key prompt information, thereby improving the training operation efficiency of users and improving the touch rate of key prompt.

Description

Technical Field

[0001] This invention belongs to the field of virtual simulation technology, specifically a virtual simulation training method and system based on AR technology. Background Technology

[0002] With the deep application of augmented reality (AR) technology in high-precision operational scenarios such as military training, industrial maintenance, and medical simulation, AR training systems have become a core tool for improving users' skill proficiency and efficiency in complex task execution. However, existing AR training systems still face many shortcomings in terms of human-computer interaction efficiency and the rationality of information presentation: First, the problem of static prompt information overload is prominent. Systems generally adopt a fixed prompt layout with full overlay, resulting in redundant information (such as general operating instructions) mixed with key information instructions. Experimental data shows that when the prompt density exceeds 0.3 prompts per degree of field of view, user task efficiency drops significantly. At the same time, prompts in fixed positions may obscure key operating areas such as device interfaces and dashboards for a long time, with an obstruction rate of 15%-20%. Second, the adaptability between prompt information and prompt area is low. Prompt information has a priority based on importance. Traditional information presentation lacks a reasonable design and division of prompt areas based on the distance between the prompt area and the operating area and the priority of the prompt information. For example, high-critical prompts are far from the operating area while low-critical prompts are close to the operating area, resulting in low reach of key prompts and low training efficiency.

[0003] Therefore, the present invention provides a virtual simulation training method and system based on AR technology. Summary of the Invention

[0004] In order to overcome the shortcomings of the prior art, at least one technical problem raised in the background art is solved.

[0005] The technical solution adopted by this invention to solve its technical problem is: a virtual simulation training method based on AR technology, comprising:

[0006] Multiple times during different stages of AR training, the system collects prompts and training scene image data from the user's perspective, and simultaneously records the user's interactive behavior data when viewing the training scene image data.

[0007] By combining training scene image data and interaction behavior data, redundant prompts are removed from the prompt information, and key prompts are retained;

[0008] Based on the prompt area of ​​the key prompt information and the operation area in the training scene image, a region comparison is performed to determine whether the key prompt information has the effect of occlusion on the operation.

[0009] If there is any obstruction during operation, the prompt area for key information will be changed.

[0010] After the replacement of the prompt area of the key prompt information, the distance between the operation area in the training scene image and the prompt area corresponding to the key prompt information is analyzed to determine whether the distribution of the current key prompt information is reasonable. If not, the prompt area corresponding to the key prompt information is optimized.

[0011] Further, the acquisition method of the redundant prompt information is:

[0012] According to the training scene image data and the interactive behavior data, a training prompt attention value and a training operation response value are obtained by processing and analysis, and a training operation performance value is summed;

[0013] If the training operation performance value is greater than the training operation performance threshold, the prompt information is marked as suspected redundant information;

[0014] The proportion of the number of times that the prompt information is marked as suspected redundant information in multiple AR training is counted. If the proportion reaches a preset number of times, the prompt information is marked as redundant prompt information, otherwise, it is marked as key prompt information.

[0015] Further, the acquisition method of the training prompt attention value is:

[0016] According to the training scene image data and the interactive behavior data, the gaze dwell time of the user for the prompt information when performing the training operation in the training scene image is determined, and a ratio processing is performed with the specified completion time of the training operation to obtain the training prompt attention value.

[0017] Further, the acquisition method of the training operation response value is:

[0018] According to the training scene image data and the interactive behavior data, the operation response time of the user training operation is determined, and a ratio processing is performed with the specified completion time of the training operation to obtain the training operation response value.

[0019] Further, the process of determining whether the key prompt information is affected by the shielding operation includes:

[0020] According to the position area of each prompt information in the training scene image, the prompt area of the key prompt information is determined;

[0021] If the prompt area of the key prompt information is completely within the operation area in the training scene image, it means that the key prompt information is affected by the shielding operation;

[0022] If the prompt area of the key prompt information is completely not within the operation area in the training scene image, it means that the key prompt information is not affected by the shielding operation;

[0023] If the prompt region of the key prompt information is not completely in the operation region in the training scene image, the overlapping region between the prompt region of the key prompt information and the operation region in the training scene image is marked, and the area proportion of the overlapping region is calculated;

[0024] If the area proportion is greater than the preset area proportion, it indicates that the key prompt information is affected by the shielding operation.

[0025] If the area proportion is less than or equal to the preset area proportion, it indicates that the key prompt information is not affected by the shielding operation.

[0026] Further, the key prompt information prompt region replacement processing process comprises:

[0027] In the training scene image, other regions except the operation region of the key prompt information are marked as temporary non-operation regions;

[0028] If there is no other key prompt information in the temporary non-operation region, the temporary non-operation region is marked as a temporary idle region;

[0029] The center point distance between the temporary idle region and the operation region of the key prompt information is measured respectively to obtain a prompt distance, and the temporary idle region with the smallest prompt distance is selected as the prompt region of the key prompt information.

[0030] Further, the process of judging whether the distribution of the current key prompt information is reasonable comprises:

[0031] The prompt regions of the key prompt information are respectively numbered once and twice;

[0032] The region numbers of the prompt regions of the key prompt information in the first numbering and the second numbering are obtained respectively, and are integrated into a region first numbering group and a region second numbering group respectively;

[0033] The Euclidean distance between the first numbering group and the second numbering group is calculated;

[0034] If the Euclidean distance between the first numbering group and the second numbering group is greater than the Euclidean distance threshold, it indicates that the distribution of the current key prompt information is unreasonable, otherwise, it indicates that the distribution of the current key prompt information is reasonable.

[0035] Further, the first numbering process comprises:

[0036] The prompt regions of the key prompt information are sorted according to the center point distance between the prompt region and the operation region from small to large, a prompt region sequence is obtained, and the prompt regions are numbered in turn according to the sorting result;

[0037] The second numbering process comprises:

[0038] The prompt regions of the respective key prompt information are sorted according to the proportion of the number of times that the key prompt information is marked as suspected redundant information from small to large, and the prompt regions are numbered in sequence according to the sorting result.

[0039] Further, the process of optimizing the prompt regions corresponding to the key prompt information is:

[0040] The key prompt information is sorted according to the proportion of the number of times that the key prompt information is marked as suspected redundant information from small to large, and a key prompt information sequence is obtained.

[0041] The key prompt information is sequentially selected in the key prompt information sequence, and the prompt regions are sequentially selected in the prompt region sequence, and the sequentially selected key prompt information and prompt regions are sequentially combined according to the selection order, to complete the optimization of the prompt regions corresponding to the key prompt information.

[0042] A virtual simulation training system based on AR technology, comprising:

[0043] An AR data acquisition module: multiple times in different links of AR training, collect prompt information and training scene image data contained in the user's perspective, and record the user's interactive behavior data when seeing the training scene image data;

[0044] An information redundancy processing module: combined with the training scene image data and the interactive behavior data, the redundant prompt information in the prompt information is removed, and the key prompt information is retained;

[0045] A prompt occlusion analysis module: according to the prompt region of the key prompt information and the operation region in the training scene image, the regions are compared, and it is judged whether the key prompt information is affected by the occlusion operation;

[0046] A prompt region replacement module: if the key prompt information is affected by the occlusion operation, the prompt region of the key prompt information is replaced;

[0047] A prompt optimization processing module: after the prompt region of the key prompt information is replaced, the distance between the operation region in the training scene image and the prompt region corresponding to the key prompt information is analyzed, it is judged whether the distribution of the current key prompt information is reasonable, and if not, the optimization of the prompt region corresponding to the key prompt information is performed.

[0048] The beneficial effects of the present application are as follows:

[0049] 1. Collecting prompt information and training scene image data under the user's perspective in different AR training links, synchronously recording the user's interactive behavior data when seeing the training scene image data, eliminating redundant prompt information in the prompt information, retaining key prompt information, eliminating redundant prompt information, reducing the amount of prompt information, preventing prompt information overload, improving AR training operation efficiency, and reducing image occlusion in AR scene training operation.

[0050] 2. According to the prompt area of the key prompt information and the operation area in the training scene image, the region is compared to determine whether the key prompt information is affected by the occlusion operation, if the key prompt information is affected by the occlusion operation, the prompt area of the key prompt information is replaced to the temporary idle area where there is no other key prompt information, and the distance from the replaced temporary idle area to the operation area is the closest, preventing multiple key prompt information in the same prompt area from affecting the user's attention during training operation and facilitating the user's timely attention to the key prompt information.

[0051] 3. After replacing the prompt area of the key prompt information, the distance between the operation area in the training scene image and the corresponding prompt area of the key prompt information is analyzed to determine whether the distribution of the current key prompt information is reasonable, if not, the corresponding prompt area of the key prompt information is optimized to improve the rationality of the corresponding prompt area of the key prompt information, thereby improving the user's training operation efficiency and improving the reach rate of the key prompt. BRIEF DESCRIPTION OF DRAWINGS

[0052] The application will be further described below with reference to the accompanying drawings.

[0053] Figure 1 is a step flow chart of a virtual simulation training method based on AR technology according to an embodiment of the application;

[0054] Figure 2 is a program block diagram of a virtual simulation training system based on AR technology according to an embodiment of the application. DETAILED DESCRIPTION

[0055] In order to make the technical means, creative features, purposes and effects of the application easy to understand, the application will be further described below with reference to the specific embodiments.

[0056] Embodiment 1: Please refer to Figure 1 The virtual simulation training method based on AR technology according to an embodiment of the application includes the following steps:

[0057] Step one: collect the prompt information and training scene image data contained in the user's perspective in different AR training links, and record the user's interactive behavior data when seeing the training scene image data;

[0058] The prompt information includes warning slogans, prompt slogans, and prompt icons, and the position area of the prompt information in the training scene image. The training scene image data includes training scene images of different AR training links and the training operation area of the user in the training scene image.

[0059] For example, in mechanical maintenance AR training, training scene images of different operation steps such as disassembling parts and installing components are collected.

[0060] The interactive behavior data includes the user's visual fixation position, visual fixation duration, and operation response duration (actual completion time of training operation) when seeing the training scene image and performing the training operation. The visual fixation position is recorded by an eye tracker to determine the user's visual fixation position when seeing the training scene image and record the visual fixation duration.

[0061] Step two: combine the training scene image data and the interactive behavior data to remove redundant prompt information and retain key prompt information.

[0062] Specifically, in step two, the redundant prompt information is obtained as follows:

[0063] In any one link of AR training, the visual fixation duration of the prompt information when the user sees the training scene image and performs the training operation and the operation response duration of the corresponding training operation are determined according to the visual fixation position and the visual fixation duration.

[0064] The visual fixation duration of the prompt information when the user sees the training scene image and performs the training operation is compared with the specified completion duration of the training operation to obtain a training prompt attention value.

[0065] The operation response duration of the corresponding training operation is compared with the specified completion duration of the training operation to obtain a training operation response value.

[0066] It should be noted that the specified completion duration of the training operation is set in advance according to the training target requirements of different training operations in the training scene.

[0067] The training prompt attention value and the training operation response value are summed to obtain a training operation performance value.

[0068] It can be understood that the training prompt attention value reflects the user's line of sight staying time (attention to the prompt information) when performing the AR simulation training operation, and the shorter the line of sight staying time, the lower the attention. The training operation response value reflects the time ratio of the user performing the AR simulation training operation within the specified completion time. If the user's attention to the prompt information is lower and the time ratio of the training operation completion is shorter, it means that the prompt information may be redundant. For example, when the user is performing mechanical maintenance AR training, the part disassembly sequence prompt information is ignored when disassembling the parts, and the disassembly training operation is completed in a short time. It is concluded that the part disassembly sequence prompt information may be redundant.

[0069] Preferably, the training operation performance value is compared with the training operation performance threshold value.

[0070] If the training operation performance value is greater than the training operation performance threshold value, the prompt information is marked as suspected redundant information.

[0071] If the training operation performance value is less than or equal to the training operation performance threshold value, no operation is performed.

[0072] Based on any one prompt information, the proportion of the number of times that the prompt information is marked as suspected redundant information in multiple AR training is counted, and compared with the preset marking proportion.

[0073] If the proportion of the number of times that the prompt information is marked as suspected redundant information reaches the preset marking proportion, the prompt information is marked as redundant prompt information.

[0074] If the proportion of the number of times that the prompt information is marked as suspected redundant information does not reach the preset marking proportion, the prompt information is marked as key prompt information.

[0075] It can be understood that the purpose of determining whether the prompt information is redundant prompt information through the proportion of the number of times that the prompt information is marked as suspected redundant information is to improve the accuracy of identifying redundant prompt information in the prompt information through multiple redundant markings of the prompt information, prevent inaccurate identification of a single time, and thus cause misjudgment of redundant prompt information, which is not conducive to subsequent AR simulation training operation.

[0076] Based on the redundant prompt information, the redundant prompt information is removed from the prompt information, and the key prompt information is retained in the prompt information.

[0077] It should be noted that the purpose of removing the redundant prompt information in the prompt information and retaining the key prompt information is:

[0078] Effect one: the redundant prompt information is deleted in the prompt information, the key prompt information can be effectively highlighted, the user is more likely to pay attention to the key prompt information during AR training operation, the existence of the redundant prompt information prevents the prompt information from being overloaded, thereby reducing the attention of the user to the key prompt information, and the efficiency of the user in AR training operation is not good;

[0079] Effect two: the elimination of the redundant prompt information reduces the amount of prompt information, reduces the image occlusion of the prompt information to the user during AR scene training operation, and reduces the operation failure rate of the user;

[0080] Effect three: the redundant prompt information is deleted in the prompt information, and the key prompt information is retained, which is beneficial to the reasonable design and planning of the prompt area of the key prompt information, thereby improving the AR training simulation effect;

[0081] The technical scheme of the embodiment of the application is: collecting prompt information and training scene image data contained in the user's visual angle at different links of AR training, synchronously recording the interactive behavior data of the user when seeing the training scene image data, eliminating the redundant prompt information in the prompt information in combination with the training scene image data and the interactive behavior data, retaining the key prompt information, eliminating the redundant prompt information, reducing the amount of prompt information, preventing the prompt information from being overloaded, improving the AR training operation efficiency, and reducing the image occlusion of the AR scene training operation;

[0082] Embodiment 2, please refer to 1, based on the basis of embodiment 1, the virtual simulation training method based on AR technology provided by the embodiment of the application further includes the following steps:

[0083] Step three: according to the prompt area of the key prompt information and the operation area in the training scene image, the region comparison is performed to determine whether the key prompt information is affected by the occlusion operation;

[0084] Based on the key prompt information, the position area of the key prompt information in the training scene image is determined according to the position area of each prompt information in the training scene image, that is, the prompt area of the key prompt information;

[0085] The prompt area of the key prompt information is compared with the operation area in the training scene image;

[0086] If the prompt area of the key prompt information is completely in the operation area in the training scene image, it indicates that the key prompt information is affected by the occlusion operation;

[0087] If the prompt area of the key prompt information is not completely in the operation area in the training scene image, it indicates that the key prompt information is not affected by the occlusion operation, and no operation is performed;

[0088] If the prompt region of the key prompt information is not entirely in the operation region in the training scene image, the overlapping region between the prompt region of the key prompt information and the operation region in the training scene image is marked, and the area proportion between the overlapping region and the operation region in the training scene image is calculated;

[0089] If the area proportion is greater than the preset area proportion, it indicates that the key prompt information is affected by the shielding operation;

[0090] If the area proportion is less than or equal to the preset area proportion, it indicates that the key prompt information is not affected by the shielding operation;

[0091] It should be noted that the key prompt information corresponds to the operation region in the training scene image. Specifically, the key prompt information is the prompt information when the user performs the training operation in the operation region corresponding to the training scene image;

[0092] Step four: if the key prompt information is affected by the operation shielding, the prompt region replacement processing of the key prompt information is performed;

[0093] Specifically, if the key prompt information is affected by the shielding operation, in the training scene image, other regions except the operation region corresponding to the key prompt information are marked as temporary non-operation regions;

[0094] It should be noted that the temporary non-operation region indicates that when the training operation is performed in the operation region corresponding to the key prompt information, the other operation regions are temporarily not operated, and therefore, the temporary non-operation region is marked;

[0095] Based on the temporary non-operation region, if there is no other key prompt information in the temporary non-operation region, the temporary non-operation region is marked as a temporary idle region, and if there is other key prompt information in the temporary non-operation region, the temporary non-operation region is marked as a temporary existing region;

[0096] In the temporary idle region, the center point distance (the distance between the center point of the temporary idle region and the center point of the operation region) between the temporary idle region and the operation region corresponding to the key prompt information is measured, the prompt distance is obtained, the temporary idle region with the smallest prompt distance is selected as the prompt region of the key prompt information, and the key prompt information is replaced to the temporary idle region for information prompting;

[0097] For example, assuming that there are operation areas A, B, C and D, the current operation area A performs training operations, and the operation areas B, C and D do not perform training operations in the current training scene, the operation areas B, C and D are marked as the temporary non-operation area B, the temporary non-operation area C and the temporary non-operation area D respectively, and in the temporary non-operation area B, the temporary non-operation area C and the temporary non-operation area D, no other key prompt information exists in the temporary non-operation area C and the temporary non-operation area D, the center point distances of the temporary non-operation area C and the temporary non-operation area D to the center point of the operation area A are measured, assuming that the center point distance is smaller in the temporary non-operation area C, the key prompt information is placed in the temporary non-operation area C, and the temporary non-operation area C is used as the prompt area of the key prompt information.

[0098] It can be understood that the prompt area of the key prompt information affected by the shielding operation is replaced by the temporary idle area in which no other key prompt information exists, and the temporary idle area is closest to the operation area, on the one hand, the prompt area of the key prompt information is replaced by the temporary idle area in which no other key prompt information exists, which is beneficial to prevent multiple key prompt information from appearing in the same prompt area and affecting the attention of the user during the AR training operation, and on the other hand, the prompt area is replaced by the temporary idle area closest to the operation area, which is beneficial to the timeliness of the attention of the user to the key prompt information.

[0099] The technical scheme of the embodiment of the application is as follows: according to the prompt area of the key prompt information and the operation area in the training scene image, the region comparison is performed to determine whether the key prompt information is affected by the shielding operation, if the key prompt information is affected by the shielding operation, the prompt area replacement processing of the key prompt information is performed, the prompt area of the key prompt information affected by the shielding operation is replaced by the temporary idle area in which no other key prompt information exists, and the temporary idle area is closest to the operation area, so as to prevent multiple key prompt information from appearing in the same prompt area and affecting the attention of the user during the training operation and to be beneficial to the timeliness of the attention of the user to the key prompt information.

[0100] Embodiment 3, please refer to Figure 1 Based on the basis of the embodiments 1 and 2, the virtual simulation training method based on the AR technology provided by the embodiment of the application further includes the following steps:

[0101] Step five: after the prompt area of the key prompt information is replaced, the distance between the operation area in the training scene image and the prompt area corresponding to the key prompt information is analyzed, whether the distribution of the current key prompt information is reasonable is determined, and if not, the prompt area corresponding to the key prompt information is optimized.

[0102] Specifically, the process of distance analysis of the operation region in the training scene image and the prompt region corresponding to the key prompt information is as follows:

[0103] The prompt regions corresponding to the key prompt information are respectively first numbered and second numbered according to the center point distance between the prompt region and the operation region and the proportion of the number of times that the key prompt information is marked as suspected redundant information;

[0104] The first numbering process is as follows:

[0105] The prompt regions corresponding to the key prompt information are sorted according to the center point distance between the prompt region and the operation region from small to large, a prompt region sequence is obtained, and the prompt regions are numbered (first numbering) according to the sorting result;

[0106] The second numbering process is as follows:

[0107] The prompt regions corresponding to the key prompt information are sorted according to the proportion of the number of times that the key prompt information is marked as suspected redundant information from small to large, and the prompt regions are numbered (second numbering) according to the sorting result;

[0108] It should be noted that in the first numbering and the second numbering, if there are two prompt regions with the same order in the sorting result, the numbers of the two prompt regions are the same;

[0109] The region numbers of the prompt regions corresponding to the key prompt information in the first numbering and the second numbering are obtained, respectively, and are integrated into a region first numbering group and a region second numbering group;

[0110] The region first numbering group contains the region numbers of the prompt regions corresponding to the key prompt information in the first numbering, and the region second numbering group contains the region numbers of the prompt regions corresponding to the key prompt information in the second numbering;

[0111] The Euclidean distance between the region first numbering group and the region second numbering group is calculated by using the Euclidean distance method;

[0112] If the Euclidean distance between the region first numbering group and the region second numbering group is greater than the Euclidean distance threshold, it indicates that the distribution of the current key prompt information is unreasonable;

[0113] If the Euclidean distance between the region first numbering group and the region second numbering group is less than or equal to the Euclidean distance threshold, it indicates that the distribution of the current key prompt information is reasonable, and no operation is performed;

[0114] It can be understood that the reason for determining whether the distribution of the current key prompt information is reasonable through the Euclidean distance between the one-numbered group and the region second-numbered group (indicating the deviation between the region one-numbered and the region second-numbered) is that: when the key prompt information machine is one-numbered, the distance between the prompt region and the operation region is numbered, the shorter the distance, the higher the priority, and when the second number is numbered, the number of times that the key prompt information is marked as suspected redundant information is numbered, the smaller the number of times, the higher the importance of the key prompt information, and the higher the priority of the corresponding prompt region, if the Euclidean distance between the one-numbered group and the region second-numbered group is smaller, it indicates that the key prompt information corresponding to the prompt region is more important, the distance from the prompt region to the operation region is shorter, and the distribution of the key prompt information is more reasonable (the more important key prompt information is closer to the operation region).

[0115] If the distribution of the current key prompt information is not reasonable, the key prompt information is sorted according to the number of times of being marked as suspected redundant information from small to large to obtain a key prompt information sequence.

[0116] The key prompt information is sequentially selected in the key prompt information sequence, and the prompt region is sequentially selected in the prompt region sequence, and the sequentially selected key prompt information and the prompt region are sequentially combined according to the selection order to complete the optimization of the key prompt information corresponding to the prompt region.

[0117] The technical scheme of the embodiment of the application is: after replacing the prompt region of the key prompt information, the distance between the operation region in the training scene image and the key prompt information corresponding prompt region is analyzed, whether the distribution of the current key prompt information is reasonable is determined, if not, the optimization of the key prompt information corresponding to the prompt region is performed, thereby improving the rationality of the key prompt information corresponding to the prompt region, thereby improving the user training operation efficiency and improving the reach rate of the key prompt information.

[0118] Embodiment 4: please refer to Figure 2 The virtual simulation training system based on AR technology comprises the following modules:

[0119] AR data acquisition module: multiple times in different links of AR training, collect prompt information and training scene image data contained in the user's perspective, and record the user's interactive behavior data when seeing the training scene image data;

[0120] Information redundancy processing module: combine the training scene image data and the interactive behavior data to remove the redundant prompt information in the prompt information, and retain the key prompt information;

[0121] The hint occlusion analysis module: according to the hint area of the key hint information and the operation area in the training scene image, region comparison is performed to determine whether the key hint information is affected by the occlusion operation;

[0122] The hint area replacement module: if the key hint information is affected by the operation occlusion, the hint area replacement processing of the key hint information is performed;

[0123] The hint optimization processing module: after the hint area replacement of the key hint information, the distance analysis of the operation area in the training scene image and the corresponding hint area of the key hint information is performed to determine whether the distribution of the current key hint information is reasonable, and if not, the optimization of the corresponding hint area of the key hint information is performed.

[0124] The basic principles, main features and advantages of the present application are shown and described above. It should be understood by those skilled in the art that the present application is not limited by the above examples, and the above examples and descriptions in the specification are only to illustrate the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A virtual simulation training method based on AR technology, characterized in that: include: Multiple times during different stages of AR training, the system collects prompts and training scene image data from the user's perspective, and simultaneously records the user's interactive behavior data when viewing the training scene image data. By combining training scene image data and interaction behavior data, redundant prompts are removed from the prompt information, and key prompts are retained; The method for obtaining the redundant prompt information is as follows: Based on the training scene image data and interaction behavior data, the training prompt attention value and training operation response value are processed and analyzed, and the training operation performance value is summed. If the training operation performance value is greater than the training operation performance threshold, the prompt message will be marked as suspected redundant information. The statistical prompt information is marked as suspected redundant information in multiple AR training sessions. If the number of times reaches the preset proportion, the prompt information is marked as redundant; otherwise, it is marked as critical prompt information. The training prompt attention value is obtained in the following way: Based on training scene image data and interaction behavior data, determine the duration of the user's gaze lingering on the prompt information during training operations in the training scene image, and process the ratio with the prescribed completion time of the training operation to obtain the training prompt attention value; The training prompt attention value is obtained in the following way: Based on training scene image data and interaction behavior data, determine the duration of the user's gaze lingering on the prompt information during training operations in the training scene image, and process the ratio with the prescribed completion time of the training operation to obtain the training prompt attention value; Based on the prompt area of ​​the key prompt information and the operation area in the training scene image, a region comparison is performed to determine whether the key prompt information has the effect of occlusion on the operation. If there is any obstruction during operation, the prompt area for key information will be changed. After changing the prompting area of ​​the key prompts, the distance analysis between the operation area and the prompting area corresponding to the key prompts in the training scene image is performed to determine whether the current distribution of each key prompt is reasonable. If it is not reasonable, the prompting area corresponding to the key prompt is optimized.

2. The virtual simulation training method based on AR technology according to claim 1, characterized in that: The process of determining whether the key prompt information is affected by obstruction includes: Based on the location of each prompt in the training scene image, determine the prompt area of ​​the key prompt information; If the prompt area of ​​the key prompt information is completely within the operation area of ​​the training scene image, it indicates that the key prompt information is affected by the operation. If the prompt area of ​​the key prompt information is not located within the operation area of ​​the training scene image, it means that the key prompt information is not affected by occlusion operation. If the prompt area of ​​the key prompt information is not completely within the operation area in the training scene image, then the overlapping area between the prompt area of ​​the key prompt information and the operation area in the training scene image is marked, and the area ratio of the overlapping area is calculated. If the area ratio is greater than the preset area ratio, it means that the key prompt information is being obscured and will affect the operation. If the area ratio is less than or equal to the preset area ratio, it means that the key prompt information is not affected by the operation of obscuring.

3. The virtual simulation training method based on AR technology according to claim 2, characterized in that: The process of changing the prompt area for key prompt information includes: In the training scene images, regions other than the operation areas of key prompt information are marked as transient non-operation regions; If there are no other key prompts within the transient non-operational area, the transient non-operational area will be marked as a transient free area. Measure the distance between the center point of the transient idle area and the operation area of ​​the key prompt information to obtain the prompt distance, and select the transient idle area with the smallest prompt distance as the prompt area of ​​the key prompt information.

4. The virtual simulation training method based on AR technology according to claim 3, characterized in that: The process for determining whether the current distribution of key prompt information is reasonable is as follows: The prompt areas for each key prompt message are numbered both initially and secondarily. Obtain the area numbers of the prompt areas corresponding to each key prompt information in the primary and secondary numbering, and integrate them into the primary area numbering group and the secondary area numbering group respectively; Calculate the Euclidean distance between the primary numbering group and the secondary numbering group of the region; If the Euclidean distance between the primary numbering group and the secondary numbering group of the region is greater than the Euclidean distance threshold, it indicates that the current distribution of key prompts is unreasonable; otherwise, it indicates that the current distribution of key prompts is reasonable.

5. The virtual simulation training method based on AR technology according to claim 4, characterized in that: The numbering process is as follows: The prompt areas of each key prompt information are sorted in ascending order of distance between the center point of the prompt area and the operation area to obtain a prompt area sequence, and the prompt areas are numbered in order of sorting result; The secondary numbering process is as follows: The prompt areas of each key prompt information are sorted from smallest to largest according to the proportion of times the key prompt information is marked as suspected redundant information, and the prompt areas are numbered in order of sorting.

6. The virtual simulation training method based on AR technology according to claim 5, characterized in that: The process of optimizing the prompt area corresponding to the key prompt information is as follows: The key prompts are sorted in ascending order of the proportion of times they are marked as suspected redundant information, resulting in a key prompt sequence. Select key prompts sequentially from the key prompt information sequence, and simultaneously select prompt areas sequentially from the prompt area sequence. Combine the selected key prompts and prompt areas in the order of selection to optimize the prompt areas corresponding to the key prompts.

7. A virtual simulation training system based on AR technology, characterized in that: include: AR data acquisition module: Collects prompt information and training scene image data from the user's perspective at different stages of AR training, and simultaneously records the user's interactive behavior data when seeing the training scene image data; Information redundancy processing module: Combines training scene image data and interaction behavior data to remove redundant prompts from the prompt information and retain key prompt information; The method for obtaining the redundant prompt information is as follows: Based on the training scene image data and interaction behavior data, the training prompt attention value and training operation response value are processed and analyzed, and the training operation performance value is summed. If the training operation performance value is greater than the training operation performance threshold, the prompt message will be marked as suspected redundant information. The statistical prompt information is marked as suspected redundant information in multiple AR training sessions. If the number of times reaches the preset proportion, the prompt information is marked as redundant; otherwise, it is marked as critical prompt information. The training prompt attention value is obtained in the following way: Based on training scene image data and interaction behavior data, determine the duration of the user's gaze lingering on the prompt information during training operations in the training scene image, and process the ratio with the prescribed completion time of the training operation to obtain the training prompt attention value; The training prompt attention value is obtained in the following way: Based on training scene image data and interaction behavior data, determine the duration of the user's gaze lingering on the prompt information during training operations in the training scene image, and process the ratio with the prescribed completion time of the training operation to obtain the training prompt attention value; The prompt occlusion analysis module compares the prompt area of ​​the key prompt information with the operation area in the training scene image to determine whether the key prompt information is affected by occlusion. Prompt Area Replacement Module: If critical prompt information is obstructed by operations, the prompt area of ​​the critical prompt information will be replaced. The prompt optimization processing module: After changing the prompt area of ​​the key prompt information, it performs distance analysis on the operation area in the training scene image and the prompt area corresponding to the key prompt information to determine whether the current distribution of each key prompt information is reasonable. If it is not reasonable, it optimizes the prompt area corresponding to the key prompt information.

Images