Method and system for creating virtual-real combined practical training courses

Abstract

This invention discloses a method and system for creating virtual-real integrated training courses. The method includes, in response to a virtual-real integrated course creation operation, retrieving a target point cloud map from a resource library that matches the point cloud map indicated by the creation operation, and providing the target point cloud map to a course creation page. The point cloud map is pre-built based on the training environment. When a course editing operation is triggered on the course creation page, chapters of the training course are created based on the instructions of the course editing operation, and matching target materials are retrieved from a resource library for each chapter and provided to the creation page. The edited target materials are obtained. The point cloud map data containing the target materials is sent to a smart glasses terminal. By integrating the point cloud map with the required course materials through visual editing, a virtual-real integrated training method can be achieved, improving the training effect.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, specifically to a method and system for creating virtual-real integrated training courses. Background Technology

[0002] In related technologies, both practical training and assessment typically rely solely on manual instruction or online immersive virtual experiences. These methods suffer from the drawback of purely manual training, which fails to provide trainees with an immersive learning experience. Consequently, trainees need to spend considerable time understanding the material, resulting in less than ideal efficiency and effectiveness. Purely online immersive virtual experiences also suffer from the same problem. Summary of the Invention

[0003] The main objective of this invention is to provide a method and system for creating virtual-real integrated training courses, in order to address the shortcomings of related technologies.

[0004] To achieve the above objectives, according to a first aspect of the present invention, a method for creating a virtual-real integrated training course is provided, comprising: responding to a virtual-real integrated course creation operation, retrieving a target point cloud map from a resource library that matches a point cloud map indicated by the creation operation, and providing the target point cloud map to a course creation page, wherein the point cloud map is pre-built based on a training environment; when a course editing operation is triggered on the course creation page, creating chapters of the training course based on the instructions of the course editing operation, and retrieving matching target materials from a material library for each chapter's materials and providing them to the creation page; obtaining the target materials edited by the editing operation; and sending point cloud map data containing the target materials to a smart glasses terminal.

[0005] Optionally, the target material is a model, image, or video. Before binding the target material with the editing information, the method includes: associating animation actions with the target material according to the animation editing information indicated by the editing information; or changing the pose of the target material according to the pose editing information indicated by the editing information; or performing appearance editing on the target material according to the appearance editing information indicated by the editing information; or disassembling and assembling the structure of the target material according to the disassembly and assembly editing information indicated by the editing information.

[0006] Optionally, before binding the target material with the editing information, the method includes: associating events with the target material based on the event information indicated by the editing information; and adding the events associated with the materials in the same chapter to the timeline in the order of playback.

[0007] Optionally, after the target point cloud map is provided to the course creation page, when the location guidance creation operation on the creation page is triggered, the location guidance information is bound to the point cloud map.

[0008] Optionally, when the course is played on the smart glasses, the smart glasses are guided to locate themselves based on the location guidance information via voice prompts; and the location is initialized once the smart glasses' field of vision includes the starting point in the location guidance information.

[0009] Optionally, after the location is initialized, the target material in the point cloud map is executed at its location according to the associated animation action; after the user actually completes the execution of the action, the next event of the current material is triggered, or the next material event is executed.

[0010] According to a second aspect of the present invention, a system for creating virtual-real integrated training courses is provided, characterized in that it comprises: a creation unit, configured to, in response to a virtual-real integrated course creation operation, retrieve a target point cloud map from a resource library that matches the point cloud map indicated by the creation operation, and provide the target point cloud map to a course creation page, wherein the point cloud map is pre-constructed based on a depth map of the training environment; when a course editing operation is triggered on the course creation page, create chapters of the training course based on the instructions of the course editing operation, and retrieve matching target materials from a material library for each chapter's materials and provide them to the creation page; and a sending unit, configured to acquire the target materials edited by the editing operation; and send the point cloud map data containing the target materials to a smart glasses terminal.

[0011] According to a third aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing the computer to perform the method described in any one of the first aspects.

[0012] According to a fourth aspect of the present invention, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform the method described in any implementation of the first aspect.

[0013] According to a fifth aspect of the present invention, a computer program product is provided, which, when executed by a processor, implements the method described in any implementation of the first aspect.

[0014] This embodiment describes a method and system for creating a virtual-real integrated training course. The method includes, in response to a virtual-real integrated course creation operation, retrieving a target point cloud map from a resource library that matches the point cloud map indicated by the creation operation, and providing the target point cloud map to the course creation page. The point cloud map is pre-built based on the training environment. When a course editing operation is triggered on the course creation page, chapters of the training course are created based on the instructions of the course editing operation, and matching target materials are retrieved from a resource library for each chapter and provided to the creation page. The edited target materials are obtained. The point cloud map data containing the target materials is sent to a smart glasses terminal. By integrating the point cloud map with the required course materials through visual editing, a virtual-real integrated training method can be achieved, improving the training effect. Attached Figure Description

[0015] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0016] Figure 1 This is a flowchart of a method for creating a virtual-real integrated training course according to an embodiment of the present invention; Figure 2 , Figure 3 This is a schematic diagram illustrating the application of the method for creating a virtual-real integrated training course according to an embodiment of the present invention; Figure 4 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0018] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of the invention described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] According to embodiments of the present invention, a method for creating virtual-real integrated practical training courses is provided, such as... Figure 1 As shown, steps 101 to 103 are included below: Step 101: In response to the virtual-real course creation operation, retrieve the target point cloud map that matches the point cloud map indicated by the creation operation from the resource library, and provide the target point cloud map to the course creation page, wherein the point cloud map is pre-built based on the training environment.

[0021] In this step, users can trigger the course creation operation on a visual page (such as the page of an APP application or a web page). When triggering this operation, they can first create a course name, such as "Emergency Response Training", and select a course type, such as virtual and real combination type. Under this type of course, they can select a point cloud map that has been pre-generated in the resource library and matches the training environment of the training. This point cloud map can be generated by the video uploaded by the user on the page (taken by a depth information camera) or by a panoramic image taken by a panoramic design camera.

[0022] Point cloud maps contain actual environmental information. If the training involves operating equipment, the point cloud map will contain the equipment to be operated. Each point cloud map, after generation, can be uniquely identified and stored in a resource library for easy retrieval. Furthermore, to ensure accurate positioning and match the point cloud map with the physical space, initial positioning point identification and location calibration are necessary. For example, marker-assisted SLAM technology can be used. By pre-setting key visual markers in the training environment, image recognition algorithms can quickly detect and locate these markers, achieving the restoration of the preset location and physical spatial scale. SLAM initialization is completed based on the known physical coordinates of the markers (e.g., a 1m spacing), precisely aligning the virtual coordinate system with the real-world physical space. This method effectively suppresses the linear error accumulation problem of pure SLAM, ensuring stable positioning accuracy over long-term use.

[0023] refer to Figure 2 The illustration shows a point cloud map created on the creation page. Within this point cloud map, users can create materials in an interactive way, such as adding or editing materials.

[0024] The materials are stored in a material library, and different material content can be designed according to the training or assessment needs of different industries. In other words, the materials in the database can be customized, added, reused, and deleted by users. The materials can be pre-built models (such as tool models and gesture guidance models for different industries), images (arrow guidance images, highlighted marker images), or multimedia, audio, etc., and can also include custom content, such as data panels.

[0025] Once the materials are added to the point cloud map, they are easily integrated into the map. For example, the point cloud map acts as a spatial coordinate system, within which the actual point cloud data forms the map. Each material is initialized at the origin when added to the point cloud coordinate system. During course editing, the position of the materials is defined through movement and rotation along the x, y, and z axes, thus establishing a relative positional relationship between the materials and the point cloud map.

[0026] Furthermore, point cloud data is characterized by its massive volume and dense point density. Overlaying it with material rendering and real-time interactive operations can easily lead to page lag, loading delays, and browser crashes. Therefore, hierarchical level of detail (LOD) rendering can be implemented based on viewing distance and field of view to balance rendering quality and performance overhead. When viewing point clouds up close or adding materials, high-precision point cloud data is loaded to ensure detail clarity; when browsing the scene from a distance, low-precision point clouds are automatically switched to reduce the number of rendered points; for point cloud areas outside the field of view, rendering is paused to release GPU and memory resources. Simultaneously, the viewing rate is bound to the rendering precision; when the view is quickly dragged, the rendering precision is temporarily reduced, and normal precision is restored when the view is still, balancing interactive smoothness and visual effects. Considering that all page-side interactive operations ultimately translate into camera pose changes, and the camera's real-time parameters directly determine the point cloud rendering strategy and interactive feedback, the two are deeply bound and inseparable. Therefore, clear thresholds can be set in the rendering loop to trigger corresponding rendering and interactive strategies, including: Distance Determination: Switching the point cloud precision level is directly triggered by the distance between the camera and the point cloud, aligning with the needs of interactive scenarios—for close-range fine operations, the camera approaches the scene, loading a high-precision point cloud; for distant browsing, the camera zooms out, switching to a low-precision point cloud. The system calculates the Euclidean distance between the camera viewpoint and the center of the point cloud tile bounding box, setting three distance thresholds (e.g., close distance ≤ 5m, medium distance 5-20m, distant distance > 20m). For close distances, high-precision tiles are automatically loaded; for medium distances, medium precision is switched; and for distant distances, low precision is loaded.

[0027] View of field determination: Controls the start and stop of rendering. Through the camera frustum culling algorithm, it determines whether the point cloud tiles are within the current view of field. Tiles outside the view of field are directly paused for rendering and unloaded from memory. Only the rendering schedule of tiles within the view of field is retained, which greatly reduces invalid rendering.

[0028] For example, to achieve dynamic precision switching, multi-level point cloud data can be pre-splittered, abandoning the single-precision full-load mode. The original point cloud undergoes hierarchical downsampling to generate 3-4 level precision tiles (which can be increased or decreased depending on scene complexity), and spatial segmentation is completed: the high-precision level retains over 90% of the original point cloud points, removing only extremely redundant noise, for adding close-up materials, fine-tuning the scene, and ensuring clarity for device model placement and detail verification; the medium-precision level voxels are downsampled and compressed to 30%-50% of the original points, preserving scene outlines and key structures, for mid-range general browsing; the low-precision level is significantly downsampled to less than 10% of the original points, preserving only the general scene outline, for long-range global browsing. Simultaneously, spatial segmentation and slicing are implemented: using existing tools, the multi-level precision point cloud is cut into regular spatial tiles, and tile coordinates are bound to bounding boxes, enabling independent loading and unloading of individual tiles, paving the way for rendering pauses outside the field of view.

[0029] During the rendering loop, the camera status is monitored in real time, and clear judgment thresholds are set to trigger corresponding rendering strategies. The thresholds can be flexibly adjusted according to the scene size and device configuration. The core monitoring items and judgment rules are as follows: calculate the Euclidean distance between the camera viewpoint and the center of the point cloud tile bounding box, automatically load high-precision tiles at close range, switch to medium precision at medium range, and load low precision at long range; through the camera frustum culling algorithm, determine whether the point cloud tile is within the current field of view. Tiles outside the field of view are directly paused for rendering and unloaded from memory, and only the rendering scheduling of tiles within the field of view is retained, which greatly reduces invalid rendering.

[0030] Step 102: When the course editing operation is triggered on the course creation page, the chapters of the training course are created based on the instructions of the course editing operation, and the target materials that match the materials contained in each chapter are called from the material library and provided to the creation page.

[0031] In this embodiment, the course can be completed through multiple operation steps. Each operation can be considered a chapter, and each chapter can contain multiple materials, for reference. Figure 2 The two newly added chapters are indicated by the white rectangle at the bottom of the page.

[0032] After the source material is provided to the point cloud map, refer to Figure 3 The illustration shows that you can edit the added materials, such as insulated bucket trucks, green button models, and images.

[0033] Step 103: Obtain the target material after editing; send the point cloud map data containing the target material to the smart glasses terminal.

[0034] In this step, the materials are edited in the creation interface. After editing, they can be published. The published course can be sent to the smart glasses terminal, where it plays in chapter order. Events within each chapter can be played in chronological order. Furthermore, besides playing in chapter order, trigger times can be set based on chapter playback. These trigger times can be triggered when the user performs different interactive gestures such as touching, clicking, or pinching, indicating the chapter to be played. For example, touching the engine will take you to the engine learning chapter, and touching the gearbox will take you to the gearbox learning chapter. Trigger times can also be set to automatically switch to the next chapter after the audio / animation finishes playing. In other words, switching to different chapters can be implemented in various ways, and no specific method is limited here.

[0035] Furthermore, point clouds are three-dimensional spatial data, unlike conventional 2D planar interactions. Mouse / touch commands are prone to spatial positioning errors, accidental touches, and misoperations. The core reason for inaccurate interaction is that mouse commands cannot be accurately mapped to the three-dimensional space of the point cloud. Precise positioning can be achieved through ray picking and intelligent snapping. Based on a ray-picking mechanism, a directional ray is emitted when the mouse is clicked. The coordinates of the intersection point between the ray and the point cloud space are calculated in real time, invalid aerial points are eliminated, and the position of the point cloud entity surface is locked (by listening to mouse movement, click, and drag events, converting the 2D mouse coordinates on the page into a 3D ray in real time, emitting the ray to detect the intersection point with the point cloud, filtering invalid aerial intersection points, and locking the coordinates of the point cloud entity surface). The core of this step is to complete the entire link transformation from 2D screen coordinates to 3D ray from the camera's perspective, to point cloud intersection calculation, to invalid point filtering, and to valid coordinate locking.

[0036] For example, this can be achieved using a ray picker: the mouse coordinates on the page are 2D coordinates based on screen pixels, which cannot be directly used for 3D spatial calculations. They need to be normalized first and converted into standardized device coordinates that the camera can recognize: the mouse's horizontal coordinate is mapped to the range [-1,1], the vertical coordinate is mapped to the range [-1,1], and the origin is aligned with the center of the canvas, conforming to the rules of the WebGL 3D coordinate system. This is the basis for coordinate transformation and avoids ray direction offset.

[0037] Furthermore, by binding the normalized mouse coordinates to the scene camera, a directional 3D ray is automatically generated, starting from the camera viewpoint, passing through the corresponding screen point of the mouse, and extending into the point cloud scene. After the ray is generated, the intersection relationship between the ray and the point cloud model is detected, the point cloud points are traversed, the shortest distance between the ray and the point cloud is calculated, the closest valid intersection point is selected, and the 3D spatial coordinates (X / Y / Z) of the intersection point are obtained. These coordinates are the corresponding landing point of the mouse in the point cloud scene, realizing a precise mapping from 2D interaction to 3D space.

[0038] Furthermore, not all ray intersections are valid. Two types of invalid points need to be eliminated: first, points where the ray does not hit the point cloud and only passes through the air without intersection; and second, invalid points where the point cloud around the intersection is sparse and consists of noise / fragmented surfaces. Only when a valid intersection is detected will the 3D coordinates be locked as the reference for material placement. If there are no valid intersections, the area is determined to be an invalid region in the air, the material is hidden, and placement is prohibited, thus eliminating the problem of model suspension and misalignment at the source.

[0039] For material placement, a plane snapping and normal alignment function has been added to automatically identify effective planes such as ground, walls, and equipment bases in the point cloud scene. When the model is dragged to the corresponding area, it automatically snaps to the plane, eliminating problems such as model suspension, clipping, and offset. (Based on existing algorithms, it can detect point cloud planes around the intersection point, identify effective placement surfaces such as ground and walls, calculate plane normals, and drive the model to automatically snap to the plane and align with the normals, completely solving the problems of suspension and clipping, while filtering out invalid fragmented surfaces.)

[0040] As an optional implementation of this embodiment, the animation action is associated with the target material according to the animation editing information indicated by the editing information; or the pose of the target material is changed according to the pose editing information indicated by the editing information; or the appearance of the target material is edited according to the appearance editing information indicated by the editing information; or the structure of the target material is disassembled and assembled according to the disassembly and assembly editing information indicated by the editing information.

[0041] In this optional implementation, refer to Figure 3 You can edit any material using the components on the page, including appearance editing, animation editing, pose editing, assembly / disassembly editing, and event editing.

[0042] Animation editing allows materials to execute corresponding actions during course playback, such as rotation and translation. Pose-based editing adjusts the final pose and orientation of the materials within the point cloud map. Appearance editing adjusts the appearance of the materials, such as color and size. Disassembly and assembly allow for the decomposition and merging of equipment structures. This is primarily used for simulated training in disassembly and assembly, helping trainees understand equipment structure and learn correct disassembly / assembly methods. After disassembly, it's easier to add materials and events to the disassembled components.

[0043] The practical training course needs to provide trainees with an immersive way to operate the equipment. Therefore, there are materials in the corresponding positions on the equipment. For example, if a screw needs to be tightened in a certain position during the training, a wrench model material can be placed at the screw position in the course design. Therefore, the position of the material can be set in any position that needs to appear in the point cloud map, including the equipment that needs to be operated during the training. When set on the equipment, the position of the model is related to the equipment being operated in the point cloud map.

[0044] As an optional implementation of this embodiment, before binding the target material with the editing information, the method includes: associating events with the target material based on the event information indicated by the editing information; and adding the events associated with the materials in the same chapter to the timeline in the order of playback.

[0045] In this optional implementation, events can be added to each piece of material. An event refers to an action to be performed on the material, and the time of each event can be recorded using a timeline.

[0046] For example, taking the engine learning section of Chapter 1 as an example, regarding the engine model: Event 1: Engine model flashing + text and voice prompts (e.g., "This is a car engine"). Event 2: Engine model displacement animation: The engine model moves out of the vehicle model for easy visual understanding by trainees. Event 3: Engine model rotation animation: The engine model rotates, allowing trainees to see the entire engine outline. Event 4: Engine model disassembly… voice guidance during the operation process, such as "Put the wrench back."

[0047] As an optional implementation of this embodiment, after the target point cloud map is provided to the course creation page, when the location guidance creation operation on the creation page is triggered, the location guidance information is bound to the point cloud map.

[0048] In this optional implementation, after importing the point cloud map, positioning guidance information needs to be added to the point cloud map, including the positioning starting point, to enable the smart glasses to initialize their location at the starting point. It also includes guiding voice prompts to direct the smart glasses to the starting point. The smart glasses will find the positioning starting point in real time through visual recognition, and when a person sees the location shown in the positioning guidance map, the smart glasses will automatically complete the recognition.

[0049] As an optional implementation method in this embodiment, after the location is initialized, the target material in the point cloud map is executed at its location according to the associated animation action; after the user actually completes the execution of the action, the next event of the current material is triggered, or the next material event is executed.

[0050] In this optional implementation, courses can be played in chapter order, with each material within the same chapter played in the order of the edited events.

[0051] refer to Figure 4 The illustration shows the view behind the smart glasses when the user wears them. It includes a hand model guiding the user's hand movements, as well as video and text guiding the user's actions. This course allows users to complete practical training or assessments in a hybrid virtual and real-world format.

[0052] The next material event can be triggered automatically after the completion of this material event, or it can be triggered manually.

[0053] Automatic triggering, using car learning as an example: Voice prompt: Please touch the part you want to learn about. Touching the engine will take you to the engine learning chapter. Touching the transmission will take you to the transmission learning chapter. You can set it to automatically switch to the next chapter after the student performs a touch / click or after the voice / animation finishes playing.

[0054] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0055] According to an embodiment of the present invention, a system for creating virtual-real integrated training courses is also provided, comprising: a creation unit, configured to, in response to a virtual-real integrated course creation operation, retrieve a target point cloud map from a resource library that matches the point cloud map indicated by the creation operation, and provide the target point cloud map to a course creation page, wherein the point cloud map is pre-constructed based on a depth map of the training environment; when a course editing operation is triggered on the course creation page, create chapters of the training course based on the instructions of the course editing operation, and retrieve matching target materials from a material library for each chapter's materials and provide them to the creation page; and a sending unit, configured to acquire the target materials edited by the editing operation; and send the point cloud map data containing the target materials to a smart glasses terminal.

[0056] According to embodiments of the present invention, the present invention also provides an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the methods described in any of the above embodiments.

[0057] According to embodiments of the present invention, the present invention also provides a readable storage medium storing computer instructions that enable a computer to perform the methods described in any of the above embodiments when executed.

[0058] According to embodiments of the present invention, the present invention also provides a computer program product that, when executed by a processor, can implement the methods described in any of the above embodiments.

[0059] Figure 4 A schematic block diagram of an example electronic device 300 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices.

[0060] like Figure 4As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of the electronic device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0061] Multiple components in electronic device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of displays, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows electronic device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0062] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as the object matching method. For example, in some embodiments, the object matching method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of the methods described above may be performed.

[0063] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0064] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0065] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

Claims

1. A method for virtual-real combined practical training course creation, characterized in that, include: In response to the creation operation of the virtual-real course, a target point cloud map that matches the point cloud map indicated by the creation operation is retrieved from the resource library, and the target point cloud map is provided to the course creation page. The point cloud map is pre-built based on the training environment. When the course editing operation is triggered on the course creation page, the chapters of the training course are created based on the instructions of the course editing operation, and the target materials that match the materials contained in each chapter are called from the material library and provided to the creation page; Acquire the target material after editing; send the point cloud map data containing the target material to the smart glasses terminal.

2. The method for virtual-real combined practical training course creation according to claim 1, characterized in that, The target material is a model, image, or video. Before binding the target material to the editing information, the method includes: Based on the animation editing information indicated by the editing information, associate animation actions with the target material according to the animation editing information; or based on the pose editing information indicated by the editing information, change the pose of the target material; or based on the appearance editing information indicated by the editing information, perform appearance editing on the target material; or based on the disassembly and assembly editing information indicated by the editing information, disassemble and assemble the structure of the target material.

3. The method for virtual-real combined practical training course creation according to claim 2, characterized in that, Before binding the target material to the editing information, the method includes: Based on the event information indicated by the editing information, associate events with the target material; Add events related to materials in the same chapter to the timeline in the order they are played.

4. The method for virtual-real combined practical training course creation according to claim 3, characterized in that, After the target point cloud map is provided to the course creation page, when the location guidance creation operation on the creation page is triggered, the location guidance information is bound to the point cloud map.

5. The method for virtual-real combined practical training course creation according to claim 4, characterized in that, When the course is played on the smart glasses, the smart glasses are guided to locate themselves based on the location guidance information via voice prompts; and the location initialization is completed when the smart glasses' field of vision includes the starting point in the location guidance information.

6. The method for virtual-real combined practical training course creation according to claim 4, characterized in that, Once the location is initialized, the target material in the point cloud map will perform actions at its location according to the associated animation actions; Once the user has actually completed the action, the next event of the current material is triggered, or the next material event is executed.

7. A system for creating virtual-real integrated practical training courses, characterized in that, include: A creation unit is used to respond to the creation operation of a virtual-real integrated course, retrieve a target point cloud map that matches the point cloud map indicated by the creation operation from the resource library, and provide the target point cloud map to the course creation page. The point cloud map is pre-built based on the depth map of the training environment. When a course editing operation is triggered on the course creation page, chapters of the training course are created based on the instructions of the course editing operation, and target materials that match the materials contained in each chapter are retrieved from the material library and provided to the creation page. The sending unit is used to acquire the target material after the editing operation; and to send the point cloud map data containing the target material to the smart glasses terminal.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the method according to any one of claims 1-6.

9. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform the method according to any one of claims 1-6.

10. A computer program product, characterized in that, When executed by a processor, the computer program implements the method described in any one of claims 1-6.

Images