Visual training system and method based on digital twin technology and storage medium

Abstract

The present application relates to the technical field of digital twinning, in particular to a visual practical training system and method based on digital twinning technology and a storage medium, comprising: a visual practical training module builds a parameterized image collector and a visual control interface to support camera parameter setting and visual control program loading respectively; a logic control signal is generated through the visual control interface; when the image collector receives a collection instruction, the image collector collects images and processes the collected images through the visual control interface to update and generate the logic control signal; a practical training scene module builds a virtual application scene through digital twinning technology, updates the position of the virtual application scene according to the action control signal, and correspondingly generates a collection instruction; an action control module is used for encapsulating a controller program and converting the logic control signal into an action control signal; the visual practical training experiment scene can be freely built, and the problems of equipment cost, safety, geographical restriction and equipment selection flexibility are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital twinning, in particular to a visual training system and method based on digital twinning technology and a storage medium. BACKGROUND

[0002] Visual training is a teaching method that trains students' cognition and technology in the visual field in a practical way, often involving computer vision, graphics processing, image recognition and other visual fields. Visual technology has a large number of application scenarios in industrial quality control detection, automation production, object recognition and classification, process monitoring, etc.

[0003] The existing visual training mode is often based on physical equipment to build a training scene. The physical visual training has the following problems:

[0004] (1) Technical equipment cost: The training equipment generally includes industrial equipment and cameras, and the camera is a kind of equipment with a relatively high market price. Different training experiments often require different cameras and lenses. In industrial scene training, the requirements for cameras and lenses are even higher, which seriously limits the popularization of visual training and cannot meet the diverse needs of visual training experiments. For industrial equipment, different experimental scenes require different industrial equipment, such as production lines, large mobile devices, and mechanical arms, which are also expensive and cannot meet the diverse needs of visual training experimental scenes.

[0005] (2) Training safety: In addition to the cost of training equipment, the safety hazards of equipment are also a limitation of visual training. In industrial visual training scenes, such as robot visual training, many students are not familiar with the performance and operation method of the related equipment, and often need professional technical personnel to accompany and guide. Otherwise, safety hazards may occur due to improper operation, which undoubtedly increases the difficulty of visual training and makes it difficult for students to get more training in a short period of time.

[0006] (3) Regional limitations: Physical visual training is easily affected by the storage of equipment. Some equipment can only be stored in experimental sites, and the training site is severely limited, with insufficient regional flexibility and difficulty in cross-regional exchange and learning.

[0007] (4) Insufficient parameter flexibility: The parameters of various equipment used in training, such as the focal length, resolution, interface, and field of view of the camera lens, are fixed from production. In visual training, parameter adjustment is often required, which requires frequent replacement of equipment, resulting in high cost, low experimental efficiency, and greatly limiting the diversity of visual training teaching. SUMMARY

[0008] In response to the above-mentioned defects, the purpose of the present invention is to propose a visual training system, method and storage medium based on digital twin technology, which can freely build visual training experimental scenes, solving the problems of equipment cost, safety, geographical restrictions and flexibility in equipment selection.

[0009] To achieve this object, the present invention adopts the following technical solutions:

[0010] A visual training system based on digital twin technology, including a visual training module, a training scene module and a motion control module;

[0011] The visual training module is used to build a parameterized image collector and a visual control interface, which respectively support camera parameter setting and visual control program loading; a logic control signal is generated through the visual control interface, and when the image collector receives an acquisition instruction, the image collector performs image acquisition and performs image processing on the acquired image through the visual control interface to update and generate the logic control signal;

[0012] The training scenario module is used to build a virtual application scenario using simulation models of various industrial equipment and cameras through digital twin technology; when an action control signal is received, the position of the virtual application scenario is updated according to the action control signal, and the acquisition instruction is generated accordingly;

[0013] The action control module is used for encapsulating the controller program; when receiving the logic control signal, the controller program converts the logic control signal into an action control signal.

[0014] Furthermore, the visual training module includes an image acquisition submodule and an image processing submodule;

[0015] The image acquisition submodule is used to build the parameterized image acquisition device and support camera parameter settings; when receiving an acquisition instruction, the image acquisition device performs image acquisition and generates image information;

[0016] The image processing submodule is used to build the visual control interface, support the loading of the visual control program, and generate the logic control signal; when the image information is received, the image processing submodule performs image processing on the image information through the visual control interface and updates the generated logic control signal;

[0017] The training scenario module includes a model rendering submodule and a model processing submodule;

[0018] The model rendering submodule is used to build the virtual application scene using simulation models of various industrial equipment and cameras through digital twin technology;

[0019] The model processing submodule is used to encapsulate the model motion algorithm; when receiving the action control signal, it interacts with the model rendering submodule data. The action control signal drives the model rendering submodule to update the position of the virtual application scene through the model motion algorithm and generates the acquisition instruction accordingly.

[0020] Furthermore, the model rendering submodule is used to build the virtual application scene through the system's own model library or externally imported models.

[0021] Furthermore, the model motion algorithm is a posture interpolation algorithm.

[0022] Furthermore, the model rendering submodule builds the virtual application scene based on an open source 3D engine;

[0023] The image acquisition submodule uses the view collector of the three-dimensional engine as the image collector.

[0024] Furthermore, the action control module is also used to build an external control interface, and the controller program is changed through the external control interface.

[0025] A visual training method based on digital twin technology includes the following steps:

[0026] S1: Using digital twin technology, we build virtual application scenarios using simulation models of various industrial equipment and cameras.

[0027] S2: Parameter setting through image acquisition device;

[0028] S3: Load the vision control program through the vision control interface and generate logic control signals;

[0029] S4: Convert the logic control signal into an action control signal through the controller program;

[0030] S5: Using digital twin technology, the virtual application scene is updated based on the motion control signal and acquisition instructions are generated accordingly;

[0031] S6: receiving an acquisition instruction through the image collector to acquire an image;

[0032] S7: Perform image processing on the collected image through the visual control program and update the generated logic control signal.

[0033] Furthermore, step S1 includes the following sub-steps:

[0034] S11: Load the system's own model library or external imported model;

[0035] S12: Build virtual application scenarios through the model library or external imported models.

[0036] Furthermore, step S4 includes the following sub-steps:

[0037] S41: Loading controller program;

[0038] S42: Convert the logic control signal into an action control signal through the controller program.

[0039] A computer-readable storage medium, which includes a visual training method program based on digital twin technology. When the visual training method program based on digital twin technology is executed by a processor, the steps of the above-mentioned visual training method based on digital twin technology are implemented.

[0040] The technical solution provided by the present invention can include the following beneficial effects: trainees select the industrial equipment and cameras to be used according to the content of the visual training experiment by using the training scene module, and through digital twin technology, use the simulation models of existing industrial equipment and cameras to simulate the functional properties of related equipment, establish an interactive relationship between the equipment, and thus build the virtual application scene required for the visual training experiment. The visual training experiment scene can be freely built, which solves the problems of equipment cost, safety, geographical restrictions and flexibility in equipment selection.

[0041] Trainees use the visual training module to build a parameterized image acquisition device and visual control interface, which support camera parameter setting and visual control program loading respectively, that is, the trainees conduct visual training experiment entrance. The image acquisition device is equivalent to the camera, and the camera parameters to be set (such as: camera type, camera position, camera shooting direction, field of view angle, focal length, exposure, blur, etc.) can be input and transmitted to the image acquisition device to define the parameters of the camera in the virtual application scene; and the visual control interface is a programming interface for processing and operating images, which is used to implement image processing algorithms, image analysis and image editing and other functions, providing users with a programming interface for developing and applying images during the visual training process.

[0042] During the specific visual training process, after the visual control program is loaded into the visual training module, the visual training module will generate a logic control signal (used to control the motion logic of the relevant equipment in the virtual application scene) according to the visual control program and transmit it to the motion control module; the controller program encapsulated by the motion control module will convert the logic control signal into a motion control signal that can be recognized by the device, so that the relevant equipment in the virtual application scene moves according to the motion control signal, and the virtual application scene is updated. The virtual application scene also supports sensor feedback. When the relevant equipment moves to the preset position and needs to collect image, it will trigger the sensor feedback collection instruction to be transmitted to the visual training module, so that the image collector can collect image. The visual training module will process the collected image according to the visual control program, update and generate a logic control signal, which is used to update the virtual application scene and execute the next step; this cycle will continue until the visual control signal ends. The trainees can intuitively know whether the visual control program of this visual training meets the experimental objectives through the virtual application scene.

[0043] To sum up, the visual training system is a technical method based on digital twin technology, which simulates the process actions of real equipment with virtual equipment, builds a parameterized camera in the virtual scene, simulates the actual image acquisition equipment, collects images in the virtual application scene, and after pre-processing, outputs them to the trainees for secondary image processing. It solves the problems of insufficient and expensive technical equipment and equipment loss in the current training model; and because the training process is completely run in a virtual environment, it avoids accident risks and improves the safety of training; more importantly, compared with physical training, the training system is more flexible. Not only is the training scene not restricted by equipment, it can simulate different visual training scenes, and the training carrier is a computer, so the training location is no longer restricted, which is conducive to distance education and cross-regional training teaching. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a schematic diagram of the principle of a visual training system based on digital twin technology in one embodiment of the present invention.

[0045] Figure 2 This is a partial control signal table of an industrial equipment simulation model in one of the virtual application scenarios.

[0046] Figure 3 This is a schematic diagram of one of the virtual application scenarios Figure 1 .

[0047] Figure 4 Yes Figure 3 Schematic diagram of the virtual application scenario shown Figure 2 .

[0048] Figure 5This is a flowchart of a visual training method based on digital twin technology in one embodiment of the present invention.

[0049] Among them: visual training module 1, training scene module 2, action control module 3, image acquisition submodule 11, image processing submodule 12, model rendering submodule 21, model processing submodule 22. DETAILED DESCRIPTION

[0050] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0051] In the description of the embodiments of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the described features. In the description of the embodiments of the present invention, "plurality" means two or more, unless otherwise specifically specified.

[0052] In the description of the embodiments of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; they may refer to direct connections or indirect connections through an intermediate medium; they may refer to internal communication between two components or an interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the embodiments of the present invention based on specific circumstances.

[0053] The following combination Figures 1 to 2 , describing a visual training system, method and storage medium based on digital twin technology in an embodiment of the present invention.

[0054] Example 1

[0055] A visual training system based on digital twin technology, comprising a visual training module 1, a training scene module 2, and an action control module 3;

[0056] Vision Training Module 1 is used to build a parameterized image collector and a vision control interface, which respectively support camera parameter setting and vision control program loading. Logic control signals are generated through the vision control interface. When the image collector receives an acquisition instruction, it acquires an image and processes the acquired image through the vision control interface to generate updated logic control signals.

[0057] The training scene module 2 is used to build a virtual application scene by using simulation models of various industrial equipment and cameras through digital twin technology; when receiving an action control signal, the virtual application scene is updated in position according to the action control signal, and acquisition instructions are generated accordingly;

[0058] The action control module 3 is used for encapsulation of the controller program; when receiving a logic control signal, the logic control signal is converted into an action control signal through the controller program.

[0059] As shown in the preferred embodiment of the visual training system based on the digital twin technology, Figure 1 According to the visual training experiment content, the training personnel selects the industrial equipment and cameras to be used by using the training scene module 2, simulates the functional properties of the related equipment by using the simulation models of the existing industrial equipment and cameras through the digital twin technology, establishes the interaction relationship between the equipment, and thus builds the virtual application scene required for the visual training experiment, which can freely build the visual training experiment scene, and solves the problems of equipment cost, safety, geographical restriction and equipment selection flexibility.

[0060] The training personnel builds a parameterized image collector and a visual control interface by using the visual training module 1, which supports camera parameter setting and visual control program loading respectively, that is, the training personnel builds a visual training experiment entrance, wherein the image collector is equivalent to the camera, and the camera parameters to be set (such as camera type, camera position, camera shooting direction, field of view, focal length, exposure, and blur) can be input to the image collector and transmitted to the image collector to define the parameters of the camera in the virtual application scene; and the visual control interface is a programming interface for image processing and operation, which is used to realize image processing algorithm, image analysis and image editing functions, and provides a programming interface for users to develop and apply images in the visual training process.

[0061] During the specific visual training process, after the visual control program is loaded into the visual training module 1, the visual training module 1 will generate a logic control signal (used to control the motion logic of the relevant equipment in the virtual application scene) according to the visual control program and transmit it to the motion control module 3; the controller program encapsulated by the motion control module 3 will be used to convert the logic control signal into a motion control signal that can be recognized by the device, so that the relevant equipment in the virtual application scene moves according to the motion control signal, and the virtual application scene is updated. The virtual application scene also supports sensor feedback. When the relevant equipment moves to the preset position and image acquisition is required, the sensor feedback acquisition instruction will be triggered and transmitted to the visual training module 1, so that the image collector performs image acquisition. The visual training module 1 will process the collected image according to the visual control program, update and generate a logic control signal, which is used to update the virtual application scene and execute the next step; this cycle will continue until the visual control signal ends, and the trainees can intuitively know whether the visual control program of this visual training meets the experimental objectives through the virtual application scene.

[0062] To sum up, the visual training system is a technical method based on digital twin technology, which simulates the process actions of real equipment with virtual equipment, builds a parameterized camera in the virtual scene, simulates the actual image acquisition equipment, collects images in the virtual application scene, and after pre-processing, outputs them to the trainees for secondary image processing. It solves the problems of insufficient and expensive technical equipment and equipment loss in the current training model; and because the training process is completely run in a virtual environment, it avoids accident risks and improves the safety of training; more importantly, compared with physical training, the training system is more flexible. Not only is the training scene not restricted by equipment, it can simulate different visual training scenes, and the training carrier is a computer, so the training location is no longer restricted, which is conducive to distance education and cross-regional training teaching.

[0063] It should be noted that the processing process of the action control module 3 is equivalent to the PLC program execution process. The PLC receives the logic control signal, and based on the control logic of the corresponding device, performs a timing processing, condition processing and waiting processing according to the input signal, and then sets the corresponding output signal (action control signal) to form a script program (controller program).

[0064] The specific training process is Figures 2 to 4 For example, this is one of the implementation scenarios of the system, such as Figure 2 As shown in the figure, it is a partial control signal table of one of the industrial equipment simulation models (such as a fan). Each item represents a certain state of a certain axis of the equipment. The checked input column in the table indicates that the signal is an input signal, and the unchecked one indicates that the signal is an output signal. Similarly, after setting the called industrial equipment simulation model and setting the parameters of the image collector, the following can be obtained: Figure 3 and 4 In the virtual application scenario shown, in addition to being able to intuitively see the operation process of the device, the lens image of the virtual camera (i.e., virtual camera) can also be seen from the virtual image display.

[0065] Furthermore, the visual training module 1 includes an image acquisition submodule 11 and an image processing submodule 12;

[0066] The image acquisition submodule 11 is used to build a parameterized image acquisition device and support camera parameter settings; when receiving an acquisition instruction, the image acquisition device performs image acquisition and generates image information;

[0067] The image processing submodule 12 is used to build a visual control interface, support the loading of visual control programs, and generate logical control signals; when image information is received, the image information is processed through the visual control interface to update and generate logical control signals;

[0068] The training scenario module 2 includes a model rendering submodule 21 and a model processing submodule 22;

[0069] The model rendering submodule 21 is used to build virtual application scenarios using simulation models of various industrial equipment and cameras through digital twin technology;

[0070] The model processing submodule 22 is used to encapsulate the model motion algorithm; when receiving the action control signal, it interacts with the model rendering submodule 21 data. The action control signal drives the model rendering submodule 21 through the model motion algorithm to update the position of the virtual application scene and generate acquisition instructions accordingly.

[0071] In this embodiment, to limit the connections between functional areas to limited interface access and thus reduce coupling, the visual training module 1 is split into an image acquisition submodule 11 and an image processing submodule 12 for encapsulation, and the training scene module 2 is split into a model rendering submodule 21 and a model processing submodule 22 for encapsulation. The image acquisition submodule 11 is responsible for setting up the image collector and its data exchange, the image processing submodule 12 is responsible for setting up the visual control interface and its data exchange, the model rendering submodule 21 is responsible for setting up and updating the virtual application scene, and the model processing submodule 22 is responsible for data exchange in the virtual application scene.

[0072] Specifically, the model rendering submodule 21 is responsible for 3D model rendering, supporting 3D graphics visualization and generating and displaying virtual devices to provide an immersive and realistic visual experience. This module supports position parameterization and can render models based on position parameter input, enabling it to support position input updates from the model processing submodule 22 and position arrival feedback, such as motion-triggered position sensors.

[0073] The model processing submodule 22 is responsible for the basic encapsulation of model actions, such as the posture interpolation operation of axis movement, and calculates the time-based position parameters of an axis during the movement process. The specific movement modes include: translation, rotation, and translation and rotation. By receiving signal data as target data, it is converted into corresponding action instructions or parameter instructions, and is responsible for sending instructions to the model rendering submodule 21 to realize the position update of the virtual application scene.

[0074] Image acquisition submodule 11 is a parameterized virtual camera responsible for capturing model image data for a virtual application scene. Upon receiving an image acquisition command, the module acquires the corresponding view data within the scene based on the configured position and orientation parameters. It then performs preprocessing based on predefined parameters, such as blur, exposure, and black and white, to simulate actual camera functionality. The data is then transmitted to image processing submodule 12 for secondary image processing by the trainee using a visual control program.

[0075] The image processing submodule 12 is the interface for secondary image processing of trainees, that is, the entrance to visual training. Secondary image processing is performed by loading the visual control program. For example, when applying visual processing, such as image recognition, quality inspection, image classification, image monitoring, etc., trainees only need to input the corresponding image processing algorithm in the form of a visual control program in this interface to simulate the image processing part of the physical training process. In visual training, it is also necessary to output the corresponding control signal (that is, update the logical control signal) according to the processing result to complete the control of the corresponding equipment in the virtual application scene, and then complete the visual control training.

[0076] Furthermore, the model rendering submodule 21 is used to build a virtual application scene through the system's own model library or externally imported models.

[0077] In this embodiment, there are usually two ways to provide simulation models of various industrial equipment and cameras used in digital twin technology. One is to establish a model library in the system and directly call it when building a virtual application scene; the other is to import models from the outside to build a virtual application scene based on the application scenarios required for visual training.

[0078] Furthermore, the model motion algorithm is a pose interpolation algorithm.

[0079] In this embodiment, the model processing submodule 22 is actually a package of device model actions, which is often aimed at controlling the device motion through vision, such as: first, the motion of the device is decomposed into a basic motion unit with an axis, and the motion of the axis is defined, including: motion mode, motion speed, motion stroke, etc.; second, the axis control signal is defined, such as designing the target signal (start-stop signal, forward-reverse signal, target pose signal, etc.) according to the motion state of an axis (rotation can start and stop, rotate to target pose, forward and reverse, etc.); then, based on the form of signal parameters, the interpolation position of the motion is calculated, that is, the pose of the axis at a certain time, so that it moves to the corresponding pose in the motion frame; finally, the calculated result is sent to the model rendering submodule 21 in the form of an instruction, so that the axis is rendered at the correct pose. Therefore, the model motion algorithm is preferably a pose interpolation algorithm, which has the advantages of simplicity, good real-time performance, high precision, good speed uniformity, and strong adaptability, and is very suitable for the field of motion control.

[0080] It should be noted that the collection instruction triggered by the sensor feedback does not need to be packaged with the corresponding action, but only needs to be packaged into a photographing instruction and sent to the image collection submodule 11.

[0081] Further, the model rendering submodule 21 builds a virtual application scene based on an open-source three-dimensional engine.

[0082] The image collection submodule 11 uses a view collector of a three-dimensional engine as an image collector.

[0083] In this embodiment, because the virtual application scene of the visual practical training experiment is diverse, the model rendering submodule 21 preferably uses an open-source three-dimensional engine (such as JMonkeyEngine) to build a virtual application scene, which is mature in technology, has many open-source data (many users), is convenient for using the method of the three-dimensional engine, quickly builds more virtual application scenes, and facilitates the popularization of the visual practical training system. More importantly, the image collection submodule 11 can use a view collector in the three-dimensional engine (such as JMonkeyEngine) as an image collector, without the need for additional building, and supports receiving the camera parameters set by the user.

[0084] Further, the action control module 3 is also used to build an external control interface, and the controller program is changed through the external control interface.

[0085] In this embodiment, because the motion control module 3 is actually equivalent to a controller (such as PLC, MCU), the controller program encapsulated by the motion control module 3 is not necessarily applicable to all experiments in the face of different visual training experiments. Therefore, in order to make the visual training system more scalable, the motion control module 3 should also be equipped with an external control interface, so that a new controller program can be loaded through the external control interface to replace the original controller program.

[0086] Example 2

[0087] A visual training method based on digital twin technology includes the following steps:

[0088] S1: Using digital twin technology, we build virtual application scenarios using simulation models of various industrial equipment and cameras.

[0089] S2: Parameter setting through image acquisition device;

[0090] S3: Load the vision control program through the vision control interface and generate logic control signals;

[0091] S4: Convert the logic control signal into an action control signal through the controller program;

[0092] S5: Using digital twin technology, the virtual application scene is updated based on the motion control signal and acquisition instructions are generated accordingly;

[0093] S6: receiving an acquisition instruction through the image collector to acquire an image;

[0094] S7: Perform image processing on the collected image through the visual control program and update the generated logic control signal.

[0095] In a preferred embodiment of the visual training method based on digital twin technology proposed by the present invention, Figure 5 As shown, to realize virtual vision training, a virtual application scene must be available first, so step S1 must be executed first; then the trainee executes steps S2 and S3 to load the program and parameters of the visual training experiment into the system; then the system executes steps S4 to S7 according to the loaded visual control program, so that the equipment in the virtual application scene performs relevant movements and image processing according to the visual control program until the visual control signal ends. The trainee can intuitively know whether the visual control program of this visual training meets the experimental objectives through the virtual application scene; thus, the whole process of the virtual vision training experiment is realized, and the visual training experiment scene can be freely built, which solves the problems of equipment cost, safety, geographical restrictions and flexibility in equipment selection.

[0096] Furthermore, step S1 includes the following sub-steps:

[0097] S11: Load the system's own model library or external imported model;

[0098] S12: Build virtual application scenarios through the model library or external imported models.

[0099] In this embodiment, in order to build a virtual application scene using simulation models of various industrial equipment and cameras, these models must first be loaded, which can be achieved by loading the system's own model library or importing external models.

[0100] Furthermore, step S4 includes the following sub-steps:

[0101] S41: Loading controller program;

[0102] S42: Convert the logic control signal into an action control signal through the controller program.

[0103] In this embodiment, in addition to converting logic control signals into action control signals by fixing the controller program in the system, it can also be achieved by externally loading the controller program, making the visual training more extensible.

[0104] Example 3

[0105] A computer-readable storage medium includes a visual training method program based on digital twin technology. When the visual training method program based on digital twin technology is executed by a processor, the steps of the above-mentioned visual training method based on digital twin technology are implemented.

[0106] In this embodiment, a person of ordinary skill in the art may understand that all or part of the steps of implementing the above-mentioned method embodiment may be completed by program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks or optical disks.

[0107] Alternatively, if the integrated system of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in various embodiments of the present invention.

[0108] The visual training system, method, storage medium, and other structures and operations based on digital twin technology according to an embodiment of the present invention are well known to ordinary technicians in the field and will not be described in detail here.

[0109] Throughout this specification, reference to terms such as "embodiment" or "example" indicates that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0110] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A visual training system based on digital twin technology, characterized by: Including visual training module, training scene module and motion control module; The visual training module is used to build a parameterized image collector and a visual control interface, which respectively support camera parameter setting and visual control program loading; a logic control signal is generated through the visual control interface, and when the image collector receives an acquisition instruction, the image collector performs image acquisition and performs image processing on the acquired image through the visual control interface to update and generate the logic control signal; The training scenario module is used to build a virtual application scenario using simulation models of various industrial equipment and cameras through digital twin technology; when an action control signal is received, the position of the virtual application scenario is updated according to the action control signal, and the acquisition instruction is generated accordingly; The action control module is used for encapsulating the controller program; when receiving the logic control signal, the controller program converts the logic control signal into an action control signal.

2. The visual training system based on digital twin technology according to claim 1, characterized in that: The visual training module includes an image acquisition submodule and an image processing submodule; The image acquisition submodule is used to build the parameterized image acquisition device and support camera parameter settings; when receiving an acquisition instruction, the image acquisition device performs image acquisition and generates image information; The image processing submodule is used to build the visual control interface, support the loading of the visual control program, and generate the logical control signal; When the image information is received, performing image processing on the image information through the visual control interface to update and generate the logic control signal; The training scenario module includes a model rendering submodule and a model processing submodule; The model rendering submodule is used to build the virtual application scene using simulation models of various industrial equipment and cameras through digital twin technology; The model processing submodule is used to encapsulate the model motion algorithm; When the action control signal is received, data is interacted with the model rendering submodule. The action control signal drives the model rendering submodule to update the position of the virtual application scene through the model motion algorithm, and generates the acquisition instruction accordingly.

3. The visual training system based on digital twin technology according to claim 2, characterized in that: The model rendering submodule is used to build the virtual application scene through the system's own model library or externally imported models.

4. The visual training system based on digital twin technology according to claim 2, characterized in that: The model motion algorithm is a posture interpolation algorithm.

5. The visual training system based on digital twin technology according to claim 2, characterized in that: The model rendering submodule builds the virtual application scene based on an open source three-dimensional engine; The image acquisition submodule uses the view collector of the three-dimensional engine as the image collector.

6. The visual training system based on digital twin technology according to claim 2, characterized in that: The motion control module is also used to build an external control interface, and to modify the controller program through the external control interface.

7. A visual training method based on digital twin technology, characterized by: The following steps are involved: S1: Using digital twin technology, we build virtual application scenarios using simulation models of various industrial equipment and cameras. S2: Parameter setting through image acquisition device; S3: Load the vision control program through the vision control interface and generate logic control signals; S4: Convert the logic control signal into an action control signal through the controller program; S5: Using digital twin technology, the virtual application scene is updated based on the motion control signal and acquisition instructions are generated accordingly; S6: receiving an acquisition instruction through the image collector to perform image acquisition; S7: Perform image processing on the collected image through the visual control program and update the generated logic control signal.

8. The visual training method based on digital twin technology according to claim 7, characterized in that: The step S1 includes the following sub-steps: S11: Load the system's own model library or external imported model; S12: Build a virtual application scenario through the model library or external imported models.

9. The visual training method based on digital twin technology according to claim 7, characterized in that: The step S4 includes the following sub-steps: S41: Loading controller program; S42: Convert the logic control signal into an action control signal through the controller program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a visual training method program based on digital twin technology. When the visual training method program based on digital twin technology is executed by a processor, the steps of a visual training method based on digital twin technology as described in any one of claims 7 to 9 are implemented.

Images