Visual training system and method based on digital twinning technology, and storage medium

Abstract

The invention relates to the technical field of digital twinning, in particular to a visual training system and method based on the digital twinning technology and a storage medium, and the method comprises the steps: a visual training module builds a parameterized image collector and a visual control interface which respectively support the setting of camera parameters and the loading of a visual control program; a logic control signal is generated through the visual control interface, and when the image collector receives a collection instruction, the image collector carries out image collection and carries out image processing and updating on the collected image through the visual control interface to generate the logic control signal; the training scene module builds a virtual application scene through a digital twinning technology, updates the position of the virtual application scene according to the action control signal, and correspondingly generates an acquisition instruction; the action control module is used for packaging a controller program and converting a logic control signal into an action control signal; a visual practical training experiment scene can be freely built, and the problems of equipment cost, safety, regional limitation and equipment selection flexibility are solved.

Description

Technical Field

[0001] The present invention relates to the field of digital twin technology, and in particular to a visual training system, method and storage medium based on digital twin technology. Background Art

[0002] Visual training is a teaching method that uses practical methods to train students' cognition and technology in the field of vision. It often involves visual fields such as computer vision, graphics processing, and image recognition. Visual technology has a large number of application scenarios in industrial quality control inspection, automated production, object recognition and classification, and process monitoring.

[0003] The existing visual training model is often based on physical equipment to build training scenes. Physical visual training has the following problems:

[0004] (1) Technical equipment costs: Training equipment generally includes industrial equipment and cameras. Cameras are a type of equipment that has always been relatively expensive in the market. Different training experiments often require different cameras and lenses. In industrial scene training, the requirements for cameras and lenses are even higher. This has extremely serious limitations on the popularization of visual training and cannot meet the needs of a wide variety of visual training experiments. For industrial equipment, different experimental scenarios require different industrial equipment. For example, production lines, large mobile equipment, robotic arms, etc. may be required, and the cost is also high, which also cannot meet the needs of a wide variety of visual training experiment scenarios.

[0005] (2) Training safety: In addition to the cost of training equipment, the safety hazard of equipment is also a limitation of visual training. In industrial vision training scenarios, such as robot vision training, many students are not familiar with the performance and operation of related equipment. They often need the guidance of professional technicians. Otherwise, safety hazards may occur due to improper operation, which undoubtedly increases the difficulty of visual training and makes it impossible for students to get more training in a short period of time.

[0006] (3) Geographical restrictions: Physical visual training is easily restricted by equipment storage. Some equipment can only be stored in experimental sites. The training location is severely restricted, the geographical flexibility is insufficient, and cross-regional communication and learning are difficult.

[0007] (4) Insufficient parameter flexibility: The various equipment parameters used in practical training, such as the focal length, resolution, adapter, field of view, etc. of the camera lens, are fixed since production. However, in visual training, parameter adjustments are often required, and the equipment needs to be replaced frequently, which is costly and has low experimental efficiency. It also greatly limits the diversity of visual training teaching. Summary of the invention

[0008] In view of the above-mentioned defects, the purpose of the present invention is to propose a visual training system, method and storage medium based on digital twin technology, which can freely build visual training experiment scenes, solving the problems of equipment cost, safety, geographical restrictions and flexibility in equipment selection.

[0009] To achieve this object, the present invention adopts the following technical solutions:

[0010] A visual training system based on digital twin technology, including a visual training module, a training scene module and a motion control module;

[0011] The visual training module is used to build a parameterized image collector and a visual control interface, which respectively support camera parameter settings and visual control program loading; a logic control signal is generated through the visual control interface, and when the image collector receives a collection instruction, the image collector performs image collection, and performs image processing on the collected image through the visual control interface, and updates and generates the logic control signal;

[0012] The training scenario module is used to build a virtual application scenario using simulation models of various industrial equipment and cameras through digital twin technology; when an action control signal is received, the position of the virtual application scenario is updated according to the action control signal, and the acquisition instruction is generated accordingly;

[0013] The action control module is used for encapsulating the controller program; when receiving the logic control signal, the controller program converts the logic control signal into an action control signal.

[0014] Furthermore, the visual training module includes an image acquisition submodule and an image processing submodule;

[0015] The image acquisition submodule is used to build the parameterized image acquisition device and support the camera parameter setting; when receiving the acquisition instruction, the image acquisition device performs image acquisition and generates image information;

[0016] The image processing submodule is used to build the visual control interface, support the loading of the visual control program, and generate the logic control signal; when the image information is received, the image information is processed through the visual control interface to update and generate the logic control signal;

[0017] The training scenario module includes a model rendering submodule and a model processing submodule;

[0018] The model rendering submodule is used to build the virtual application scene using simulation models of various industrial equipment and cameras through digital twin technology;

[0019] The model processing submodule is used for encapsulating the model motion algorithm; when receiving the action control signal, it interacts with the model rendering submodule data. The action control signal drives the model rendering submodule to update the position of the virtual application scene through the model motion algorithm, and generates the acquisition instruction accordingly.

[0020] Furthermore, the model rendering submodule is used to build the virtual application scene through the system's own model library or an external imported model.

[0021] Furthermore, the model motion algorithm is a posture interpolation algorithm.

[0022] Furthermore, the model rendering submodule builds the virtual application scene based on an open source three-dimensional engine;

[0023] The image acquisition submodule uses the view collector of the three-dimensional engine as the image collector.

[0024] Furthermore, the action control module is also used to build an external control interface, and the controller program is changed through the external control interface.

[0025] A visual training method based on digital twin technology includes the following steps:

[0026] S1: Use digital twin technology to build virtual application scenarios using simulation models of various industrial equipment and cameras;

[0027] S2: Parameter setting through image acquisition device;

[0028] S3: Load the visual control program through the visual control interface to generate a logic control signal;

[0029] S4: Convert the logic control signal into an action control signal through the controller program;

[0030] S5: Through digital twin technology, the position of the virtual application scene is updated according to the action control signal, and the acquisition instructions are generated accordingly;

[0031] S6: receiving an acquisition instruction through the image acquisition device to acquire images;

[0032] S7: Perform image processing on the collected image through the visual control program and update the generated logic control signal.

[0033] Furthermore, the step S1 includes the following sub-steps:

[0034] S11: Load the system's own model library or external imported model;

[0035] S12: Build virtual application scenarios through the model library or external imported models.

[0036] Furthermore, the step S4 includes the following sub-steps:

[0037] S41: Loading controller program;

[0038] S42: Convert the logic control signal into an action control signal through the controller program.

[0039] A computer-readable storage medium, wherein the computer-readable storage medium includes a visual training method program based on digital twin technology. When the visual training method program based on digital twin technology is executed by a processor, the steps of the above-mentioned visual training method based on digital twin technology are implemented.

[0040] The technical solution provided by the present invention may include the following beneficial effects: trainees select the industrial equipment and cameras to be used according to the content of the visual training experiment by using the training scene module, and through the digital twin technology, use the simulation models of the existing industrial equipment and cameras to simulate the functional attributes of the relevant equipment, establish the interactive relationship between the equipment, and thus build the virtual application scene required for the visual training experiment. The visual training experiment scene can be freely built, which solves the problems of equipment cost, safety, geographical restrictions and flexibility in equipment selection.

[0041] Trainees use the visual training module to build parameterized image acquisition devices and visual control interfaces, which support camera parameter settings and visual control program loading respectively. This is the entrance for trainees to conduct visual training experiments. The image acquisition device is equivalent to the camera, and the camera parameters to be set (such as camera type, camera position, camera shooting direction, field of view, focal length, exposure, blur, etc.) can be input and transmitted to the image acquisition device to define the parameters of the camera in the virtual application scene. The visual control interface is a programming interface for processing and operating images. It is used to implement image processing algorithms, image analysis, image editing and other functions, providing users with a programming interface for developing and applying images during visual training.

[0042] During the specific visual training process, after the visual control program is loaded into the visual training module, the visual training module will generate a logic control signal (used to control the motion logic of related equipment in the virtual application scene) according to the visual control program and transmit it to the motion control module; the controller program encapsulated in the motion control module will be used to convert the logic control signal into a motion control signal that can be recognized by the device, so that the related equipment in the virtual application scene moves according to the motion control signal, thereby realizing the update of the virtual application scene. The virtual application scene also supports sensor feedback. When the related equipment moves to the preset position and needs to collect image, it will trigger the sensor feedback collection instruction to be transmitted to the visual training module, so that the image collector can collect image. The visual training module will perform image processing on the collected image according to the visual control program, update and generate a logic control signal, which is used to update the virtual application scene and execute the next step; this cycle will continue until the visual control signal ends, and the trainees can intuitively know whether the visual control program of this visual training meets the experimental objectives through the virtual application scene.

[0043] To sum up, the visual training system is a technical method based on digital twin technology, which simulates the process actions of real equipment with virtual equipment, builds a parameterized camera in the virtual scene, simulates the actual image acquisition equipment, collects images in the virtual application scene and outputs them to the trainees for secondary image processing after preprocessing. It solves the problems of insufficient and expensive technical equipment and equipment loss in the current training model; and because the training process is completely run in a virtual environment, it avoids accident risks and improves the safety of training; more importantly, compared with physical training, the training system is more flexible. Not only is the training scene not restricted by equipment, it can simulate different visual training scenes, and the training carrier is a computer, so the training location is no longer restricted, which is conducive to distance education and cross-regional training teaching. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of the principle of a visual training system based on digital twin technology in one of the embodiments of the present invention.

[0045] Figure 2 It is a partial control signal table of an industrial equipment simulation model in one of the virtual application scenarios.

[0046] Figure 3 This is a schematic diagram of one of the virtual application scenarios. Figure 1 .

[0047] Figure 4 Yes Figure 3 Schematic diagram of the virtual application scenario shown Figure 2 .

[0048] Figure 5It is a flowchart of a visual training method based on digital twin technology in one of the embodiments of the present invention.

[0049] Among them: visual training module 1, training scene module 2, action control module 3, image acquisition submodule 11, image processing submodule 12, model rendering submodule 21, model processing submodule 22. DETAILED DESCRIPTION

[0050] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0051] In the description of the embodiments of the present invention, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0052] In the description of the embodiments of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be directly connected or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present invention can be understood according to specific circumstances.

[0053] Combine the following Figure 1 to Figure 2 , describing a visual training system, method and storage medium based on digital twin technology in an embodiment of the present invention.

[0054] Example 1

[0055] A visual training system based on digital twin technology, comprising a visual training module 1, a training scene module 2 and an action control module 3;

[0056] Visual training module 1 is used to build a parameterized image collector and visual control interface, which respectively support camera parameter setting and visual control program loading; generate logic control signals through the visual control interface, and when the image collector receives the acquisition instruction, the image collector performs image acquisition, and processes the acquired image through the visual control interface to update the generated logic control signal;

[0057] Training scenario module 2 is used to build virtual application scenarios using simulation models of various industrial equipment and cameras through digital twin technology; when an action control signal is received, the position of the virtual application scenario is updated according to the action control signal, and acquisition instructions are generated accordingly;

[0058] The action control module 3 is used for encapsulating the controller program; when receiving a logic control signal, the controller program converts the logic control signal into an action control signal.

[0059] The present invention proposes a preferred embodiment of a visual training system based on digital twin technology, such as Figure 1 As shown, the trainees select the industrial equipment and cameras to be used according to the content of the visual training experiment using the training scenario module 2. Through the digital twin technology, the functional attributes of the relevant equipment are simulated using the simulation models of the existing industrial equipment and cameras, and the interactive relationship between the equipment is established, thereby building the virtual application scenario required for the visual training experiment. The visual training experiment scenario can be freely built, which solves the problems of equipment cost, safety, geographical restrictions and flexibility in equipment selection.

[0060] The trainees use the visual training module 1 to build a parameterized image acquisition device and a visual control interface, which support camera parameter settings and visual control program loading respectively, i.e., the trainees conduct visual training experiment entrance, wherein the image acquisition device is equivalent to the camera, and the camera parameters to be set (such as: camera type, camera position, camera shooting direction, field of view, focal length, exposure, blur, etc.) can be input and transmitted to the image acquisition device to define the parameters of the camera in the virtual application scene; and the visual control interface is a programming interface for processing and operating images, which is used to implement image processing algorithms, image analysis, image editing and other functions, and provides users with a programming interface for developing and applying images during visual training.

[0061] During the specific visual training process, after the visual control program is loaded into the visual training module 1, the visual training module 1 will generate a logic control signal (used to control the motion logic of the relevant equipment in the virtual application scene) according to the visual control program and transmit it to the action control module 3; the controller program encapsulated in the action control module 3 is used to convert the logic control signal into an action control signal that can be recognized by the device, so that the relevant equipment in the virtual application scene moves according to the action control signal, thereby realizing the update of the virtual application scene, and the virtual application scene also supports sensor feedback. When the relevant equipment moves to the preset position and needs to collect image, it will trigger the sensor feedback collection instruction to be transmitted to the visual training module 1, so that the image collector performs image collection, and the visual training module 1 will perform image processing on the collected image according to the visual control program, update and generate a logic control signal, which is used to update the virtual application scene and execute the next step; this cycle will continue until the visual control signal ends, and the trainees can intuitively know whether the visual control program of this visual training meets the experimental objectives through the virtual application scene.

[0062] To sum up, the visual training system is a technical method based on digital twin technology, which simulates the process actions of real equipment with virtual equipment, builds a parameterized camera in the virtual scene, simulates the actual image acquisition equipment, collects images in the virtual application scene and outputs them to the trainees for secondary image processing after preprocessing. It solves the problems of insufficient and expensive technical equipment and equipment loss in the current training model; and because the training process is completely run in a virtual environment, it avoids accident risks and improves the safety of training; more importantly, compared with physical training, the training system is more flexible. Not only is the training scene not restricted by equipment, it can simulate different visual training scenes, and the training carrier is a computer, so the training location is no longer restricted, which is conducive to distance education and cross-regional training teaching.

[0063] It should be noted that the processing process of the action control module 3 is equivalent to the PLC program execution process. The PLC receives the logic control signal, and based on the control logic of the corresponding device, performs a timing processing, condition processing, and waiting processing according to the input signal, and then sets the corresponding output signal (action control signal) to form a script program (controller program).

[0064] The specific training process is Figures 2 to 4 For example, this is one of the implementation scenarios of the system, such as Figure 2 As shown in the figure, it is a partial control signal table of one of the industrial equipment simulation models (such as a fan). Each item represents a certain state of a certain axis of the equipment. The checked signal in the input column of the table indicates that the signal is an input signal, and the unchecked signal indicates that the signal is an output signal. Similarly, after setting the called industrial equipment simulation model and setting the parameters of the image collector, the following can be obtained. Figure 3 and 4 In the virtual application scenario shown, in addition to being able to intuitively see the operation process of the device, the lens image of the virtual camera (i.e., virtual camera) can also be seen from the virtual image display.

[0065] Furthermore, the visual training module 1 includes an image acquisition submodule 11 and an image processing submodule 12;

[0066] The image acquisition submodule 11 is used to build a parameterized image acquisition device and support camera parameter settings; when receiving an acquisition instruction, the image acquisition device performs image acquisition and generates image information;

[0067] The image processing submodule 12 is used to build a visual control interface, support the loading of a visual control program, and generate a logic control signal; when image information is received, the image information is processed through the visual control interface to update and generate a logic control signal;

[0068] The training scene module 2 includes a model rendering submodule 21 and a model processing submodule 22;

[0069] The model rendering submodule 21 is used to build virtual application scenes using simulation models of various industrial equipment and cameras through digital twin technology;

[0070] The model processing submodule 22 is used for encapsulating the model motion algorithm; when receiving the action control signal, it interacts with the model rendering submodule 21 data. The action control signal drives the model rendering submodule 21 through the model motion algorithm to update the position of the virtual application scene and generate acquisition instructions accordingly.

[0071] In this embodiment, in order to limit the connection between the functional areas to limited interface mutual access and thus reduce coupling, the visual training module 1 is split into an image acquisition submodule 11 and an image processing submodule 12 for encapsulation, and the training scene module 2 is split into a model rendering submodule 21 and a model processing submodule 22 for encapsulation. The image acquisition submodule 11 is responsible for the construction of the image collector and its data interaction, the image processing submodule 12 is responsible for the construction of the visual control interface and its data interaction, the model rendering submodule 21 is responsible for the construction and update of the virtual application scene, and the model processing submodule 22 is responsible for the data interaction of the virtual application scene.

[0072] Specifically, the model rendering submodule 21 is a module for three-dimensional model rendering, supports the visualization of three-dimensional graphics, and is responsible for generating and displaying virtual devices to provide an immersive and realistic visual experience. This module supports position parameterization and can render models based on the input of position parameters, so that it supports the position input update of the model processing submodule 22 and supports position arrival feedback, such as motion triggering position sensors.

[0073] The model processing submodule 22 is responsible for the basic encapsulation of model actions, such as the interpolation operation of the axis movement, and the calculation of the time-based position parameters of an axis during the movement process. The specific movement modes include: translation, rotation, and translation and rotation. By receiving signal data as target data, it is converted into corresponding action instructions or parameter instructions, and is responsible for sending instructions to the model rendering submodule 21 to realize the position update of the virtual application scene.

[0074] The image acquisition submodule 11 is a parameterized virtual camera responsible for collecting model image data of the virtual application scene. When the module receives the image acquisition instruction, it obtains the corresponding view data in the scene based on the set position and direction parameters, and performs preprocessing based on some set parameters, such as blur, exposure, black and white, to simulate the actual camera function, and finally transmits it to the image processing submodule 12 for the trainees to perform secondary image processing through the visual control program.

[0075] The image processing submodule 12 is the interface for secondary image processing for trainees, that is, the entrance to visual training. Secondary image processing is performed by loading the visual control program. For example, when applying visual processing, such as image recognition, quality inspection, image classification, image monitoring, etc., trainees only need to input the corresponding image processing algorithm in the form of a visual control program in this interface to simulate the image processing part of the physical training process. In visual training, it is also necessary to output the corresponding control signal (that is, update the logical control signal) according to the processing result to complete the control of the corresponding equipment in the virtual application scene, and then complete the visual control training.

[0076] Furthermore, the model rendering submodule 21 is used to build a virtual application scene through the system's own model library or an externally imported model.

[0077] In this embodiment, simulation models of various industrial equipment and cameras used in digital twin technology are usually provided in two ways. One is to establish a model library in the system and directly call it when building a virtual application scene; the other is to import models from the outside to build a virtual application scene based on the application scenarios required for visual training.

[0078] Furthermore, the model motion algorithm is a pose interpolation algorithm.

[0079] In this embodiment, the model processing submodule 22 is actually an encapsulation of the device model action, which is often aimed at controlling the movement of the device through vision, such as: first, the action of the device should be decomposed into a basic action unit with an axis, and the movement of the axis should be defined, including: movement mode, movement speed, movement stroke, etc.; secondly, the axis control signal is defined, such as designing the target signal (start-stop signal, forward-reverse signal, target posture signal, etc.) according to the movement state of an axis (rotation can be started and stopped, rotated to the target posture, forward and reverse, etc.); then, based on the form of signal parameters, the interpolation position of the movement is calculated, that is, the posture of the axis at a certain moment, so that it moves to the corresponding posture in the motion frame; finally, the calculation result is sent to the model rendering submodule 21 in the form of an instruction, so that the axis is rendered in the correct posture. Therefore, the model motion algorithm is preferably a posture interpolation algorithm, which has the advantages of being simple and easy to operate, good real-time performance, high precision, good speed uniformity and strong adaptability, and is very suitable for the field of motion control.

[0080] It should be noted that the acquisition instruction triggered by the sensor feedback does not need to be encapsulated into a corresponding action, but only needs to be encapsulated into a photo taking instruction and sent to the image acquisition submodule 11.

[0081] Furthermore, the model rendering submodule 21 builds a virtual application scene based on an open source 3D engine;

[0082] The image acquisition submodule 11 uses the view collector of the three-dimensional engine as the image collector.

[0083] In this embodiment, because the visual training experiment is oriented to many virtual application scenarios, the model rendering submodule 21 preferably uses an open source three-dimensional engine (such as JMonkeyEngine) to build a virtual application scenario. The technology is mature, and there are many open source materials (many users), which is convenient for learning from the use of the three-dimensional engine, quickly building more virtual application scenarios, and facilitating the popularization of the visual training system. More importantly, the image acquisition submodule 11 can use the view collector in the three-dimensional engine (such as JMonkeyEngine) as an image collector, without the need for additional construction, and supports receiving camera parameters set by the user.

[0084] Furthermore, the action control module 3 is also used to build an external control interface and change the controller program through the external control interface.

[0085] In this embodiment, because the motion control module 3 is actually equivalent to a controller (such as PLC, MCU), the controller program encapsulated by the motion control module 3 is not necessarily suitable for all experiments in the face of different visual training experiments. Therefore, in order to make the visual training system more expansible, the motion control module 3 should also be built with an external control interface, so that a new controller program can be loaded through the external control interface to replace the original controller program.

[0086] Example 2

[0087] A visual training method based on digital twin technology includes the following steps:

[0088] S1: Use digital twin technology to build virtual application scenarios using simulation models of various industrial equipment and cameras;

[0089] S2: Parameter setting through image acquisition device;

[0090] S3: Load the visual control program through the visual control interface to generate a logic control signal;

[0091] S4: Convert the logic control signal into an action control signal through the controller program;

[0092] S5: Through digital twin technology, the position of the virtual application scene is updated according to the action control signal, and the acquisition instructions are generated accordingly;

[0093] S6: receiving an acquisition instruction through the image acquisition device to acquire images;

[0094] S7: Perform image processing on the collected image through the visual control program and update the generated logic control signal.

[0095] In a preferred embodiment of the visual training method based on digital twin technology proposed by the present invention, Figure 5 As shown, to realize virtual vision training, a virtual application scene must be prepared first, so step S1 must be executed first; then the trainee executes steps S2 and S3 to load the program and parameters of the visual training experiment into the system; then the system executes steps S4 to S7 according to the loaded visual control program, so that the equipment in the virtual application scene performs relevant movements and image processing according to the visual control program until the visual control signal ends. The trainee can intuitively know whether the visual control program of this visual training meets the experimental objectives through the virtual application scene; thereby realizing the whole process of the virtual vision training experiment, the visual training experiment scene can be freely built, and the problems of equipment cost, safety, geographical restrictions and flexibility in equipment selection are solved.

[0096] Furthermore, step S1 includes the following sub-steps:

[0097] S11: Load the system's own model library or external imported model;

[0098] S12: Build virtual application scenarios through the model library or external imported models.

[0099] In this embodiment, in order to use simulation models of various industrial equipment and cameras to build virtual application scenarios, these models must be loaded first, which can be achieved by loading the system's own model library or externally importing models.

[0100] Further, step S4 includes the following sub-steps:

[0101] S41: Loading controller program;

[0102] S42: Convert the logic control signal into an action control signal through the controller program.

[0103] In this embodiment, in addition to fixing the controller program in the system to realize the conversion of logic control signals into action control signals, it can also be realized by externally loading the controller program, so that the expansibility of visual training is better.

[0104] Example 3

[0105] A computer-readable storage medium, the computer-readable storage medium includes a visual training method program based on digital twin technology. When the visual training method program based on digital twin technology is executed by a processor, the steps of the above-mentioned visual training method based on digital twin technology are implemented.

[0106] In this embodiment, a person of ordinary skill in the art may understand that all or part of the steps of implementing the above method embodiment may be completed by program instructions, and the aforementioned program may be stored in a computer-readable storage medium, which, when executed, executes the steps of the above method embodiment; and the aforementioned storage medium includes various media that can store program codes, such as a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0107] Alternatively, if the above-mentioned integrated system of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention.

[0108] The other structures and operations of a visual training system, method and storage medium based on digital twin technology according to an embodiment of the present invention are well known to ordinary technicians in the field and will not be described in detail here.

[0109] In the description of this specification, the description with reference to the terms "embodiment", "example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0110] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A visual training system based on digital twin technology, characterized by: Including visual training module, training scene module and motion control module; The visual training module is used to build a parameterized image collector and a visual control interface, which respectively support camera parameter settings and visual control program loading; a logic control signal is generated through the visual control interface, and when the image collector receives a collection instruction, the image collector performs image collection, and performs image processing on the collected image through the visual control interface, and updates and generates the logic control signal; The training scenario module is used to build a virtual application scenario using simulation models of various industrial equipment and cameras through digital twin technology; when an action control signal is received, the position of the virtual application scenario is updated according to the action control signal, and the acquisition instruction is generated accordingly; The action control module is used for encapsulating the controller program; when receiving the logic control signal, the controller program converts the logic control signal into an action control signal.

2. According to claim 1, a visual training system based on digital twin technology is characterized in that: The visual training module includes an image acquisition submodule and an image processing submodule; The image acquisition submodule is used to build the parameterized image acquisition device and support the camera parameter setting; when receiving the acquisition instruction, the image acquisition device performs image acquisition and generates image information; The image processing submodule is used to build the visual control interface, support the loading of the visual control program, and generate the logic control signal; When the image information is received, the image information is processed through the visual control interface to update and generate the logic control signal; The training scenario module includes a model rendering submodule and a model processing submodule; The model rendering submodule is used to build the virtual application scene using simulation models of various industrial equipment and cameras through digital twin technology; The model processing submodule is used for encapsulating the model motion algorithm; When the action control signal is received, data is interacted with the model rendering submodule. The action control signal drives the model rendering submodule to update the position of the virtual application scene through the model motion algorithm, and generates the acquisition instruction accordingly.

3. According to claim 2, a visual training system based on digital twin technology is characterized in that: The model rendering submodule is used to build the virtual application scene through the system's own model library or externally imported models.

4. According to claim 2, a visual training system based on digital twin technology is characterized in that: The model motion algorithm is a posture interpolation algorithm.

5. According to claim 2, a visual training system based on digital twin technology is characterized in that: The model rendering submodule builds the virtual application scene based on an open source three-dimensional engine; The image acquisition submodule uses the view collector of the three-dimensional engine as the image collector.

6. According to claim 2, a visual training system based on digital twin technology is characterized in that: The action control module is also used to build an external control interface, and the controller program is changed through the external control interface.

7. A visual training method based on digital twin technology, characterized by: The following steps are involved: S1: Use digital twin technology to build virtual application scenarios using simulation models of various industrial equipment and cameras; S2: Parameter setting through image acquisition device; S3: Load the visual control program through the visual control interface to generate a logic control signal; S4: Convert the logic control signal into an action control signal through the controller program; S5: Through digital twin technology, the position of the virtual application scene is updated according to the action control signal, and the acquisition instructions are generated accordingly; S6: receiving an acquisition instruction through the image acquisition device to acquire images; S7: Perform image processing on the collected image through the visual control program and update the generated logic control signal.

8. A visual training method based on digital twin technology according to claim 7, characterized in that: The step S1 comprises the following sub-steps: S11: Load the system's own model library or external imported model; S12: Build virtual application scenarios through the model library or external imported models.

9. The visual training method based on digital twin technology according to claim 7 is characterized in that: The step S4 comprises the following sub-steps: S41: Loading controller program; S42: Convert the logic control signal into an action control signal through the controller program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a visual training method program based on digital twin technology. When the visual training method program based on digital twin technology is executed by a processor, the steps of a visual training method based on digital twin technology as described in any one of claims 7 to 9 are implemented.

Images