Simulation method, system and device for scientific experiments based on generative adversarial networks

Abstract

The present invention is oriented to STEM education, and discloses a simulation method, system and device for scientific experiments based on generative adversarial networks. The method includes: obtaining the original image of the experimental observation object through the picture library module; obtaining the target operation instruction input by the user through the operation item module; generating the scientific experiment theoretical result corresponding to the target operation instruction through the knowledge base module; inputting the original image and the scientific experiment theoretical result into the target network model through the image generation module to obtain the simulated image of the experimental observation object; and displaying the simulated image and the original image through the experimental simulation module. Through the technical solution of the present invention, scientific experiments can be free from the restrictions of time, place and experimental equipment, so that every student can participate in scientific experiments and experience the fun of learning. In addition, users can adjust the experimental plan according to the simulated image generated by the artificial intelligence GAN model with real-time feedback, thereby improving the effect of experimental education.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of artificial intelligence technology, and in particular, to a simulation method, system, and device for scientific experiments based on a generative adversarial network. Background Art

[0002] In teaching activities, scientific simulation experiments are often required. However, due to the high cost of experimental equipment and required materials, scientific experiments cannot be carried out on a large scale. Often, teachers can only demonstrate the experiments or watch experimental videos. Students cannot personally participate in the experimental exploration process, resulting in a low sense of participation. In addition, some scientific experiments require a period of several days or even months. It is impossible to predict the feedback results corresponding to the experimental operation in a short period of time, and it is impossible to adjust the experimental plan in time according to the feedback results, resulting in a high experimental failure rate, high time cost, and unsatisfactory experimental results.

[0003] Therefore, due to the limitations of hardware facilities and experimental time, scientific experiments are difficult to popularize among primary and secondary school students. Students have few opportunities to participate in experiments, and it is difficult for them to understand abstract theoretical knowledge without the assistance of experiments. As a result, they cannot experience the joy of learning science, increase their scientific inquiry capabilities, and form scientific knowledge from experimental experience. Summary of the invention

[0004] The embodiment of the present invention provides a simulation method, system and device for scientific experiments based on generative adversarial networks, so as to realize the generation of simulation images corresponding to the experiment based on the original images of the experimental observation objects and the target operation instructions uploaded by the user, so that the scientific experiment is not limited by time, place and the instruments and supplies required for the experiment, so that every student can have the opportunity to participate in the scientific experiment and experience the fun of learning science from the experiment. In addition, since the simulation image is fed back in real time, the user can adjust the experimental plan according to the real-time feedback simulation image, thereby improving the effect of experimental education.

[0005] In a first aspect, an embodiment of the present invention provides a simulation method of a scientific experiment based on a generative adversarial network, which is applied to a simulation system of a scientific experiment. The simulation system of the scientific experiment includes a picture library module, an operation item module, a knowledge base module, an image generation module, and an experiment simulation module. The method includes:

[0006] Obtain the original image of the experimental observation object through the image library module;

[0007] Acquire a target operation instruction input by a user through an operation item module, wherein the target operation instruction is an experimental condition of a scientific experiment corresponding to the experimental observation object;

[0008] Generate a scientific experiment theoretical result corresponding to the target operation instruction through a knowledge base module; the scientific experiment theoretical result is a theoretical experiment result corresponding to the experimental observation object after executing the target operation instruction;

[0009] The original image and the scientific experimental theoretical result are input into the target network model through the image generation module to obtain a simulated image of the experimental observation object, wherein the simulated image is an image obtained by adjusting the original image by the target network model based on the scientific experimental theoretical result;

[0010] The simulated image and the original image are displayed through an experimental simulation module.

[0011] Furthermore, obtaining the original image of the experimental observation object through the image library module includes:

[0012] The original image of the experimental observation object is obtained from a database, or the original image of the experimental observation object input by a user is obtained.

[0013] Furthermore, obtaining the target operation instruction input by the user through the operation item module includes:

[0014] Displaying alternative operation items of the scientific experiment corresponding to the experimental observation object through the operation item module, so that the user can select a target operation item from the alternative operation items;

[0015] Acquire, through the operation item module, a target operation instruction of at least one target operation item input by at least one user;

[0016] The target operation instruction includes at least one of: a text, a numerical value, a selection instruction of a list option, a drag instruction of a progress bar, a touch instruction, and a trigger instruction of a physical button.

[0017] Furthermore, the generating of the scientific experimental theoretical results corresponding to the target operation instructions by the knowledge base module includes:

[0018] Determine the scientific experimental theoretical result corresponding to the target operation instruction through the scientific experimental principle stored in the knowledge base module;

[0019] The scientific experiment principle is pre-stored in the system or input by the user.

[0020] Furthermore, the target network model is a fully trained generative adversarial network model.

[0021] Furthermore, after displaying the simulated image and the original image through the experimental simulation module, the method further includes:

[0022] Save scientific experiment data through the experiment simulation module, and send the scientific experiment data to the server for storage;

[0023] The scientific experiment data includes: simulation process information of the scientific experiment and notes written by the user.

[0024] Furthermore, it also includes:

[0025] Provide users with auxiliary tools and auxiliary information required for scientific experiment simulation through the experiment simulation module;

[0026] The auxiliary information includes: experimental problem description information, experimental theoretical knowledge information and experimental instructions information.

[0027] In a second aspect, an embodiment of the present invention further provides a scientific experiment simulation system, the system comprising:

[0028] Image library module, used to obtain the original images of the experimental observation objects;

[0029] The operation item module is used to obtain the target operation instruction input by the user, and the target operation instruction is the experimental condition of the scientific experiment corresponding to the experimental observation object.

[0030] The knowledge base module is used to generate the scientific experiment theoretical result corresponding to the target operation instruction; the scientific experiment theoretical result is the theoretical result of the scientific experiment corresponding to the experimental observation object after executing the target operation instruction.

[0031] An image generation module is used to input the original image and the theoretical results of the scientific experiment into a target network model to obtain a simulated image of the experimental observation object, wherein the simulated image is an image obtained by adjusting the original image by the target network model based on the theoretical results of the scientific experiment.

[0032] The experimental simulation module is used to display the simulated image and the original image.

[0033] Furthermore, the picture library module is specifically used for:

[0034] The original image of the experimental observation object is obtained from a database, or the original image of the experimental observation object uploaded by a user is obtained.

[0035] Furthermore, the operation item module is specifically used to:

[0036] Displaying alternative operation items of the scientific experiment corresponding to the experimental observation object through the operation item module, so that the user can select a target operation item from the alternative operation items;

[0037] Acquire, through the operation item module, a target operation instruction of at least one target operation item input by at least one user;

[0038] The target operation instruction includes at least one of: a text, a numerical value, a selection instruction of a list option, a drag instruction of a progress bar, a touch instruction, and a trigger instruction of a physical button.

[0039] Furthermore, the knowledge base module is specifically used for:

[0040] Determine the scientific experimental theoretical result corresponding to the target operation instruction through the scientific experimental principle stored in the knowledge base module;

[0041] The scientific experiment principle is pre-stored in the system or input by the user.

[0042] Optionally, the target network model is a fully trained generative adversarial network model.

[0043] Furthermore, the experimental simulation module is specifically used for:

[0044] Save scientific experiment data and send the scientific experiment data to a server for storage;

[0045] The scientific experiment data includes: simulation process information of the scientific experiment and notes written by the user.

[0046] Furthermore, the experimental simulation module is also used for:

[0047] Provide users with auxiliary tools and auxiliary information required for scientific experiment simulation through the experiment simulation module;

[0048] The auxiliary information includes: experimental problem description information, experimental theoretical knowledge information and experimental instructions information.

[0049] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a simulation method for a scientific experiment based on a generative adversarial network as described in any one of the embodiments of the present invention is implemented.

[0050] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a simulation method for a scientific experiment based on a generative adversarial network as described in any one of the embodiments of the present invention.

[0051] The embodiment of the present invention is oriented to STEM education, and provides a simulation method, system and device for scientific experiments based on generative adversarial networks. The original image of the experimental observation object and the scientific experiment theoretical results corresponding to the target operation instruction input by the user are input into the target network model through the image generation module to obtain a simulated image of the experimental observation object, and the simulated image and the original image are displayed through the experimental simulation module, so as to solve the problem that the actual scientific experiment is limited by hardware facilities and experimental time, and it is difficult to popularize among primary and secondary school students. Students have few opportunities to participate in experiments and cannot experience the fun of learning science from experiments. According to the original image of the experimental observation object uploaded by the user or the picture library module, the original image is extracted, and the target operation instruction generates the simulated image corresponding to the experimental observation object, so that the scientific experiment is not limited by time, place and the instruments and supplies required for the experiment, so that every student can have the opportunity to participate in the scientific experiment and experience the fun of learning science from the experiment. In addition, since the simulated image is fed back in real time, the user can adjust the experimental plan according to the simulated image fed back in real time, thereby improving the success rate of the experiment. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 It is a flow chart of a simulation method of a scientific experiment based on a generative adversarial network in Embodiment 1 of the present invention;

[0054] Figure 2 This is a flowchart of another simulation method of a scientific experiment based on a generative adversarial network in Embodiment 1 of the present invention.

[0055] Figure 3 It is a flow chart of a simulation method of a scientific experiment based on a generative adversarial network in Embodiment 2 of the present invention;

[0056] Figure 4 This is a schematic diagram of an interface of an experimental simulation module in the second embodiment of the present invention;

[0057] Figure 5 It is a structural schematic diagram of a simulation system for scientific experiments in Embodiment 3 of the present invention;

[0058] Figure 6 It is a structural diagram of a computer device in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0059] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0060] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0061] Embodiment 1

[0062] Figure 1 A flowchart of a simulation method of a scientific experiment based on a generative adversarial network provided in the first embodiment of the present invention, this embodiment can be applied to STEM education, and the situation of simulating scientific experiments through a simulation system, the method can be executed by the simulation system of the scientific experiment in the embodiment of the present invention, and the system can be implemented in software and / or hardware. Conventional scientific experiments refer to a form of social practice that observes and studies natural phenomena and their regularities under artificial control conditions using certain instruments, equipment and other material means according to a certain purpose. The simulation of scientific experiments in the embodiment of the present invention is based on the operation process of simulating actual scientific experiments by a scientific experiment simulation system, and the simulation system of the scientific experiment includes a picture library module, an operation item module, a knowledge base module, an image generation module and an experiment simulation module.

[0063] like Figure 1 As shown, the method specifically comprises the following steps:

[0064] S110, obtaining an original image of the experimental observation object through a picture library module.

[0065] The scientific experiments in the embodiments of the present invention may include experiments in any subject or field, for example, scientific experiments in STEM education courses. STEM is the general term for four subjects: science, technology, engineering, and mathematics. One of the focuses of STEM education courses is to enhance scientific literacy, that is, to use scientific knowledge (such as physics, chemistry, biological sciences, and earth and space sciences) to understand nature and participate in processes that affect nature.

[0066] The experimental observation object can be understood as the observation object in the process of scientific experiments. For example, the experimental observation object in the convex lens imaging experiment can be a burning candle flame. The original image of the experimental observation object can be understood as the image presented by the experimental observation object before the experiment begins. For example, the original image of the experimental observation object in the convex lens imaging experiment is an image of an upright candle flame. The image library module is used to provide users with original image materials of the experimental observation object of scientific experiments.

[0067] Optionally, the original image of the experimental observation object is obtained through the image library module, including:

[0068] The original image of the experimental observation object is obtained from a database, or the original image of the experimental observation object uploaded by a user is obtained.

[0069] In the database corresponding to the picture library module of the scientific experiment system, some original image materials of common scientific experiments can be preset, such as the candle flame image often used in convex lens imaging experiments, the copper oxide image required in metal oxide reduction experiments, or the plant cell slices required in the experiment of observing plant cells under a microscope.

[0070] Users can also upload original images of the experimental observation objects corresponding to scientific experiments through the image library module. The original images of the experimental observation objects can be images taken by users in real life, or drawn or synthesized images. For example, in the convex lens imaging experiment, users can upload images of candle flames taken by themselves, or images of any content through the image library module.

[0071] The original image in the image library module may have multiple attributes, for example, it may include image attributes, such as image format, image size, image dimension, image resolution, total image pixels, etc., and may also include user-defined attributes, such as image name, image tag, etc. The embodiment of the present invention does not limit this.

[0072] In addition, different scientific experiments have different requirements for experimental observation objects. For example, in the convex lens imaging experiment, the experimental observation object can be not only a candle flame but also any object, but in the metal oxide reduction experiment, the experimental observation object required can only be a metal oxide, such as copper oxide or iron oxide. Therefore, the image library module also includes: an image recognition unit and an image processing unit; the image recognition unit is used to identify whether the experimental observation object in the image uploaded by the user meets the requirements of the experimental observation object; the image processing unit is used to perform shearing, filtering and image enhancement on the images uploaded by the user that may contain noise, multiple types of objects, and insufficient clarity, and process the image materials uploaded by the user into the original image of the experimental observation object that meets the experimental requirements.

[0073] S120, obtaining a target operation instruction input by the user through the operation item module.

[0074] In scientific experiments, setting experimental conditions is an essential and crucial step. Experimental conditions directly affect the presentation of actual experimental results. Setting different experimental conditions may result in completely different actual experimental results. Therefore, the operation item module provides users with an operation entry that can control experimental conditions, and obtains the target operation instructions input by the user through the operation item module, where the target operation instructions are the experimental conditions of the scientific experiment corresponding to the experimental observation object.

[0075] It is understandable that the experimental conditions required for different experiments are also different, especially for experiments of different disciplines, the required experimental conditions may vary greatly. Exemplarily, the experimental conditions may include: environmental conditions, material conditions and time conditions. Environmental conditions may include environmental factors such as temperature, humidity, light, wind, pressure, rainfall and pH value of the environment in which the experimental observation object is located, which may affect the actual experimental results, or external interference conditions such as external forces and noise interference; Material conditions may include: materials and material properties required for the experiment, the required materials may be experimental equipment, chemicals and experimental tools, etc., the material properties may be the distance or position relationship between the experimental tools, the amount and state of the drugs, the parameters of the experimental equipment, etc.; Time conditions may be the duration of the experiment, such as heating time, standing time, reaction time, etc. The target operation instruction input by the user is the user's control instruction for the above-mentioned experimental conditions. The target operation instruction in the embodiment of the present invention is only used as an example, and there may be more different conditions in the actual experiment. The type of the target operation instruction can be determined according to the subject category, experimental content and experimental purpose of the simulation experiment.

[0076] In a specific example, in a convex lens imaging experiment, the target operation instruction input by the user may be that the distance between the candle flame and the convex lens is 30 cm, and the focal length of the convex lens is 10 cm. In a microscope observation of plant cells experiment, the target operation instruction input by the user may be that the magnification of the microscope eyepiece is 1000 times, and the distance between the eyepiece and the plant cell slice is 5 mm.

[0077] In another specific example, taking the experiment of water evaporation into water vapor as an example, the required experimental condition is the heating temperature. Therefore, the operation instruction input by the user can be to heat from a room temperature of 25 degrees at a rate of 5 degrees per second. The time required to complete the boiling of water is 15 seconds, so that the ancient simulation process of scientific experiments is not restricted by time and place.

[0078] S130, generating a scientific experiment theoretical result corresponding to the target operation instruction through a knowledge base module.

[0079] The knowledge base module is used to store the principles of scientific experiments, as well as the theoretical experimental results corresponding to the experimental observation objects that can be obtained based on the scientific experimental principles under the target operation instructions. The expression form of the principles of scientific experiments stored in the knowledge base module can be a logical expression, and the logical expression includes but is not limited to: any combination of formulas, principles, equations, functions, words or mathematical symbols. Among them, the scientific experiment theoretical results are the theoretical experimental results corresponding to the experimental observation objects after executing the target operation instructions. Therefore, the target operation instructions input by the user can also be used as part of the logical expression.

[0080] For example, in a convex lens imaging experiment, the imaging law of the convex lens is: when the object distance (the distance between the candle flame and the convex lens) is within one focal length, an upright and enlarged virtual image is obtained; when it is between one focal length and two focal lengths, an inverted and enlarged real image is obtained; when it is beyond two focal lengths, an inverted and reduced real image is obtained; when the object distance is equal to the focal length, no image is formed at this time. When the user selects the focal length attribute of the convex lens as 10cm, the target operation instruction input by the user in the operation item module is to set the distance between the candle flame and the convex lens to 30cm. Through the knowledge base module, it is known that when the object distance is beyond two focal lengths, an inverted and reduced real image is obtained. Therefore, the scientific experimental theoretical result corresponding to the target operation instruction is to obtain an inverted and reduced candle flame.

[0081] S140, inputting the original image and the scientific experimental theoretical result into the target network model through the image generation module to obtain a simulated image of the experimental observation object.

[0082] Among them, the simulated image is the image obtained by adjusting the original image by the target network model based on the theoretical results of scientific experiments.

[0083] The image generation module is used to use the original image and the scientific experimental theoretical results as input conditions of the target network model, and the target network model automatically generates a simulated image of the experimental observation object.

[0084] For example, taking the convex lens imaging experiment as an example, the upright candle flame image is used as the original image of the experimental observation object, the focal length of the convex lens is 10 cm, and the distance between the candle flame and the convex lens is 30 cm. By inputting the target network model, an inverted and reduced candle flame image can be obtained.

[0085] S150, displaying the simulated image and the original image through an experimental simulation module.

[0086] The experiment simulation module is used to provide the original image of the experimental observation object and the target operation instruction to the user, and then provide the user with real-time feedback of the simulated image corresponding to the target operation instruction. A display screen can be provided in the experiment simulation module to display the simulated image and the original image of the experimental observation object in real time.

[0087] In a specific example, taking the convex lens imaging experiment as an example, as the user drags the distance adjustment progress bar, different simulated images can be seen through the experimental simulation module, or an experimental simulation video composed of simulated images at different times, that is, when the object distance increases within one focal length, an upright, gradually enlarged candle flame image is presented on the experimental simulation module; when the object distance is equal to the focal length, no image is formed; when the object distance increases to between one and two focal lengths, an inverted, gradually enlarged candle flame image is presented on the experimental simulation module; when the object distance increases to beyond two focal lengths, an inverted, gradually shrinking candle flame image is presented on the experimental simulation module.

[0088] Displaying the simulated image and the original image through the experimental simulation module helps the user observe the changes in the experimental object and adjust the target operation instructions or correct the experimental plan in real time according to the experimental results reflected by the simulated image, so that the user can complete the scientific experiment and obtain the simulated image or video corresponding to the theoretical knowledge, thereby helping the user to understand the abstract theoretical knowledge and experience the fun of learning science.

[0089] For example, Figure 2 As shown, the specific steps of the embodiment of the present invention are: obtaining the original image of the experimental observation object through the picture library module, obtaining the target operation instruction data input by the user through the operation item module, inputting the target operation instruction data into the knowledge base module to obtain the scientific experiment theoretical result corresponding to the target operation instruction, inputting the original image and the scientific experiment theoretical result into the picture generation module, obtaining the simulated image of the experimental observation object, and feeding back the simulated image to the user in real time through the experimental simulation module.

[0090] The technical solution of this embodiment is to obtain the original image of the experimental observation object through the picture library module; obtain the target operation instruction input by the user through the operation item module, and the target operation instruction is the experimental condition of the scientific experiment corresponding to the experimental observation object; generate the scientific experiment theoretical result corresponding to the target operation instruction through the knowledge base module; the scientific experiment theoretical result is the theoretical experimental result corresponding to the experimental observation object after executing the target operation instruction; input the original image and the scientific experiment theoretical result into the target network model through the image generation module to obtain the simulated image of the experimental observation object, and the simulated image is the image adjusted by the target network model based on the scientific experiment theoretical result to the original image; display the simulated image and the original image through the experimental simulation module, so that the scientific experiment is not limited by time, place and the equipment and supplies required for the experiment, so that every student can have the opportunity to participate in the scientific experiment and experience the fun of learning science from the experiment. In addition, since the simulated image is fed back in real time, the user can adjust the experimental plan according to the simulated image fed back in real time, thereby improving the success rate of the experiment.

[0091] Optionally, obtaining a target operation instruction input by a user through the operation item module includes:

[0092] Displaying alternative operation items of the scientific experiment corresponding to the experimental observation object through the operation item module, so that the user can select a target operation item from the alternative operation items;

[0093] Acquire, through the operation item module, a target operation instruction of at least one target operation item input by at least one user;

[0094] The target operation instruction includes at least one of: a text, a numerical value, a selection instruction of a list option, a drag instruction of a progress bar, a touch instruction, and a trigger instruction of a physical button.

[0095] Since different scientific experiments require different experimental conditions, the operation items corresponding to the experimental conditions are also different. Therefore, in order to facilitate users to control the main experimental conditions of scientific experiments, the user or the system can set the operation items of the experimental observation objects as alternative operation items, and display the alternative operation items of the scientific experiments corresponding to the experimental observation objects through the operation item module, so that the user can select the current target operation item from the alternative operation items, and enter the operation instructions in the input box or adjustment bar corresponding to the target operation item.

[0096] In order to make the scientific experiment simulation system closer to the actual experimental operation process, the operation items that enable users to perform experimental operations and actual experimental operations in the scientific experiment simulation system can be input boxes, list options, progress bars, physical buttons or touch keys. Correspondingly, the target operation instructions include: text and values ​​entered in the input box, selection instructions of list options, drag instructions of progress bars, touch instructions of touch keys and at least one of the trigger instructions of physical buttons.

[0097] It should be noted that an experimental task can have one or more operation items. A user can operate one or more experimental objects, or multiple users can cooperate to operate multiple experimental objects separately. After the user operates through the human-computer interaction interface, the operation item results can be saved in various forms, including but not limited to: numerical values, matrices, coordinates, time, options, text, etc.

[0098] Optionally, the generating of the scientific experimental theoretical results corresponding to the target operation instruction by the knowledge base module includes:

[0099] Determine the scientific experimental theoretical result corresponding to the target operation instruction through the scientific experimental principle stored in the knowledge base module;

[0100] The scientific experiment principle is pre-stored in the system or input by the user.

[0101] In a specific example, the scientific experimental principles of basic experiments or commonly used experiments for primary and secondary school students can be preset in the database of the scientific experiment simulation system. The user only needs to select the experiment name in the experiment category, and then enter the target operation instructions according to the prompt information, and determine the scientific experiment theoretical results corresponding to the target operation instructions through the scientific experimental principles pre-stored in the knowledge base module.

[0102] In another specific example, the experiment simulation module can be used to obtain the scientific experiment principle, the original image of the experimental observation object, and the target operation instruction corresponding to the operation item input by the user, and determine the scientific experiment theoretical result corresponding to the target operation instruction.

[0103] Based on the scientific experimental principles pre-stored in the system or input by the user, and the user's operating instructions, the experiment is completed and the experimental results are obtained, so that scientific experiments are not restricted by the experimental location and the experimental equipment required for the experiment. Users can conduct scientific simulation experiments anytime and anywhere and experience the fun of learning science from experiments.

[0104] Embodiment 2

[0105] Figure 3This is a flow chart of a simulation method of a scientific experiment based on a generative adversarial network in Example 2 of the present invention. This embodiment is optimized based on the above embodiment. In this embodiment, the method also includes: after displaying the simulated image and the original image through the experimental simulation module, it also includes: saving scientific experiment data through the experimental simulation module, and sending the scientific experiment data to the server for storage; the scientific experiment data includes: simulation process information of the scientific experiment and notes written by the user.

[0106] like Figure 3 As shown, the method of this embodiment specifically includes the following steps:

[0107] S210, obtaining an original image of the experimental observation object through a picture library module.

[0108] S220, obtaining a target operation instruction input by a user through an operation item module, wherein the target operation instruction is an experimental condition of a scientific experiment corresponding to the experimental observation object.

[0109] S230, generating a scientific experiment theoretical result corresponding to the target operation instruction through a knowledge base module; the scientific experiment theoretical result is a theoretical experiment result corresponding to the experimental observation object after executing the target operation instruction.

[0110] S240, inputting the original image and the theoretical result of the scientific experiment into the target network model through the image generation module to obtain a simulated image of the experimental observation object, wherein the simulated image is an image obtained by adjusting the original image by the target network model based on the theoretical result of the scientific experiment.

[0111] The target network model is a well-trained generative adversarial network model (GAN). The GAN model is a deep learning model. The model produces a fairly good output through the mutual game learning of two modules: the generative model and the discriminative model.

[0112] The generative model is to model the conditional probability of the sample's feature vector x when the sample's label value y is known, that is, to model the conditional probability p(x|y). It studies what kind of probability distribution each sample follows. The discriminative model is just the opposite. When the sample's feature vector x is known, the probability of the sample's label value y is modeled, that is, to model the conditional probability p(y|x). This is generally used for components, that is, given a sample x, calculate the probability that it belongs to each class. Generative adversarial network (GAN) is to train the generator and discriminator models simultaneously through an adversarial process. The generator is mainly used to learn the distribution of real images so that the images it generates are more realistic, so that the discriminator cannot tell whether the generated data is real data. The discriminator can be a binary classifier that outputs 0 for generated images and 1 for real images. During the training process, the generator gradually becomes better at creating images that look real, while the discriminator becomes better at distinguishing them. The whole process can be seen as a game between the generator and the discriminator. As time goes by, the two networks eventually reach a dynamic equilibrium, that is, the two models compete with each other and finally reach the global optimum.

[0113] In this embodiment, the image generation module uses the original image of the experimental observation object and the scientific experimental theoretical results generated by the knowledge base module based on the target operation instructions as conditional inputs of the GAN model, and generates a simulated image that meets the conditions through the confrontation between the generator and the discriminator of the GAN model.

[0114] In a specific example, a candle flame picture is used as the original image of the experimental observation object, and an inverted and reduced real image is used as the target operation instruction as the theoretical result of the scientific experiment when the object distance is 30 cm. The original image and the theoretical result of the scientific experiment are used as the input of the GAN model. The GAN model will generate an inverted and reduced real image of the flame. Even if the GAN model has not seen the input candle flame picture during training, after learning the distribution of the real flame image and various semantic representations, it can also generate corresponding images according to the conditions.

[0115] It should be noted that the embodiments of the present invention do not limit the GAN model, and may be a derivative model based on the GAN model principle. For example, it may be CGAN, DCGAN, DRGAN. As long as the target network model can automatically generate a synthetic image that meets the condition description according to the conditions and the image.

[0116] For example, the training process of the GAN model can be to input the randomly sampled noise z~N(0,1) and text features into the generator, and input the real image and its corresponding text features, as well as the synthetic image and any text features into the discriminator. The discriminator needs to identify two situations: one is to identify the synthetic image, and the other is the real image and the mismatched text. In order to enable the discriminator to better learn the correspondence between text features and image content, the input of the discriminator can be increased: real images and wrong text features, so that the discriminator can better judge whether the generator generates images according to the text.

[0117] S250, displaying the simulated image and the original image through an experimental simulation module.

[0118] Specifically, the experiment simulation module includes: a simulation image display unit, which is used to display the simulation image and the original image.

[0119] S260, saving scientific experiment data through the experiment simulation module, and sending the scientific experiment data to the server for storage.

[0120] The scientific experiment data includes: simulation process information of the scientific experiment and notes written by the user.

[0121] Specifically, the experimental simulation module can provide guidance and recording functions while presenting the experimental simulation image. Therefore, the experimental simulation module also includes: an experimental recording unit. The experimental recording unit is used to allow users to record the observed experimental process and results, as well as the experimental analysis and experimental feelings, and answer according to the learning tasks and test requirements. The recording method of scientific experimental data includes but is not limited to: text boxes, tables, data uploaded by handwriting boards, and voice, etc. The data recorded by users will be transmitted to the learning system of the server for processing or storage. Furthermore, when multiple users work together to complete the scientific experiment simulation, the data recorded by each user and the data recorded by the group can be transmitted to the learning system of the server respectively.

[0122] Since the results of some experiments are often fleeting, by saving scientific experiment data records, users can repeatedly watch the experimental process and experimental notes to learn and fill in the gaps, and review the experimental operation process that did not produce the expected results, standardize the user's experimental operation behavior, and improve the user's experimental operation ability.

[0123] The technical solution of this embodiment is to obtain the original image of the experimental observation object through the picture library module; obtain the target operation instruction input by the user through the operation item module, and the target operation instruction is the experimental condition of the scientific experiment corresponding to the experimental observation object; generate the scientific experiment theoretical result corresponding to the target operation instruction through the knowledge base module; the scientific experiment theoretical result is the theoretical experimental result corresponding to the experimental observation object after executing the target operation instruction; input the original image and the scientific experiment theoretical result into the target network model through the image generation module to obtain the simulated image of the experimental observation object, and the simulated image is the image adjusted by the target network model based on the scientific experiment theoretical result to the original image; display the simulated image and the original image through the experimental simulation module, so that the scientific experiment is not limited by time, place and the equipment and supplies required for the experiment, so that every student can have the opportunity to participate in the scientific experiment and experience the fun of learning science from the experiment. In addition, since the simulated image is fed back in real time, the user can adjust the experimental plan according to the simulated image fed back in real time, thereby improving the success rate of the experiment.

[0124] Optionally, the experimental simulation module provides users with auxiliary tools and auxiliary information required for scientific experimental simulation;

[0125] The experimental content information includes: experimental problem description information, experimental theoretical knowledge information and experimental instructions information.

[0126] Specifically, the experiment simulation module also includes: an auxiliary information unit, which includes: an experiment task bar, an experiment theory knowledge bar, an experiment instructions bar and an auxiliary tool bar. The auxiliary information unit is to provide auxiliary tools and auxiliary information for users to conduct scientific experiments and observe experimental simulation images.

[0127] Among them, the experimental problem description column is used to describe the experimental problem information, the experimental theoretical knowledge column can be used to display the experimental theoretical knowledge information, such as information describing the basic principles of the experiment, and the experimental instructions column is used to display the experimental instructions information, such as the experimental scoring requirements and experimental precautions, etc.; the auxiliary toolbar includes a ruler, a standard measuring cup and an electronic scale, etc.

[0128] For example, Figure 4 is a schematic diagram of an interface of an experimental simulation module in an embodiment of the present invention, such as Figure 4 As shown, the experiment simulation module includes: a simulation image display unit, an experiment recording unit and an auxiliary information unit. The auxiliary information unit includes: an auxiliary toolbar, an experiment task bar, an experiment theory knowledge bar and an experiment instructions bar. The experiment simulation module can interact with the user, so that the user can complete the experiment through the simulation system of scientific experiments anytime and anywhere, reflecting the fun of learning theoretical knowledge from experiments.

[0129] Embodiment 3

[0130] Figure 5 This is a schematic diagram of the structure of a scientific experiment simulation system provided in the third embodiment of the present invention. This embodiment can be applied to STEM education, and the simulation system simulates the situation of scientific experiments. The system can be implemented in software and / or hardware, and the system can be integrated in any device that provides the function of simulating scientific experiments, such as Figure 5 As shown, the simulation system of the scientific experiment specifically includes: a picture library module 310, an operation item module 320, a knowledge base module 330, an image generation module 340 and an experiment simulation module 350.

[0131] The image library module 310 is used to obtain the original image of the experimental observation object;

[0132] The operation item module 320 is used to obtain the target operation instruction input by the user, and the target operation instruction is the experimental condition of the scientific experiment corresponding to the experimental observation object.

[0133] The knowledge base module 330 is used to generate the scientific experiment theoretical result corresponding to the target operation instruction; the scientific experiment theoretical result is the theoretical result of the scientific experiment corresponding to the experimental observation object after executing the target operation instruction.

[0134] The image generation module 340 is used to input the original image and the theoretical result of the scientific experiment into the target network model to obtain a simulated image of the experimental observation object, wherein the simulated image is an image obtained by adjusting the original image by the target network model based on the theoretical result of the scientific experiment.

[0135] The experiment simulation module 350 is used to display the simulated image and the original image.

[0136] Optionally, the picture library module 310 is specifically used for:

[0137] The original image of the experimental observation object is obtained from a database, or the original image of the experimental observation object uploaded by a user is obtained.

[0138] Optionally, the operation item module 320 is specifically used to:

[0139] Displaying alternative operation items of the scientific experiment corresponding to the experimental observation object through the operation item module, so that the user can select a target operation item from the alternative operation items;

[0140] Acquire, through the operation item module, a target operation instruction of at least one target operation item input by at least one user;

[0141] The target operation instruction includes at least one of: a text, a numerical value, a selection instruction of a list option, a drag instruction of a progress bar, a touch instruction, and a trigger instruction of a physical button.

[0142] Optionally, the knowledge base module 330 is specifically used for:

[0143] Determine the scientific experimental theoretical result corresponding to the target operation instruction through the scientific experimental principle stored in the knowledge base module;

[0144] The scientific experiment principle is pre-stored in the system or input by the user.

[0145] Optionally, the target network model is a fully trained generative adversarial network model.

[0146] Optionally, the experiment simulation module 350 is specifically used for:

[0147] Save scientific experiment data and send the scientific experiment data to a server for storage;

[0148] The scientific experiment data includes: simulation process information of the scientific experiment and notes written by the user.

[0149] Optionally, the experiment simulation module 350 is further used for:

[0150] Provide users with auxiliary tools and auxiliary information required for scientific experiment simulation through the experiment simulation module;

[0151] The auxiliary information includes: experimental problem description information, experimental theoretical knowledge information and experimental instructions information.

[0152] The above-mentioned product can execute the simulation method of scientific experiments based on generative adversarial networks provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0153] Embodiment 4

[0154] Figure 6 This is a schematic diagram of the structure of a computer device in Embodiment 4 of the present invention. Figure 6 A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 6 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0155] like Figure 6As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processor 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processor 16).

[0156] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0157] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0158] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 6 not shown, usually called a "hard drive"). Although Figure 6 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0159] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0160] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), may also communicate with one or more devices that enable a user to interact with the computer device 12, and / or communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 22. In addition, the computer device 12 in this embodiment, the display 24 does not exist as an independent individual, but is embedded in the mirror surface, and when the display surface of the display 24 is not displayed, the display surface of the display 24 and the mirror surface are visually integrated. In addition, the computer device 12 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) through a network adapter 20. As shown in the figure, the network adapter 20 communicates with other modules of the computer device 12 through a bus 18. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in conjunction with the computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0161] The processor 16 executes various functional applications and data processing by running the program stored in the system memory 28, for example, implementing the simulation method of scientific experiments based on generative adversarial networks provided in an embodiment of the present invention: obtaining the original image of the experimental observation object through the picture library module; obtaining the target operation instruction input by the user through the operation item module, and the target operation instruction is the experimental condition of the scientific experiment corresponding to the experimental observation object; generating the scientific experiment theoretical result corresponding to the target operation instruction through the knowledge base module; the scientific experiment theoretical result is the theoretical experimental result corresponding to the experimental observation object after executing the target operation instruction; inputting the original image and the scientific experiment theoretical result into the target network model through the image generation module to obtain a simulated image of the experimental observation object, and the simulated image is the image after the target network model adjusts the original image based on the scientific experiment theoretical result; and displaying the simulated image and the original image through the experimental simulation module.

[0162] Embodiment 5

[0163] Embodiment 5 of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a simulation method for a scientific experiment based on a generative adversarial network as provided in all the embodiments of the invention of this application: obtaining an original image of an experimental observation object through a picture library module; obtaining a target operation instruction input by a user through an operation item module, wherein the target operation instruction is the experimental condition of the scientific experiment corresponding to the experimental observation object; generating a scientific experiment theoretical result corresponding to the target operation instruction through a knowledge base module; the scientific experiment theoretical result is the theoretical experimental result corresponding to the experimental observation object after executing the target operation instruction; inputting the original image and the scientific experiment theoretical result into a target network model through an image generation module to obtain a simulated image of the experimental observation object, wherein the simulated image is an image obtained after the target network model adjusts the original image based on the scientific experiment theoretical result; and displaying the simulated image and the original image through an experimental simulation module.

[0164] Any combination of one or more computer-readable media can be used. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0165] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0166] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0167] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0168] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A simulation method for scientific experiments based on generative adversarial networks, characterized in that: A simulation system applied to scientific experiments, the simulation system comprising a picture library module, an operation item module, an image generation module, a knowledge base module and an experiment simulation module, the method comprising: Obtain the original image of the experimental observation object through the image library module; Acquire a target operation instruction input by a user through an operation item module, wherein the target operation instruction is an experimental condition of a scientific experiment corresponding to the experimental observation object; Generate a scientific experiment theoretical result corresponding to the target operation instruction through a knowledge base module; the scientific experiment theoretical result is a theoretical experiment result corresponding to the experimental observation object after executing the target operation instruction; The original image and the scientific experimental theoretical result are input into the target network model through the image generation module to obtain a simulated image of the experimental observation object, wherein the simulated image is an image obtained by adjusting the original image by the target network model based on the scientific experimental theoretical result; Displaying the simulated image and the original image through an experimental simulation module; Wherein, obtaining the target operation instruction input by the user through the operation item module includes: Displaying alternative operation items of the scientific experiment corresponding to the experimental observation object through the operation item module, so that the user can select a target operation item from the alternative operation items; Acquire, through the operation item module, a target operation instruction of at least one target operation item input by at least one user; The target operation instruction includes at least one of: a text, a numerical value, a selection instruction of a list option, a drag instruction of a progress bar, a touch instruction, and a trigger instruction of a physical button.

2. The method according to claim 1, characterized in that: The original images of the experimental observation object obtained through the image library module include: The original image of the experimental observation object is obtained from a database, or the original image of the experimental observation object uploaded by a user is obtained.

3. The method according to claim 1, characterized in that The generating of the scientific experimental theoretical results corresponding to the target operation instructions by the knowledge base module includes: Determine the scientific experimental theoretical result corresponding to the target operation instruction through the scientific experimental principle stored in the knowledge base module; The scientific experiment principle is pre-stored in the system or input by the user.

4. The method according to claim 1, characterized in that The target network model is a fully trained generative adversarial network model.

5. The method according to claim 1, characterized in that After displaying the simulated image and the original image through the experimental simulation module, the method further includes: Save scientific experiment data through the experiment simulation module, and send the scientific experiment data to the server for storage; The scientific experiment data includes: simulation process information of the scientific experiment and notes written by the user.

6. The method according to claim 1, characterized in that Also includes: Provide users with auxiliary tools and auxiliary information required for scientific experiment simulation through the experiment simulation module; The auxiliary information includes: experimental problem description information, experimental theoretical knowledge information and experimental instructions information.

7. A simulation system for scientific experiments, characterized in that: include: Image library module, used to obtain the original images of the experimental observation objects; An operation item module is used to obtain a target operation instruction input by a user, wherein the target operation instruction is an experimental condition of a scientific experiment corresponding to the experimental observation object; The operation item module is specifically used for: Displaying alternative operation items of the scientific experiment corresponding to the experimental observation object through the operation item module, so that the user can select a target operation item from the alternative operation items; Acquire, through the operation item module, a target operation instruction of at least one target operation item input by at least one user; The target operation instruction includes: at least one of: a text, a numerical value, a list option selection instruction, a progress bar drag instruction, a touch instruction, and a physical button trigger instruction; A knowledge base module, used to generate a scientific experiment theoretical result corresponding to the target operation instruction; the scientific experiment theoretical result is a theoretical result of the scientific experiment corresponding to the experimental observation object after executing the target operation instruction; An image generation module, used for inputting the original image and the scientific experimental theoretical result into a target network model to obtain a simulated image of the experimental observation object, wherein the simulated image is an image obtained by adjusting the original image by the target network model based on the scientific experimental theoretical result; The experimental simulation module is used to display the simulated image and the original image.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the simulation method of scientific experiments based on generative adversarial networks as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it implements the simulation method of scientific experiments based on generative adversarial networks as described in any one of claims 1-6.

Images