Virtual experiment method, system and equipment and storage medium
Through virtual experimental methods and visual enhancement equipment, the chemical experimental process was simulated, and the dangerous problems of students operating chemical reagents during chemical experimental training were solved, and the experimental safety was improved.
Patent Information
- Application Number
- CN202411440740.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-15
- Publication Date
- 2025-02-11
AI Technical Summary
During the chemistry experiment training, students are at risk when operating chemical reagents, especially those who are just beginning to learn chemistry, are prone to accidents and fires due to lack of protective experience.
Using virtual experimental methods, the operation steps of the experimental process are simulated throughout the entire process through visual enhancement devices (such as MR glasses or AR glasses), and feedback corresponding visual effects are implemented to avoid contact with chemical reagents.
It greatly improves the safety of experiments, avoids direct contact and operation of chemical reagents, reduces the risk of accidents, and is especially suitable for beginners.
Smart Images
Figure CN119418571A_ABST
Abstract
Description
Background Art
[0002] Chemical experiment training refers to the process of conducting experimental exploration and practical training through actual operations on the basis of learning chemical theoretical knowledge. Chemical experiment training can help students consolidate theoretical knowledge, cultivate experimental skills, and improve their ability in experimental design and data analysis.
[0003] The current chemical experiment training requires trainees to actually mix and operate chemical reagents. However, since chemical reagents are often corrosive, they are dangerous to trainees. In addition, since trainees are still in the operation learning period, their operation proficiency is relatively low. They are prone to mistakes and burns from chemical reagents during operation, which further increases the danger. Especially for students who have just learned chemistry, due to the lack of protection experience, students are more likely to be in danger when directly operating chemical reagents, alcohol lamps, etc., and even fires in the laboratory.
[0004] Therefore, the present invention provides a virtual experiment method, system, device and storage medium. Summary of the invention
[0005] In response to the problems in the prior art, the purpose of the present invention is to provide a virtual experiment method, system, equipment and storage medium, which overcomes the difficulties of the prior art, can simulate the operating steps of the experimental process throughout and implement corresponding visual effects for feedback, avoid contact with chemical reagents, and greatly improve the safety of the experiment.
[0006] An embodiment of the present invention provides a virtual experiment method, comprising the following steps:
[0007] S110, generating independent suspension areas respectively according to the chemical reagent information of the preset experiment in the display screen of the visual enhancement device;
[0008] S120, identifying the current image collected in real time by the visual enhancement device to obtain a partial image of each vessel in the current image;
[0009] S130, pairing the vessel with the independent suspension area, and generating a chemical reagent posture image based at least on the corresponding multimedia content of the chemical reagent information and superimposing it on a partial image corresponding to the vessel;
[0010] S150, when detecting that chemical reagent posture images corresponding to at least two chemical reagent information are overlapped, generate a chemical reagent posture image according to multimedia content corresponding to the chemical reagent information after the preset reaction, and at least superimpose the picture in the chemical reagent posture image onto the partial image corresponding to the vessel.
[0011] Preferably, in step S110, the visual enhancement device is MR glasses, the vessel is a virtual vessel selected by an interactive device based on the MR glasses, and the independent floating area is a virtual periodic table generated based on the MR glasses, and a corresponding floating mark area is generated according to an element area corresponding to the chemical reagent information of a preset experiment.
[0012] Preferably, the step S110 further includes a physical display cabinet of the periodic table, the visual enhancement device is MR glasses, the vessel is a virtual vessel selected by an interactive device based on the MR glasses, and the independent floating area is based on the MR glasses collecting a picture of the physical display cabinet of the periodic table, and after image recognition, a corresponding floating marking area is generated according to the cabinet corresponding to the chemical reagent information of the preset experiment.
[0013] Preferably, in step S110, the visual enhancement device is an AR glasses, the vessel is a physical vessel, and the independent floating area is a virtual area generated based on the AR glasses.
[0014] Preferably, the step S110 further includes a physical display cabinet of the periodic table, in which various physical objects of various elements are stored in various drawers inside, the visual enhancement device is AR glasses, the vessel is a physical vessel, and the independent floating area is based on the AR glasses collecting a picture of the physical display cabinet of the periodic table, and after image recognition, a corresponding floating marking area is generated according to the drawers corresponding to the chemical reagent information of the preset experiment.
[0015] Preferably, the step S110 includes:
[0016] S111, generating non-overlapping independent suspension areas in a display screen of a visual enhancement device according to chemical reagent information of each preset experiment, wherein each of the chemical reagent information includes textual description information about the chemical reagent and at least one of the mass, dosage, and molar amount of the chemical reagent;
[0017] S112: Displaying text description information in the chemical reagent information in the independent suspension area.
[0018] Preferably, the step S130 includes:
[0019] S131, pairing according to the interaction action between the partial image of the vessel and the independent floating area after the partial image of the vessel is lifted;
[0020] S132, generating a chemical reagent posture image matching the partial image based on the corresponding multimedia content of the chemical reagent information and the partial image of the vessel;
[0021] S133, superimposing the chemical reagent posture image onto the local image corresponding to the vessel.
[0022] Preferably, in step S131, the interactive action is to lift the vessel so that the local image corresponding to the vessel overlaps with the independent floating area, and if the overlapping time and / or the area of the overlapping area exceeds a preset threshold, the vessel is paired with the chemical reagent information corresponding to the independent floating area.
[0023] Preferably, after step S130 and before step S150, the following steps are further included:
[0024] S140, detecting the posture of the container, and updating the chemical reagent posture image in real time at least according to the change of the tilting angle of the container.
[0025] Preferably, the step S150 includes:
[0026] S151, detecting in real time whether chemical reagent posture images corresponding to at least two types of chemical reagent information overlap in the local image of the same vessel;
[0027] S152, when overlap occurs, generating a chemical reagent posture image according to the preset post-reaction chemical reagent information corresponding to the combination of the two chemical reagent information and the corresponding multimedia content;
[0028] S153, at least superimpose the image in the chemical reagent posture image onto the partial image corresponding to the vessel.
[0029] Preferably, the step S151 further comprises: superimposing a preset chemical reaction image when the combination of the chemical reagent information reacts onto a local image corresponding to the vessel.
[0030] An embodiment of the present invention further provides a virtual experiment system for implementing the above virtual experiment method, wherein the virtual experiment system comprises:
[0031] A suspension display module generates independent suspension areas in the display screen of the visual enhancement device according to the chemical reagent information of the preset experiment;
[0032] A vessel recognition module recognizes the current image collected in real time by the visual enhancement device to obtain a partial image of each vessel in the current image;
[0033] A vessel pairing module pairs the vessel with the independent suspension area, and generates a chemical reagent posture image superimposed on a local image corresponding to the vessel based at least on the corresponding multimedia content of the chemical reagent information;
[0034] The reaction interaction module detects that when chemical reagent posture images corresponding to at least two chemical reagent information overlap, a chemical reagent posture image is generated according to multimedia content corresponding to the chemical reagent information after the preset reaction, and at least the picture in the chemical reagent posture image is superimposed on the partial image corresponding to the vessel.
[0035] An embodiment of the present invention further provides a virtual experiment device, comprising:
[0036] processor;
[0037] a memory storing executable instructions of the processor;
[0038] Wherein, the processor is configured to perform the steps of the above-mentioned virtual experiment method by executing the executable instructions.
[0039] An embodiment of the present invention further provides a computer-readable storage medium for storing a program, wherein the program implements the steps of the above-mentioned virtual experiment method when executed.
[0040] The purpose of the present invention is to provide a virtual experiment method, system, equipment and storage medium, which can simulate the operation steps of the experimental process throughout and implement the corresponding visual effects of feedback, avoid contact with chemical reagents, and greatly improve the safety of the experiment. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Other features, objectives and advantages of the present invention will become more apparent from a reading of the detailed description of non-limiting embodiments made with reference to the following accompanying drawings.
[0042] Figure 1 It is a flow chart of the virtual experiment method of the present invention.
[0043] Figures 2 to 7 It is a schematic diagram of the first implementation process of the virtual experiment method of the present invention.
[0044] Figures 8 to 13 It is a schematic diagram of the second implementation process of the virtual experiment method of the present invention.
[0045] Fig.14 It is a structural schematic diagram of the virtual experiment system of the present invention.
[0046] Fig.15 It is a structural schematic diagram of the virtual experiment equipment of the present invention.
[0047] Fig.16 It is a schematic diagram of the structure of a computer-readable storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION
[0048] The following describes the implementation methods of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in the present application. The present application can also be implemented or applied through other different specific implementation methods, and the details in the present application can also be modified or changed in various ways according to different viewpoints and application systems without departing from the spirit of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.
[0049] The following is a detailed description of the embodiments of the present application with reference to the accompanying drawings so that those skilled in the art can easily implement the present application. The present application can be embodied in a variety of different forms and is not limited to the embodiments described herein.
[0050] In the representations of the present application, the representations with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" etc. mean that the specific features, structures, materials or characteristics represented in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the represented specific features, structures, materials or characteristics may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples represented in the present application and the features of the different embodiments or examples, unless they contradict each other.
[0051] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the representation of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0052] In order to clearly describe the present application, components irrelevant to the description are omitted, and the same reference symbols are given to the same or similar components throughout the specification.
[0053] Throughout the specification, when a device is said to be "connected" to another device, this includes not only the case of "direct connection" but also the case of "indirect connection" by placing other elements therebetween. In addition, when a device is said to "include" a certain component, unless otherwise stated, it does not exclude other components, but means that other components may be included.
[0054] When a device is said to be "on" another device, it may be directly on the other device, but there may also be other devices between it. In contrast, when a device is said to be "directly" on another device, there are no other devices between it.
[0055] Although the terms first, second, etc. are used to represent various elements in some examples, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, the first interface and the second interface, etc. are represented. Moreover, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising" and "including" indicate the existence of features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Therefore, "A, B or C" or "A, B and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C". Only when the combination of elements, functions, steps or operations is inherently mutually exclusive in some way, will there be an exception to this definition.
[0056] The technical terms used herein are only used to refer to specific embodiments and are not intended to limit the present application. The singular form used herein also includes the plural form unless the sentence clearly indicates the contrary meaning. The meaning of "including" used in the specification is to specify specific characteristics, regions, integers, steps, operations, elements and / or components, and does not exclude the existence or addition of other characteristics, regions, integers, steps, operations, elements and / or components.
[0057] Although not defined differently, all terms, including technical and scientific terms used herein, have the same meaning as those generally understood by those skilled in the art to which this application belongs. Terms defined in commonly used dictionaries are additionally interpreted as having meanings consistent with the relevant technical literature and the contents of the current disclosure, and unless defined, shall not be overly interpreted as ideal or very formal meanings.
[0058] Figure 1 It is a flow chart of the virtual experiment method of the present invention. Figure 1 As shown, the virtual experiment method of the present invention includes:
[0059] S110, generating independent suspension areas respectively according to the chemical reagent information of the preset experiment in the display screen of the visual enhancement device.
[0060] S120, identifying the current image collected in real time by the visual enhancement device to obtain a partial image of each vessel in the current image.
[0061] S130, pairing the vessel with the independent suspension area, and generating a chemical reagent posture image superimposed on a partial image of the corresponding vessel based at least on the corresponding multimedia content of the chemical reagent information.
[0062] S150, when detecting that chemical reagent posture images corresponding to at least two chemical reagent information are overlapped, generate a chemical reagent posture image according to multimedia content corresponding to the preset post-reaction chemical reagent information, and at least superimpose the picture in the chemical reagent posture image onto a partial image of the corresponding vessel.
[0063] In a preferred embodiment, in step S110, the visual enhancement device is MR glasses, the vessel is a virtual vessel selected by an interactive device based on the MR glasses, the independent floating area is in a virtual periodic table generated based on the MR glasses (each grid in the virtual periodic table has the name of each element written on it), and a corresponding floating mark area is generated according to the element area corresponding to the chemical reagent information of the preset experiment, but not limited to this.
[0064] In a preferred embodiment, step S110 further includes a physical display cabinet of the periodic table, the visual enhancement device is MR glasses, the vessel is a virtual vessel selected by an interactive device based on the MR glasses, and the independent floating area is based on the MR glasses collecting a picture of the physical display cabinet of the periodic table, and after image recognition, a corresponding floating marking area is generated according to the cabinet corresponding to the chemical reagent information of the preset experiment, but it is not limited to this.
[0065] In a preferred embodiment, in step S110, the visual enhancement device is an AR glasses, the vessel is a physical vessel, and the independent floating area is a virtual area generated based on the AR glasses, but not limited thereto.
[0066] In a preferred embodiment, step S110 further includes a physical display cabinet of the periodic table, in which various physical elements are stored in various drawers, the visual enhancement device is AR glasses, the utensils are physical utensils, and the independent floating area is based on the AR glasses collecting pictures of the physical display cabinet of the periodic table, and after image recognition, the corresponding floating marking area is generated according to the drawers corresponding to the chemical reagent information of the preset experiment, but it is not limited to this.
[0067] After using the virtual experiment method of the present invention, users can use empty virtual vessels (no need to load chemicals throughout the process) or virtual experiment vessels to conduct chemical experiments. No real chemical reagents are needed throughout the process. The MR device / AR device recognizes and tracks the vessel, superimposes the real-time liquid (or solid particles, powder) pattern representing the chemical reagent on the local pattern of the vessel, and adjusts the accumulation or flow posture of the liquid (or solid particles, powder) in the vessel according to the posture of the vessel, so that when observed from the MR device / AR device, it looks like real chemical reagents are stored in the vessel. The operation steps of the experimental process can be simulated throughout the process and the corresponding visual effects can be implemented to avoid contact with chemical reagents, thereby greatly improving the safety of the experiment.
[0068] In a preferred embodiment, step S110 includes:
[0069] S111. Generate non-overlapping independent suspended areas in the display screen of the visual enhancement device according to the chemical reagent information of each preset experiment, wherein each chemical reagent information includes text description information about the chemical reagent and at least one of the mass, dosage, and molar amount of the chemical reagent.
[0070] S112: Displaying text description information in the chemical reagent information in the independent suspension area, but not limited to this.
[0071] In a preferred embodiment, step S130 includes:
[0072] S131, pairing is performed according to the interaction action between the partial image of the vessel and the independent floating area after the partial image of the vessel is lifted up.
[0073] S132: Generate a chemical reagent posture image that matches the local image based on the corresponding multimedia content of the chemical reagent information and the local image of the vessel.
[0074] S133, superimposing the chemical reagent posture image onto the local image of the corresponding container, but not limited to this.
[0075] In a preferred embodiment, in step S131, the interactive action is to lift the vessel so that the local image corresponding to the vessel overlaps with the independent floating area, and the overlapping time and / or the area of the overlapping area exceeds a preset threshold, then the chemical reagent information corresponding to the vessel and the independent floating area is paired, but not limited to this.
[0076] In a preferred embodiment, after step S130 and before step S150, the method further includes:
[0077] S140, detecting the posture of the vessel, at least updating the chemical reagent posture image in real time according to the change of the tilting angle of the vessel, but not limited to this. In the present invention, the chemical reagent posture image can be updated in real time according to the change of the tilting angle of the vessel through the existing physical simulation software, for example: adjusting the posture of the liquid (or solid particles, powder) stacking or flowing in the vessel according to the posture of the vessel, but not limited to this. Among them, physical simulation software plays an important role in scientific research and education. They can simulate various physical phenomena and help researchers and educators better understand and predict natural laws. For example: RealFlow software, produced by Spain's Next Limit company, is a fluid dynamics simulation software that can calculate the movement of moving objects in the real world, including liquids. It provides a series of well-designed tools, such as fluid simulation, grid generator, rigid body dynamics with constraints, etc., which are widely used in film, TV series and game production.
[0078] In a preferred embodiment, step S150 includes:
[0079] S151. Detect in real time whether chemical reagent posture images corresponding to at least two types of chemical reagent information overlap in a partial image of the same container.
[0080] S152: When overlap occurs, a chemical reagent posture image is generated according to the preset post-reaction chemical reagent information corresponding to the combination of the two chemical reagent information and the corresponding multimedia content.
[0081] S153, at least superimpose the image in the chemical reagent posture image onto the partial image of the corresponding container, but not limited thereto.
[0082] In a preferred embodiment, step S151 also includes: superimposing a preset chemical reaction image when the combination of chemical reagent information reacts onto a local image of the corresponding vessel. The preset chemical reaction image can be in video form, for example: the process of reagent luminescence, reagent color change, generation of flames, color explosion sounds, etc., but is not limited to this.
[0083] The present invention provides a new experimental simulation experience. Through the simplified interactive operation process, it can greatly reduce the difficulty of users in conducting chemical experiments and provide safety reminders. In particular, the entire process no longer requires tedious computer operations, command input, etc., which greatly optimizes the user experience of using MR devices / AR devices for experiments. It is especially suitable for student users who need as many mature chemical experiments as possible but lack experimental experience.
[0084] Figures 2 to 7 Schematic diagram of the second implementation process of the virtual experiment method of the present invention. Figures 2 to 7To introduce the second specific implementation process of the present invention ( Figures 3 to 7 ) is a schematic diagram of viewing through MR glasses 32:
[0085] like Figure 2 As shown, there is a physical display cabinet of the periodic table, MR glasses 32, and an interactive device 31 (control handle) on site, wherein the physical display cabinet of the periodic table has several drawers, each of which is provided with a corresponding chemical element, and the chemical name is written on the outside of the drawer. The user wears the MR glasses 32 and the interactive device 31, and activates the preset experimental project of this time. The MR glasses 32 in this embodiment have functions such as image acquisition, image display, and audio playback. After activating the preset experimental project of this time in the MR glasses 32, three empty virtual experimental vessels are generated, namely the first test tube 11, the beaker 12, and the second test tube 13. The user can drag and lift the virtual experimental vessel through the interactive device 31.
[0086] like Figure 3 As shown, the user can directly see the first test tube 11, the beaker 12, and the second test tube 13 through the transparent lenses of the MR glasses 32. When the user uses the MR glasses 32 to view the physical display cabinet of the periodic table, the MR glasses 32 collects a picture of the physical display cabinet of the periodic table, and after image recognition, generates an independent suspension area in the corresponding cabinet area according to the chemical name of the chemical element on the cabinet corresponding to the chemical reagent information of the preset experiment. A, B, C (the independent suspension areas A, B, C can be blocks of different colors, which are displayed in the transparent lens). In addition, each chemical reagent information includes text description information about the chemical reagent and at least one of the mass, dosage, and molar amount of the chemical reagent, and the text description information in the chemical reagent information is displayed in the independent suspension area. At this time, the user can observe from the MR glasses 32 that the floating areas A, B, and C are floating above the desktop and do not overlap with the virtual experimental equipment. There is corresponding text description information "XXXXXXXX" in the floating area A, corresponding text description information "YYYY" in the floating area B, and corresponding text description information "ZZZZZZZZ" in the floating area C. The specific text content will not be repeated here.
[0087] like Figure 4As shown, the current image captured in real time by the MR glasses 32 is recognized to obtain a partial image of each virtual laboratory vessel (first test tube 11, beaker 12, second test tube 13) in the current image. Pairing is performed according to the interaction between the partial image of the virtual laboratory vessel and the independent suspension area after it is lifted up. The interaction is to lift the virtual laboratory vessel so that the partial image corresponding to the virtual laboratory vessel overlaps with the independent suspension area, and the overlapping time and / or the area of the overlapping area exceeds a preset threshold, then the virtual laboratory vessel is paired with the chemical reagent information corresponding to the independent suspension area, but not limited to this. For example, when the user lifts the first test tube 11, so that the partial image corresponding to the first test tube 11 overlaps with the independent suspension area A by 1, and the overlapping area 41 is greater than 50% of the partial image area, the virtual laboratory vessel is paired with the chemical reagent information corresponding to the independent suspension area. The beaker 12 and the second test tube 13 are paired in the same process, which will not be repeated here.
[0088] like Figure 5 As shown, the posture of the virtual laboratory vessel is detected, and the chemical reagent posture image is updated in real time according to the change of the tilting angle of the virtual laboratory vessel at least. In the present invention, the chemical reagent posture image can be updated in real time according to the change of the tilting angle of the virtual laboratory vessel through the existing physical simulation software, for example: the posture of the liquid (or solid particles, powder) stacking or flowing in the virtual laboratory vessel is adjusted according to the posture of the virtual laboratory vessel. At this time, the first test tube 11 and the second test tube 13 are both configured with multimedia images corresponding to different chemical reagents (chemical reagent posture image 42 and chemical reagent posture image 43, respectively showing the image of two chemical reagents deposited at the bottom of the test tube, at this time, the user can see the first test tube 11 and the second test tube 13 through the transparent lens, and can also see the corresponding chemical reagent posture image 42 and chemical reagent posture image 43 from the transparent display area, thereby realizing the visual superposition state, the chemical reagent posture image 42 is superimposed on the first test tube 11, and the chemical reagent posture image 43 is superimposed on the second test tube 13, just like seeing the first test tube 11 and the second test tube 13 filled with real chemical reagents). Based on the corresponding multimedia content of the chemical reagent information and the local image of the virtual laboratory vessel, a chemical reagent posture image matching the local image is generated. The chemical reagent posture image can be updated in real time according to the change of the pouring angle of the virtual laboratory vessel through the existing physical simulation software (for example, under normal circumstances, the posture of the chemical reagent is mainly deposited at the bottom of the virtual laboratory vessel).
[0089] like Figure 6As shown, when the user pours the first test tube 11 and the second test tube 13 into the beaker 12 respectively according to the prompt of the preset experiment, a chemical reagent posture image matching the local image is generated again based on the corresponding multimedia content of the chemical reagent information and the local image of the virtual laboratory vessel. The chemical reagent posture image can be updated in real time according to the change of the pouring angle of the virtual laboratory vessel through the existing physical simulation software (when the virtual laboratory vessel is tilted, the chemical reagent will pour along the side wall of the virtual laboratory vessel), and the updated chemical reagent posture images 42 and 43 are superimposed on the local images corresponding to the first test tube 11 and the second test tube 13 respectively.
[0090] like Figure 7 As shown, when the chemical reagent posture images corresponding to at least two chemical reagent information are detected to overlap, the pictures in the preset chemical reaction image 45 corresponding to the chemical reagents corresponding to the two chemical reagent information when they react are first superimposed on the local image of the corresponding virtual experimental vessel (the upper part of the beaker 12 where the chemical reaction occurs and the beaker mouth area). Among them, the preset chemical reaction image 45 can be in the form of a video, for example: the process of reagent luminescence, reagent color change, generation of fire, color explosion sound, etc. Then, the chemical reagent posture image 44 (deposited at the bottom of the beaker 12) is generated according to the multimedia content corresponding to the chemical reagent information after the preset reaction, so as to reflect the experimental results and the final product, so that the user can simulate the process and results of the chemical reaction both visually and auditorily.
[0091] In this embodiment, by using a physical display cabinet of the periodic table of elements that displays physical chemical elements, the present invention can provide users with an intuitive chemical process without the need to use real chemical reagents. In particular, it provides student users with an intuitive appearance of chemical elements, and can also safely train student users in chemical experiments, avoiding the hidden dangers caused by the irregular use of chemical reagents and improving safety.
[0092] Figures 8 to 13 Schematic diagram of the second implementation process of the virtual experiment method of the present invention. Figures 8 to 13 To introduce the second specific implementation process of the present invention ( Figures 9 to 13 , which is a schematic diagram of viewing through AR glasses 33):
[0093] like Figure 8 As shown, three empty physical experimental vessels are placed on the surface, namely, a first test tube 11, a beaker 12, and a second test tube 13. The user wears AR glasses 33 and activates the preset experimental project. The AR glasses 33 in this embodiment have functions such as image acquisition, image display, and audio playback.
[0094] like Fig. 9As shown, the user can directly see the first test tube 11, the beaker 12, and the second test tube 13 through the transparent lens of the AR glasses 33, and in the display screen of the visual enhancement device, independent non-overlapping suspension areas A, B, and C are generated according to the chemical reagent information of each preset experiment (the suspension areas A, B, and C can be blocks of different colors, which are displayed in the display area of the transparent lens). In addition, each chemical reagent information includes text description information about the chemical reagent and at least one of the mass, dosage, and molar amount of the chemical reagent, and the text description information in the chemical reagent information is displayed in the independent suspension area. At this time, the user can see from the AR glasses 33 that the suspension areas A, B, and C are suspended above the desktop and do not overlap with the physical experimental utensils, and there is corresponding text description information "XXXXXXXX" in the suspension area A, corresponding text description information "YYYY" in the suspension area B, and corresponding text description information "ZZZZZZZZZ" in the suspension area C. The specific text content is not repeated here.
[0095] like Fig.10 As shown, the current image collected by the visual enhancement device in real time is recognized through the existing neural network model to obtain the partial image of each physical laboratory vessel (first test tube 11, beaker 12, second test tube 13) in the current image. Pairing is performed according to the interaction action between the partial image of the physical laboratory vessel and the independent suspension area after it is lifted up. The interaction action is to lift the physical laboratory vessel so that the partial image corresponding to the physical laboratory vessel overlaps with the independent suspension area, and the overlapping time and / or the area of the overlapping area exceeds the preset threshold, then the physical laboratory vessel is paired with the chemical reagent information corresponding to the independent suspension area, but not limited to this. For example, when the user lifts the first test tube 11, so that the partial image corresponding to the first test tube 11 overlaps with the independent suspension area A by 1, and the overlapping area 41 is greater than 50% of the partial image area, the physical laboratory vessel is paired with the chemical reagent information corresponding to the independent suspension area. The beaker 12 and the second test tube 13 are paired in the same process, which will not be repeated here.
[0096] like Fig.11As shown, the posture of the physical laboratory vessel is detected, and the chemical reagent posture image is updated in real time according to the change of the pouring angle of the physical laboratory vessel at least. In the present invention, the chemical reagent posture image can be updated in real time according to the change of the pouring angle of the physical laboratory vessel through the existing physical simulation software, for example: the posture of the liquid (or solid particles, powder) accumulated or flowing in the physical laboratory vessel is adjusted according to the posture of the physical laboratory vessel. At this time, the first test tube 11 and the second test tube 13 are both configured with multimedia images corresponding to different chemical reagents (chemical reagent posture image 42 and chemical reagent posture image 43, respectively showing the image of two chemical reagents deposited at the bottom of the test tube, at this time, the user can see the first test tube 11 and the second test tube 13 through the transparent lens, and can also see the corresponding chemical reagent posture image 42 and chemical reagent posture image 43 from the transparent display area, thereby realizing the visual superposition state, the chemical reagent posture image 42 is superimposed on the first test tube 11, and the chemical reagent posture image 43 is superimposed on the second test tube 13, just like seeing the first test tube 11 and the second test tube 13 filled with real chemical reagents). Based on the corresponding multimedia content of the chemical reagent information and the local image of the physical laboratory vessel, a chemical reagent posture image matching the local image is generated. The chemical reagent posture image can be updated in real time according to the change of the pouring angle of the physical laboratory vessel through the existing physical simulation software (for example, under normal circumstances, the posture of the chemical reagent is mainly deposited at the bottom of the physical laboratory vessel).
[0097] like Fig.12 As shown, when the user pours the first test tube 11 and the second test tube 13 into the beaker 12 respectively according to the prompt of the preset experiment, a chemical reagent posture image matching the local image is generated again based on the corresponding multimedia content of the chemical reagent information and the local image of the physical laboratory vessel. The chemical reagent posture image can be updated in real time according to the change of the pouring angle of the physical laboratory vessel through the existing physical simulation software (when the physical laboratory vessel is tilted, the chemical reagent will pour along the side wall of the physical laboratory vessel), and the updated chemical reagent posture images 42 and 43 are superimposed on the corresponding local images of the first test tube 11 and the second test tube 13 respectively.
[0098] like Fig.13As shown, when the chemical reagent posture images corresponding to at least two chemical reagent information are detected to overlap, the pictures in the preset chemical reaction image 45 corresponding to the chemical reagents corresponding to the two chemical reagent information when they react are first superimposed on the local image of the corresponding physical experimental vessel (the upper part of the beaker 12 where the chemical reaction occurs and the beaker mouth area). Among them, the preset chemical reaction image 45 can be in the form of a video, for example: the process of reagent luminescence, reagent color change, generation of fire, color explosion sound, etc. Then, the chemical reagent posture image 44 (deposited at the bottom of the beaker 12) is generated according to the multimedia content corresponding to the chemical reagent information after the preset reaction, so as to reflect the experimental results and the final product, so that the user can truly simulate the process and results of the chemical reaction both visually and auditorily.
[0099] Fig.14 Schematic diagram of the structure of the virtual experiment system of the present invention. Fig.14 As shown, the embodiment of the present invention further provides a virtual experiment system for implementing the above virtual experiment method. The virtual experiment system 5 includes:
[0100] The suspension display module 51 generates independent suspension areas in the display screen of the visual enhancement device according to the chemical reagent information of the preset experiment.
[0101] The vessel recognition module 52 recognizes the current image collected in real time by the visual enhancement device to obtain a partial image of each vessel in the current image.
[0102] The vessel pairing module 53 pairs the vessel with the independent suspension area, and generates a chemical reagent posture image superimposed on a local image of the corresponding vessel based at least on the corresponding multimedia content of the chemical reagent information.
[0103] The reaction interaction module 55 detects that when chemical reagent posture images corresponding to at least two chemical reagent information overlap, it generates a chemical reagent posture image according to the multimedia content corresponding to the chemical reagent information after the preset reaction, and at least superimposes the picture in the chemical reagent posture image onto the partial image of the corresponding vessel.
[0104] In a preferred embodiment, the visual enhancement device is MR glasses, the vessel is a virtual vessel selected by an interactive device based on the MR glasses, and the independent floating area is a virtual periodic table generated based on the MR glasses, in which a corresponding floating mark area is generated according to the element area corresponding to the chemical reagent information of a preset experiment, but this is not limited to this.
[0105] In a preferred embodiment, it also includes a physical display cabinet of the periodic table, the visual enhancement device is MR glasses, the vessel is a virtual vessel selected by the interactive device based on the MR glasses, and the independent floating area is based on the MR glasses collecting pictures of the physical display cabinet of the periodic table, and after image recognition, the corresponding floating marking area is generated according to the cabinet corresponding to the chemical reagent information of the preset experiment, but it is not limited to this.
[0106] In a preferred embodiment, the visual enhancement device is AR glasses, the vessel is a physical vessel, and the independent floating area is a virtual area generated based on the AR glasses, but is not limited to this.
[0107] In a preferred embodiment, it also includes a physical display cabinet of the periodic table, in which various physical elements are stored in each drawer, the visual enhancement device is AR glasses, the utensils are physical utensils, and the independent floating area is based on the AR glasses collecting pictures of the physical display cabinet of the periodic table, and after image recognition, the corresponding floating marking area is generated according to the drawers corresponding to the chemical reagent information of the preset experiment, but it is not limited to this.
[0108] In a preferred embodiment, the suspension display module 51 is also configured to generate non-overlapping independent suspension areas in the display screen of the visual enhancement device according to the chemical reagent information of each preset experiment, each chemical reagent information includes text description information about the chemical reagent and at least one of the mass, dosage, and molar amount of the chemical reagent, and the text description information in the chemical reagent information is displayed in the independent suspension area, but not limited to this.
[0109] In a preferred embodiment, the vessel pairing module 53 is further configured to pair the partial image of the vessel according to the interaction between the partial image of the vessel and the independent suspension area after the partial image is lifted. A chemical reagent posture image matching the partial image is generated based on the corresponding multimedia content of the chemical reagent information and the partial image of the vessel. The chemical reagent posture image is superimposed on the partial image of the corresponding vessel, but is not limited thereto.
[0110] In a preferred embodiment, the vessel pairing module 53 is further configured to perform an interactive action of lifting the vessel so that the local image corresponding to the vessel overlaps with the independent suspension area, and if the overlapping time and / or the area of the overlapping area exceeds a preset threshold, the vessel is paired with the chemical reagent information corresponding to the independent suspension area, but this is not limited to this.
[0111] In a preferred embodiment, it further comprises: a posture adaptation module 54 for detecting the posture of the vessel and updating the chemical reagent posture image in real time at least according to the change of the tilting angle of the vessel, but the present invention is not limited thereto.
[0112] In a preferred embodiment, the reaction interaction module 55 is further configured to detect in real time whether the chemical reagent posture images corresponding to at least two chemical reagent information overlap in the local image of the same vessel. When overlap occurs, a chemical reagent posture image is generated according to the multimedia content corresponding to the preset post-reaction chemical reagent information corresponding to the combination of the two chemical reagent information. At least the picture in the chemical reagent posture image is superimposed on the local image of the corresponding vessel, but this is not limited to this.
[0113] In a preferred embodiment, the reaction interaction module 55 is further configured to superimpose a preset chemical reaction image when the combination of chemical reagent information reacts onto a partial image of the corresponding vessel, but the present invention is not limited thereto.
[0114] The virtual experiment system of the present invention can simulate all the operation steps of the experiment process and implement the corresponding visual effects for feedback, thus avoiding contact with chemical reagents and greatly improving the safety of the experiment.
[0115] The embodiment of the present invention further provides a virtual experiment device, comprising a processor and a memory, wherein executable instructions of the processor are stored, wherein the processor is configured to execute the steps of the virtual experiment method by executing the executable instructions.
[0116] As shown above, the virtual experiment equipment of this embodiment of the present invention can simulate all the operation steps of the experiment process and implement the corresponding visual effects of feedback, avoid contact with chemical reagents, and greatly improve the safety of the experiment.
[0117] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".
[0118] Fig.15 It is a schematic diagram of the structure of the virtual experiment equipment of the present invention. Fig.15 The electronic device 600 according to this embodiment of the present invention is described. Fig.15 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0119] like Fig.15 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.
[0120] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above method section of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 Follow the steps shown in .
[0121] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .
[0122] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0123] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
[0124] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.
[0125] The embodiment of the present invention also provides a computer-readable storage medium for storing a program, and the steps of the virtual experiment method are implemented when the program is executed. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary embodiments of the present invention described in the above method section of this specification.
[0126] As shown above, the virtual experiment system of the present invention in this embodiment can simulate all the operation steps of the experiment process and implement the corresponding visual effects for feedback, thus avoiding contact with chemical reagents and greatly improving the safety of the experiment.
[0127] Fig.16 Schematic diagram of the structure of the computer-readable storage medium of the present invention. Fig.16 As shown, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.
[0128] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.
[0129] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.
[0130] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).
[0131] In summary, the purpose of the present invention is to provide a virtual experiment method, system, equipment and storage medium, which can simulate the operation steps of the experimental process throughout and implement the corresponding visual effects of feedback, avoid contact with chemical reagents, and greatly improve the safety of the experiment.
[0132] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.
Claims
1. A virtual experiment method, characterized in that: The following steps are involved: S110, generating independent suspension areas respectively according to the chemical reagent information of the preset experiment in the display screen of the visual enhancement device; S120, identifying the current image collected in real time by the visual enhancement device to obtain a partial image of each vessel in the current image; S130, pairing the vessel with the independent suspension area, and generating a chemical reagent posture image based at least on the corresponding multimedia content of the chemical reagent information and superimposing it on a partial image corresponding to the vessel; S150, when detecting that chemical reagent posture images corresponding to at least two chemical reagent information are overlapped, generate a chemical reagent posture image according to multimedia content corresponding to the chemical reagent information after the preset reaction, and at least superimpose the picture in the chemical reagent posture image onto the partial image corresponding to the vessel.
2. The virtual experiment method according to claim 1, characterized in that: In step S110, the visual enhancement device is MR glasses, the vessel is a virtual vessel selected by an interactive device based on the MR glasses, and the independent floating area is a virtual periodic table generated based on the MR glasses, in which a corresponding floating mark area is generated according to an element area corresponding to the chemical reagent information of a preset experiment.
3. The virtual experiment method according to claim 1, characterized in that: The step S110 further includes a physical display cabinet of the periodic table, the visual enhancement device is MR glasses, the vessel is a virtual vessel selected by an interactive device based on the MR glasses, and the independent floating area is based on the MR glasses collecting a picture of the physical display cabinet of the periodic table, and after image recognition, a corresponding floating marking area is generated according to the cabinet corresponding to the chemical reagent information of the preset experiment.
4. The virtual experiment method according to claim 1, characterized in that: In step S110, the visual enhancement device is an AR glasses, the vessel is a physical vessel, and the independent floating area is a virtual area generated based on the AR glasses.
5. The virtual experiment method according to claim 1, characterized in that: The step S110 further includes a physical display cabinet of the periodic table, in which various elements are stored in each drawer. The visual enhancement device is an AR glasses, and the vessel is a physical vessel. The independent floating area is based on the AR glasses collecting a picture of the physical display cabinet of the periodic table, and after image recognition, a corresponding floating marking area is generated according to the drawers corresponding to the chemical reagent information of the preset experiment.
6. The virtual experiment method according to any one of claims 1 to 5, characterized in that: The step S110 includes: S111, generating non-overlapping independent suspension areas in a display screen of a visual enhancement device according to chemical reagent information of each preset experiment, wherein each of the chemical reagent information includes textual description information about the chemical reagent and at least one of the mass, dosage, and molar amount of the chemical reagent; S112: Displaying text description information in the chemical reagent information in the independent suspension area.
7. The virtual experiment method according to claim 6, characterized in that: The step S130 includes: S131, pairing according to the interaction action between the partial image of the vessel and the independent floating area after the partial image of the vessel is lifted; S132, generating a chemical reagent posture image matching the partial image based on the corresponding multimedia content of the chemical reagent information and the partial image of the vessel; S133, superimposing the chemical reagent posture image onto the local image corresponding to the vessel.
8. The virtual experiment method according to claim 7, characterized in that: In step S131, the interactive action is to lift the vessel so that the local image corresponding to the vessel overlaps with the independent floating area, and if the overlapping time and / or the area of the overlapping area exceeds a preset threshold, the chemical reagent information corresponding to the vessel and the independent floating area is paired.
9. The virtual experiment method according to claim 6, characterized in that: After step S130 and before step S150, the following steps are further included: S140, detecting the posture of the container, and updating the chemical reagent posture image in real time at least according to the change of the tilting angle of the container.
10. The virtual experiment method according to claim 9, characterized in that: The step S150 includes: S151, detecting in real time whether chemical reagent posture images corresponding to at least two types of chemical reagent information overlap in the local image of the same vessel; S152, when overlap occurs, generating a chemical reagent posture image according to the preset post-reaction chemical reagent information corresponding to the combination of the two chemical reagent information and the corresponding multimedia content; S153, at least superimpose the image in the chemical reagent posture image onto the partial image corresponding to the vessel.
11. The virtual experiment method according to claim 10, characterized in that: The step S151 further includes: superimposing a preset chemical reaction image when the combination of the chemical reagent information reacts onto a local image corresponding to the vessel.
12. A virtual experiment system, used to implement the virtual experiment method according to claim 1, characterized in that: include: A suspension display module generates independent suspension areas in the display screen of the visual enhancement device according to the chemical reagent information of the preset experiment; A vessel recognition module recognizes the current image collected in real time by the visual enhancement device to obtain a partial image of each vessel in the current image; A vessel pairing module pairs the vessel with the independent suspension area, and generates a chemical reagent posture image superimposed on a local image corresponding to the vessel based at least on the corresponding multimedia content of the chemical reagent information; The reaction interaction module detects that when chemical reagent posture images corresponding to at least two chemical reagent information overlap, a chemical reagent posture image is generated according to multimedia content corresponding to the chemical reagent information after the preset reaction, and at least the picture in the chemical reagent posture image is superimposed on the partial image corresponding to the vessel.
13. A virtual experiment device, characterized in that: include: processor; a memory storing executable instructions of the processor; Wherein, the processor is configured to perform the steps of the virtual experiment method according to any one of claims 1 to 7 by executing the executable instructions.
14. A computer-readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps of the virtual experiment method according to any one of claims 1 to 7 are implemented.