Biological comprehensive recognition system based on mixed reality technology

Abstract

The invention discloses a biological comprehensive identification system based on a mixed reality technology, and belongs to the cross technical field of information technology, education technology and biological research. Comprising a biodiversity recognition module, a biological life cycle simulation module, a biological anatomical structure three-dimensional interaction module, MR equipment used for capturing biological images and an image recognition algorithm based on deep learning, and the output end of the biodiversity recognition module is connected with the input end of the biological life cycle simulation module; according to the system, the advantages of the mixed reality technology are utilized, deep learning and a three-dimensional modeling algorithm are combined, biological recognition, life cycle simulation and three-dimensional display are integrated, comprehensive integration of biological recognition, life cycle display and three-dimensional model display is achieved, and immersive interaction experience is provided for a user.

Description

Technical Field

[0001] The present invention belongs to the interdisciplinary field of information technology, educational technology and biological research, and in particular relates to a biological comprehensive recognition system based on mixed reality technology. Background Art

[0002] Biometrics is a high-tech authentication method that combines computer technology, optics, acoustics, biosensors, and biostatistics. It uses the body's inherent physiological characteristics (such as fingerprints, irises, and facial features) and behavioral traits (such as voice, gait, and signature) to verify personal identity. The development of biometrics has evolved from manual visual recognition and specific measurement tools to the application of digital biometrics, and then to the development of multi-biometric fusion and intelligent biometrics. With the continuous advancement of artificial intelligence and deep learning technologies, the accuracy and security of biometrics have been significantly improved.

[0003] Traditional biometric recognition technology, limited by equipment and application scenarios, struggles to provide comprehensive and accurate biological information. Furthermore, in biology education and scientific research, the presentation of biological life cycles and the study of anatomical structures often rely on static images or two-dimensional models, lacking intuitiveness and interactivity. To address these issues, a comprehensive biometric recognition system based on mixed reality technology is urgently needed. Summary of the Invention

[0004] The purpose of this invention is to propose a comprehensive biological recognition system based on mixed reality technology in order to solve the problems that traditional biometric technology is limited by equipment and application scenarios and is difficult to provide comprehensive and accurate biological information; in the process of biological education and scientific research, the display of biological life cycle and the study of anatomical structure often rely on static images or two-dimensional models, which lack intuitiveness and interactivity.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a comprehensive biological identification system based on mixed reality technology, including a biodiversity identification module, a biological life cycle simulation module, a biological anatomical structure three-dimensional interactive module, an MR device for capturing biological images, and an image recognition algorithm based on deep learning. The output end of the biodiversity identification module is connected to the input end of the biological life cycle simulation module, and the output end of the biological life cycle simulation module is connected to the input end of the biological anatomical structure three-dimensional interactive module.

[0006] As a further description of the above technical solution: The MR device for capturing biological images and the image recognition algorithm based on deep learning are used to achieve high-precision and fast biometric recognition functions.

[0007] As a further description of the above technical solution: The biological life cycle simulation module adopts a machine learning algorithm to dynamically generate a growth model according to the biological species and growth conditions, thereby realizing dynamic simulation and display of the biological life cycle.

[0008] As a further description of the above technical solution: The biological anatomical structure three-dimensional interactive module supports users to perform interactive operations such as rotating, scaling, and cutting three-dimensional models through mixed reality glasses, providing an intuitive and vivid immersive experience.

[0009] The present invention also discloses a biological comprehensive recognition method based on mixed reality technology, comprising the following steps: S1. Conduct high-precision identification of biodiversity; S2. Simplify the biological life cycle; S3. Conduct professional analysis and expansion of biodiversity; S4. Conduct dynamic simulation and experiments on biological life cycles; S5. Conduct three-dimensional interactive display of biological anatomical structures; S6. Integrate the system into a virtual reality platform.

[0010] As a further description of the above technical solution: In S1, high-precision identification of biodiversity is performed, and the specific steps are: data collection and labeling, model training and optimization, and system integration and development.

[0011] As a further description of the above technical solution: In said S2, a simplified display of the biological life cycle is performed, and the specific steps are: data collection and animation production, system integration and interaction design.

[0012] As a further description of the above technical solution: In S3, professional analysis and expansion of biodiversity are carried out, with the specific steps of: data expansion and detailed annotation, model upgrade and training, system integration and information display.

[0013] As a further description of the above technical solution: In the above-mentioned S4, dynamic simulation and experiment of biological life cycle are carried out, and the specific steps are: data collection and model training, system integration and virtual experiment design.

[0014] As a further description of the above technical solution: In said S5, a three-dimensional interactive display of biological anatomical structures is performed, and the specific steps are: three-dimensional modeling and optimization, system integration and interactive design. In said S6, the system is integrated into the virtual reality platform, and the relevant information and data in the system are imported into the MR glasses. When the user uses the system, he only needs to log in with his identity information to directly enter the system. When the user's glasses are aimed at the organism to be identified, the relevant information and video of the organism will all appear in the wearer's glasses. At this time, the wearer can quickly understand the relevant information of the organism and can also interact with the organism in real time in an immersive manner. At the same time, some new information can also be transmitted to the system of the MR glasses through the data cable to realize regular updating of information.

[0015] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: In the present invention, the system utilizes the advantages of mixed reality technology, combines deep learning and three-dimensional modeling algorithms, and integrates biometrics, life cycle simulation, and three-dimensional display into one, realizing the full integration of biometrics, life cycle display and three-dimensional model display, improving the comprehensive application effect of the system, and providing users with an immersive biological learning, identification and scientific research platform. The system can also be associated with real-life MR glasses, and relevant data and identification technology in the system can be transmitted to the MR glasses. Users can use MR glasses to identify biodiversity and share information, further improving the practical application effect of the system. It combines virtual reality (MR) and augmented reality (AR) technologies, and interacts with the real world in real time through virtual objects generated in MR glasses. It can superimpose virtual elements on the real environment and make these virtual objects have the physical properties of the real world. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Schematic diagram of the module structure of the biometric integrated recognition system based on mixed reality technology.

[0017] Figure 2 Schematic diagram of the process of comprehensive biological recognition method based on mixed reality technology. DETAILED DESCRIPTION

[0018] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0019] See also Figure 1-Figure 2The present invention provides a technical solution: a comprehensive biological identification system based on mixed reality technology, including a biodiversity identification module, a biological life cycle simulation module, a biological anatomical structure three-dimensional interactive module, an MR device for capturing biological images, and an image recognition algorithm based on deep learning. The output end of the biodiversity identification module is connected to the input end of the biological life cycle simulation module, and the output end of the biological life cycle simulation module is connected to the input end of the biological anatomical structure three-dimensional interactive module. The MR device for capturing biological images and the image recognition algorithm based on deep learning are used to achieve high-precision and rapid biological identification functions. The biological life cycle simulation module uses a machine learning algorithm to dynamically generate a growth model based on the biological species and growth conditions to achieve dynamic simulation and display of the biological life cycle. The biological anatomical structure three-dimensional interactive module supports users to perform interactive operations such as rotating, scaling, and cutting three-dimensional models through mixed reality glasses, providing an intuitive and vivid immersive experience.

[0020] The present invention also provides a biological comprehensive recognition method based on mixed reality technology, comprising the following steps: S1. Conduct high-precision identification of biodiversity. The specific steps are: data collection and annotation, model training and optimization, and system integration and development; S2. Simplify the biological life cycle. The specific steps are: data collection and animation production, system integration and interaction design; S3. Conduct professional biodiversity analysis and expansion, including: data expansion and detailed annotation, model upgrade and training, system integration and information display; S4. Conduct dynamic simulation and experiments on biological life cycles. The specific steps are: data collection and model training, system integration and virtual experiment design; S5. Conduct three-dimensional interactive display of biological anatomical structures. The specific steps are: three-dimensional modeling and optimization, system integration and interactive design; S6. Integrate the system into a virtual reality platform.

[0021] In this embodiment, the system takes advantage of mixed reality technology and combines it with deep learning and three-dimensional modeling algorithms to integrate biometrics, life cycle simulation, and three-dimensional display. It achieves a full integration of biometrics, life cycle display, and three-dimensional model display, improves the comprehensive application effect of the system, and can provide users with an immersive biological learning, identification, and scientific research platform.

[0022] It should be noted that: In S1, high-precision identification of biodiversity is carried out. The specific steps are as follows: data collection and annotation, specifically, extensive collection of biological image data covering common animals and plants, bacterial communities, mushroom species, etc., to ensure the comprehensiveness and representativeness of the data set; detailed annotation of image data, including key information such as biological species, basic characteristics, ecological habits, safety tips, etc.; model training and optimization, specifically, using advanced deep learning technologies, such as convolutional neural networks (CNN), to train the annotated image data, and through iterative optimization of algorithms and model parameters, improve the recognition accuracy and generalization ability of the model; system integration and development, specifically, integrating MR equipment with autofocus, optical image stabilization and other functions to ensure image capture quality; pre-processing of captured images, such as denoising, contrast enhancement, color correction, etc., to improve recognition effect; integrating the trained deep learning model into the system to achieve fast and accurate biometric recognition; designing a simple and intuitive information display interface, and providing biological information cards, including name, characteristics, habits, safety tips, etc.

[0023] In S2, a simplified display of the biological life cycle is performed. The specific steps are: data collection and animation production, specifically, collecting growth data of common horticultural plants and domestic pets from infancy to old age, including morphological changes, growth rate, etc. Based on the collected data, a series of preset biological life cycle animations are produced to display the growth process of the organisms. System integration and interaction design are carried out, specifically, the produced animations are integrated into the system, which users can watch through mixed reality glasses, and a simple and easy-to-use interactive interface is designed to support gestures or voice commands to select biological life cycle animations.

[0024] In S3, professional analysis and expansion of biodiversity are carried out. The specific steps are as follows: data expansion and detailed annotation. Specifically, on the basis of the basic version, further image data of professional biological categories (such as microorganisms and marine organisms) are collected, and the data are annotated in detail, including deep information such as ecological distribution, genetic information, and protection status. Model upgrades and training are carried out. Specifically, more complex deep learning models, such as residual networks (ResNet) and Inception, are used to train the annotated data. Technologies such as transfer learning are used to improve the model's recognition ability and generalization performance in professional biological categories. System integration and information display are carried out. Specifically, the trained model is integrated into the system to achieve rapid identification of professional biological categories and provide detailed biological information reports, including comprehensive information such as ecological distribution, genetic characteristics, and protection status.

[0025] In S4, dynamic simulation and experiments of biological life cycles are carried out. The specific steps are as follows: data collection and model training, specifically, collecting growth data of various organisms, covering growth rate, morphological changes, environmental adaptability and other aspects; using machine learning algorithms, such as recurrent neural networks (RNN) or long short-term memory networks (LSTM), to train biological growth models; and conducting system integration and virtual experiment design, specifically, integrating the trained biological growth model into the system to realize dynamic simulation of biological life cycles and designing a virtual experimental platform. Users can customize biological species and growth conditions. The system dynamically displays the biological growth process according to the model, and supports users to adjust parameters such as observation angle and growth rate through gestures or voice commands to realize interaction with simulated organisms.

[0026] In S5, a three-dimensional interactive display of biological anatomical structures is carried out. The specific steps are as follows: 3D modeling and optimization, specifically, using high-precision 3D modeling technology to build a biological anatomical structure model, covering all levels from microscopic cells to macroscopic organisms, and performing detail optimization and texture processing on the model to ensure the accuracy and realism of the model. At the same time, considering the interactivity and operability of the model, system integration and interactive design are carried out. Specifically, the optimized 3D model is integrated into the system, and users can use mixed reality glasses to perform interactive operations such as rotating, scaling, and cutting the 3D model. An intuitive interactive interface and operation process are designed to improve the user experience. In S6, the system is integrated into the virtual reality platform, and the relevant information and data in the system are imported into the MR glasses. When using the system, users only need to log in with their identity information to enter the system directly. When the user's glasses are aimed at the creature that needs to be identified, the relevant information and video of the creature will all appear in the wearer's glasses. At this time, the wearer can quickly understand the relevant information of the creature and can also interact with the creature in real time in an immersive way. At the same time, some new information can also be transmitted to the MR glasses system through the data cable to realize regular updates of information.

[0027] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A comprehensive organism recognition system based on mixed reality technology, comprising a biodiversity recognition module, an organism life cycle simulation module, a three-dimensional interaction module for organism anatomical structures, an MR device for capturing organism images, and a deep learning-based image recognition algorithm. The output of the biodiversity recognition module is connected to the input of the organism life cycle simulation module, and the output of the organism life cycle simulation module is connected to the input of the three-dimensional interaction module for organism anatomical structures.

2. The biometric integrated recognition system based on mixed reality technology according to claim 1, characterized in that: The MR device for capturing biological images and the image recognition algorithm based on deep learning are used to achieve high-precision and fast biometric recognition functions.

3. The biometric comprehensive recognition system based on mixed reality technology according to claim 2, characterized in that: The biological life cycle simulation module adopts a machine learning algorithm to dynamically generate a growth model according to the biological species and growth conditions, thereby realizing dynamic simulation and display of the biological life cycle.

4. The biometric integrated recognition system based on mixed reality technology according to claim 3, characterized in that: The biological anatomical structure three-dimensional interactive module supports users to perform interactive operations such as rotating, scaling, and cutting three-dimensional models through mixed reality glasses, providing an intuitive and vivid immersive experience.

5. A method for identifying biological integrated systems based on mixed reality technology, characterized in that: The steps include: S1. Perform model training; S2, conduct system development and model integration; S3, import MR equipment; S4. Conduct high-precision identification of biodiversity; S5, artificial selection of target species; S6. The system dynamically displays the life cycle of the species; S7, manually select the target organ at the target time point; S8. Systematically display the internal and external 3D structure of organs.

6. The method for comprehensive biological identification based on mixed reality technology according to claim 5, characterized in that: In said S1, model training is performed, including biological species identification model training, biological growth and development dynamic model training, and biological morphology and anatomical structure three-dimensional model training.

7. The method for comprehensive biological identification based on mixed reality technology according to claim 6, characterized in that: In the S1, biological species recognition model training is carried out, and the specific steps are: collecting biological pictures, organizing and annotating the pictures, specifically: organizing and classifying the collected pictures according to the biological taxonomy system, including the animal kingdom, the plant kingdom, and the fungal kingdom, adding detailed annotation information to each picture, including the name of the organism, classification status, shooting location, and time metadata, combining the organized pictures and annotation information into a training data set for training the deep learning model, and collecting and organizing biological sound data, specifically: organizing and classifying the collected sound data according to the biological taxonomy system, and adding detailed annotation information to each sound segment, including the name of the organism, classification status, recording location, and time metadata, and training the deep learning model, wherein the model selection selects a deep learning model architecture suitable for biological recognition, using convolutional neural network (CNN) for image processing and recurrent neural network ( RNN) or its variant (long short-term memory network LSTM) is used for sound processing, or a multimodal fusion model is used to simultaneously process image and sound data to display species information. A user-friendly interface is designed to display species information in the form of text, pictures, videos, three-dimensional models or sound playback. Interactive functions are provided to allow users to further explore information about related species based on the recognition results (images or sounds), including viewing more photos, listening to more sounds, and reading detailed descriptions. This assists in modeling and optimization. When photo, image or sound data is insufficient or of low quality, modeling technology is used to generate or enhance biometric data. 3D modeling technology is used to create a three-dimensional model of the organism, or sound synthesis technology is used to generate the call of the organism. The modeling results and actual data (photos, images, sounds) are combined to further optimize the deep learning model to improve recognition accuracy and robustness, including adjusting the model architecture and optimizing hyperparameters.

8. The method for comprehensive biological identification based on mixed reality technology according to claim 6, characterized in that: In the said S1, the dynamic model training of biological growth and development is carried out, and the specific steps are: collecting biological photos and videos, specifically: collecting actual photos and video materials covering the entire process of biological life from birth to death, for building a life cycle simulation model, the photos and videos contain various growth stages of the organism, including infancy, adulthood, old age, and key life events, including reproduction, migration, and molting, and data collation and sequence construction, specifically: arranging and classifying the collected photos and videos according to the species and growth stage of the organism, arranging the photos and videos in chronological order according to the life cycle characteristics of the organism, constructing a complete sequence of the biological life cycle, and constructing a life cycle simulation model, specifically: using animation production software, timeline Editing tools or specialized biological simulation software are used to construct life cycle simulation models, and organized photos and video sequences are imported into the model. By adjusting the timeline and adding transition effects, the life cycle process of the organism is simulated. A user interaction interface is designed to allow users to select different species of organisms and watch the simulation process of their life cycle, so as to display and educate about the life cycle. Specifically, the life cycle process of the organism can be intuitively displayed through animation, video or interactive interface. In combination with biological knowledge, explanations, annotations or interactive questions and answers can be added to the life cycle simulation to enhance the educational effect and assist in modeling and optimization. Specifically, when photo and video materials are insufficient or of low quality, three-dimensional modeling technology can be used to supplement or enhance certain stages or events in the life cycle.

9. The method for comprehensive biological identification based on mixed reality technology according to claim 6, characterized in that: In the above-mentioned S1, the three-dimensional model training of the biological morphology and anatomical structure is carried out, and the specific steps are as follows: constructing a three-dimensional model, specifically: collecting high-definition photos, CT scan data or MRI images of the organism for constructing a three-dimensional anatomical structure model, using three-dimensional modeling software or professional biomedical modeling tools to construct the three-dimensional anatomical structure model of the organism, processing the model in detail, including adding texture, adjusting lighting, and setting transparency to enhance the realism and visualization effect of the model, and designing interactive functions, specifically: designing a user interaction interface to allow the user to rotate and scale the three-dimensional model using a mouse or touch screen to observe the anatomical structure of the organism from different angles, providing disassembly and assembly functions to allow the user to disassemble the three-dimensional model into various parts or assemble it in a specific order to understand the internal structure and organ relationship of the organism, adding annotations and explanatory information to the three-dimensional model, including organ names and function introductions, to help users better understand the anatomical structure of the organism, and updating and optimizing the model, specifically: continuously updating the data source and modeling technology of the three-dimensional anatomical structure model with the emergence of new biological research results and technologies, optimizing the performance of the three-dimensional interactive module, and improving the loading speed, rendering effect and interactive response speed of the model.

10. The biometric comprehensive identification method based on mixed reality technology according to claim 5, characterized in that: In S3, an MR device is imported, wherein the MR device is selected to have a high-resolution display, low-latency tracking, a high refresh rate, and a wide field of view. The selected device is one of HoloLens, Magic Leap, or other compatible devices. The specific steps are as follows: integrating data from the biodiversity identification module, the biological life cycle simulation module, and the biological anatomical structure three-dimensional interactive module to form a unified biological database, ensuring data format compatibility to facilitate seamless switching and display of content from different modules in the MR environment; developing a dedicated MR application as an interface for users to interact with each module; connecting the MR device and the interactive device to a computer or mobile device to ensure unimpeded data transmission and communication; and conducting comprehensive testing of the integrated system, including functional testing, performance testing, and user experience testing. In S5-S8, the species identification module identifies the biological species appearing in the MR device. The user selects the corresponding species, and the system outputs its detailed information. Then, the user can choose to enter the life cycle observation module to view the life cycle simulation process of the organism. Finally, the user can also choose to enter the anatomical structure observation module for a specific organ in a specific life cycle to observe and understand the internal structure and organ relationships of the specific organ of the organism at different life cycle stages through the three-dimensional interactive function.

Images