A three-dimensional target detection and data processing system based on a meta universe

Abstract

The application discloses a three-dimensional target detection and data processing system based on a meta universe, reduces the number of detection hardware devices, saves a large amount of resources, reduces CPU load, and can keep the simulated data in a virtual environment consistent with the data of human eyes and three-dimensional targets in reality. The technical scheme is as follows: the system comprises a three-dimensional detection and transmission module, which acquires information of a three-dimensional object to be identified and transmits the information of the three-dimensional object to a three-dimensional target identification module; the three-dimensional target identification module acquires an optical focal point of an eyeball at a current time; a three-dimensional detection and calculation module performs identification detection based on the information of the three-dimensional object and the eyeball data; and a data recording module stores the result of the three-dimensional detection and calculation module.

Description

Technical Field

[0001] This invention relates to a three-dimensional target detection technology, specifically to a three-dimensional target detection and data processing system based on a metaverse. Background Technology

[0002] The metaverse is a virtual world constructed by humans using digital technology. It is a virtual world that maps to or transcends the real world, can interact with the real world, and is a digital living space with a new social system.

[0003] Virtual Reality (VR) technology is an important direction in simulation technology. It is a combination of simulation technology with computer graphics, human-computer interface technology, multimedia technology, sensing technology, network technology, and other technologies. It is a challenging interdisciplinary frontier discipline and research field. Virtual Reality technology mainly includes simulated environments, perception, natural skills, and sensing devices. The simulated environment consists of computer-generated, real-time, dynamic, three-dimensional, realistic images. Perception refers to the ideal VR system possessing all human senses. In addition to visual perception generated by computer graphics, it includes auditory, tactile, force, and motion perception, and even olfactory and gustatory senses, also known as multi-sensory perception. Natural skills refer to head movements, eye movements, gestures, or other bodily actions. The computer processes data corresponding to the participant's actions and responds to user input in real time, providing feedback to the user's five senses. Sensing devices refer to three-dimensional interactive devices.

[0004] Current methods for detecting 3D targets rely on a large number of detection hardware devices and algorithms. However, existing 3D target detection technologies suffer from the following drawbacks:

[0005] 1) Existing 3D target detection not only requires a large number of algorithm support, but also a lot of detection hardware equipment, which is costly;

[0006] 2) Existing 3D object detection methods calculate the data between the detection hardware and the 3D object, rather than the data between the human eye and the 3D object;

[0007] 3) Existing 3D target detection methods directly detect 3D targets after they are enabled, and continuously detect them, which puts a high load on the CPU. Summary of the Invention

[0008] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not an exhaustive summary of all conceived aspects, nor is it intended to identify key or decisive elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form to prepare for the more detailed descriptions that follow.

[0009] The purpose of this invention is to solve the above-mentioned problems and provide a 3D target detection and data processing system based on the metaverse, which reduces the number of detection hardware devices, saves a lot of resources, reduces CPU load, and can keep the data simulated in the virtual environment consistent with the data of the human eye and 3D targets in reality.

[0010] The technical solution of this invention is as follows: This invention discloses a three-dimensional target detection and data processing system based on the metaverse, the system comprising:

[0011] The 3D detection and transmission module acquires information about the 3D object to be identified and sends the information about the 3D object to the 3D target recognition module.

[0012] The 3D target recognition module obtains the optical focus of the eye at the current moment;

[0013] The 3D detection and calculation module performs recognition and detection based on information about 3D objects and eye data.

[0014] The data recording module stores the results of the 3D detection and calculation module.

[0015] According to an embodiment of the metaverse-based three-dimensional target detection and data processing system of the present invention, the three-dimensional target recognition module obtains the optical focus of the eye at the current moment by capturing the user's eye information.

[0016] According to an embodiment of the metaverse-based 3D target detection and data processing system of the present invention, the 3D detection calculation module is further configured to: firstly convert the vertex data of the 3D model into vertex attributes, then process the vertex attributes into pixel data, then identify the pixel data through a neural network, and finally calculate the time taken for the eye to recognize the 3D object.

[0017] According to an embodiment of the metaverse-based 3D target detection and data processing system of the present invention, vertex data includes coordinates, texture coordinates, normals, and colors.

[0018] According to an embodiment of the metaverse-based 3D target detection and data processing system of the present invention, vertex data is converted into vertex attributes by a vertex shader, and vertex attributes are converted into pixel data by a fragment shader.

[0019] According to one embodiment of the metaverse-based 3D target detection and data processing system of the present invention, the neural network is a Mask R-CNN neural network.

[0020] According to an embodiment of the metaverse-based 3D target detection and data processing system of the present invention, the results stored by the data recording module include the time taken for the eye to recognize the 3D object, the recognized 3D object, and user information.

[0021] Compared with existing technologies, this invention offers the following advantages: When applied to a virtual scene, the 3D detection transmitter is generated through a virtual algorithm, eliminating the need for a large amount of actual detection hardware; only a device for entering the metaverse is required, saving significant resources. The 3D detection transmitter of this invention corresponds to the position of the human eye, and the data simulated in the virtual environment is consistent with the data of the human eye and 3D targets in reality. The 3D detection transmitter of this invention has an event trigger that can control its activation and deactivation. Attached Figure Description

[0022] The above-described features and advantages of the present invention will be better understood after reading the following detailed description of embodiments of the present disclosure in conjunction with the accompanying drawings. In the drawings, components are not necessarily drawn to scale, and components having similar related characteristics or features may have the same or similar reference numerals.

[0023] Figure 1 A schematic diagram of an embodiment of the metaverse-based 3D target detection and data processing system of the present invention is shown.

[0024] Figure 2 It shows Figure 1 The flowchart of the system embodiment shown is as follows.

[0025] Figure 3 and Figure 4 A schematic diagram of an actual teaching example is shown. Detailed Implementation

[0026] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should be noted that the aspects described below with reference to the accompanying drawings and specific embodiments are merely exemplary and should not be construed as limiting the scope of protection of the present invention in any way.

[0027] Figure 1 This illustrates the principle of an embodiment of the metaverse-based 3D target detection and data processing system of the present invention. Please refer to [link / reference]. Figure 1 The system in this embodiment includes: a three-dimensional detection and emission module, a three-dimensional target recognition module, a three-dimensional detection and calculation module, a data recording module, and an automated detection and recognition module.

[0028] The 3D detection and emission module acquires information about the 3D object to be identified, then starts the 3D target recognition module, which acquires the optical focus of the eye at the current moment. Then, the 3D detection and calculation module is started to perform recognition and detection algorithms on the information of the object and the data of the eye. The results are recorded in the data recording module.

[0029] The 3D detection and transmission module is configured to allow users to select the objects to be identified, and then send the 3D object information to the 3D target recognition module.

[0030] The 3D target recognition module is configured to capture the user's eye information and obtain the optical focus of the eye at the current moment.

[0031] The 3D detection calculation module is further configured as follows: the 3D detection emission module acquires information about the 3D object to be detected and the eye data of the 3D target recognition module for calculation. First, the vertex data of the 3D model (vertex data includes coordinates, texture coordinates, normals, and colors) is converted into vertex attributes through the vertex shader. Then, the vertex attributes are processed into pixel data through the fragment shader. Then, the pixel data is recognized through the Mask R-CNN neural network. Finally, the time taken for the eye to recognize the 3D object is calculated.

[0032] The data recording module is configured to record the time of the 3D detection calculation module, the identified 3D objects, and user information into the database.

[0033] The system's operating process is as follows: Figure 2 As shown.

[0034] A three-dimensional detection transmitter is set at the eyes of the virtual human in the metaverse (i.e. Figure 1 The 3D detection and emission module in the metaverse scene starts the 3D detection and emission module when the event trigger is turned on. Each 3D simulation model in the metaverse scene has its own identifier. The CPU continuously calculates to identify each identifier of the 3D simulation model, records the time point of each identification of a certain identifier, and records the time point in the database. The 3D detection and emission module is turned off when the event trigger is turned off.

[0035] In practical teaching applications, the system starts timing from the moment an emergency occurs until the user's line of sight detects the person experiencing the emergency. It calculates the time to assess the user's ability to identify risks and handle problems in emergency situations, and generates relevant data reports based on this, forming a user profile and improving the overall business level of the civil aviation industry.

[0036] First, upon entering the VR training course, the instructor selects "Lithium Battery Fire Handling" without informing the trainees.

[0037] Then, an anomaly occurred on the student's end: a laptop bag caught fire in one of the overhead bins on the plane, and passengers began to rummage through the bins to check their laptop bags.

[0038] Next, as Figure 3 As shown, the student closest to the point where the anomaly occurred must accurately observe the location of the anomaly within 2 seconds.

[0039] Finally, as Figure 4As shown, the trainee quickly stepped forward to stop the incident and completed all operations according to the Standard Operating Procedure (SOP).

[0040] When the instructor selects "Lithium Battery Fire Handling", the event trigger controls the 3D detection transmitter to turn on. The 3D detection transmitter simulates the data of the human eye and the 3D target in the virtual environment. When the simulation reaches the point where the human eye recognizes the 3D target, the timer ends, and the event trigger controls the 3D detection transmitter to turn off. The time of recognizing the 3D target is recorded in the database.

[0041] Although the methods described above are illustrated and depicted as a series of actions for the sake of simplicity, it should be understood and appreciated that these methods are not limited by the order of the actions, as some actions may occur in a different order and / or concurrently with other actions from the illustrations and descriptions herein or not illustrated and described herein but which may be understood by those skilled in the art, according to one or more embodiments.

[0042] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the invention.

[0043] The various illustrative logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein can be implemented or performed using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but in alternatives, it may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration.

[0044] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of both. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor such that the processor can read and write information to / from the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside as discrete components in the user terminal.

[0045] In one or more exemplary embodiments, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functionality may be stored or transmitted as one or more instructions or code on or through a computer-readable medium. A computer-readable medium includes both computer storage media and communication media, encompassing any medium that facilitates the transfer of a computer program from one location to another. A storage medium may be any available medium accessible to a computer. By way of example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and is accessible to a computer. Any connection is also legitimately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of a medium. As used in this article, disk and disc include compact discs (CDs), laser discs, optical discs, digital multi-purpose discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of these should also be included within the scope of computer-readable media.

[0046] The prior description of this disclosure is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A three-dimensional target detection and data processing system based on the metaverse, characterized in that, The system includes: The 3D detection and transmission module acquires information about the 3D object to be identified and sends the information about the 3D object to the 3D target recognition module. The 3D target recognition module obtains the optical focus of the user's eye at the current moment by capturing the user's eye information; The 3D detection and calculation module performs recognition and detection based on information about 3D objects and eye data. The data recording module stores the results of the 3D detection and calculation module; The 3D detection and calculation module is further configured as follows: first, the vertex data of the 3D model is converted into vertex attributes; then, the vertex attributes are processed into pixel data; then, the pixel data is recognized through a neural network; and finally, the time taken for the eye to recognize the 3D object is calculated.

2. The 3D target detection and data processing system based on the metaverse according to claim 1, characterized in that, Vertex data includes coordinates, texture coordinates, normals, and colors.

3. The 3D target detection and data processing system based on the metaverse according to claim 1, characterized in that, Vertex data is converted into vertex attributes by the vertex shader, and vertex attributes are converted into pixel data by the fragment shader.

4. The 3D target detection and data processing system based on the metaverse according to claim 1, characterized in that, The neural network is the Mask R-CNN neural network.

5. The 3D target detection and data processing system based on the metaverse according to claim 1, characterized in that, The data recording module stores results including the time it takes for the eye to recognize a 3D object, the recognized 3D object, and user information.

Images