Object recognition wearable device based on virtual reality

Abstract

The invention discloses an object recognition wearable device based on virtual reality, which comprises a head-mounted shell, a video recording device, a mixed display device and a picture analysis processing module, and is characterized in that the mixed display device is arranged on a helmet-mounted display, and the video recording device and the picture analysis processing module are both arranged in the head-mounted shell; the video recording device can record video pictures in front of human eyes and transmit the pictures to the picture analyzing and processing module, and the mixed display device is of a transparent display screen structure. When the system is implemented, the image of the target area is acquired, and by means of the comparison module and the analysis module, whether equipment (parts) in the target area needs to be maintained or a maintenance mode method can be displayed, so that whether the equipment needs to be maintained and maintenance operation are guided in a standardized manner, and the maintenance efficiency is improved. The defect that in the prior art, operation is carried out mainly depending on the experience of maintenance personnel is overcome.

Description

Technical Field

[0001] The present application relates to the technical field of VR wearable devices, and in particular to a wearable device capable of object recognition based on virtual reality. Background Art

[0002] As an emerging technology, wearable VR devices have broad application space, especially in the industrial field.

[0003] In the existing industrial field, when performing maintenance and inspection on equipment, except for some obvious locations that need repair, other locations are generally judged based on the experience of maintenance personnel, which is highly subjective. Moreover, when judging that a certain location needs repair or maintenance, it is difficult for maintenance personnel with less experience to make a judgment and perform maintenance on the spot, which may waste maintenance time and cause irreversible damage to the equipment.

[0004] To this end, the present application proposes a wearable device capable of object recognition based on virtual reality. Summary of the invention

[0005] To this end, the present application provides an object recognition wearable device based on virtual reality to solve the problems in the prior art that the subjective judgment has large errors and it is difficult to make timely and correct responses.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] In a first aspect, a wearable device capable of object recognition based on virtual reality includes a head-mounted housing, a video recording device, a hybrid display device, and a picture analysis and processing module, wherein the hybrid display device is arranged on a helmet-type display, and the video recording device and the picture analysis and processing module are both arranged in the head-mounted housing;

[0008] The video recording device can record the video images in front of the human eye and transmit the images to the image analysis and processing module. The hybrid display device is a transparent display screen structure, and the hybrid display device is located in front of the left eye / right eye. After the image analysis and processing module analyzes and processes the images, it can project the analyzed and processed images onto the hybrid display device.

[0009] It also includes a data processing module, a comparison module, a storage module, an analysis module, an output module and a projection module;

[0010] The data processing module can perform image processing on the images collected by the video recording device, and the image processing includes image noise reduction and feature extraction. The comparison module can compare the image processed by the data processing module with the image in the storage module, and determine the difference image between the collected image and the stored image. The storage module is used to store images and processing solutions. The analysis module can analyze the difference image and match the corresponding processing solution from the storage module according to the difference image. The output module can output the matched processing solution. The projection module can project the matched processing solution so that it is projected onto the hybrid display device.

[0011] Preferably, it also includes a wireless communication module, and the storage module is a cloud database. After the comparison module obtains the difference image, the wireless communication module can send the difference image to the cloud database. The analysis module is provided with an AI analysis model, and the AI ​​analysis module can analyze the data in the cloud database according to the difference image and obtain the processing solution from the storage module.

[0012] Preferably, it also includes an eye tracking module, a tactile feedback module, an ambient light detection module and an audio collection module. The eye tracking module can collect user eye movement data, the tactile feedback module is arranged on the surface of the head-mounted shell, the ambient light detection module is used to collect the intensity of the ambient light, and the audio collection module can collect sound information.

[0013] When the eye tracking module captures the user observing the solution on the hybrid display unit, the projection module can scroll the projected solution according to the preset scrolling speed, and during the playback process, when the user's eyes look in other directions, the solution played by the projection module can be paused, that is, the played solution stays on the hybrid display device and no longer scrolls.

[0014] Preferably, when the tactile feedback module detects a touch signal, the hybrid display device can change the light transmittance to enhance / weaken the clarity of the projected image.

[0015] Preferably, the ambient light detection module can adjust the brightness of the hybrid display device according to the intensity of the ambient light.

[0016] Preferably, the audio acquisition module can acquire audio information of the user, and when the acquired audio information is consistent with the solution projected on the hybrid display device, the projected solution is played synchronously according to the audio speed of the user.

[0017] Preferably, the video recording device is further provided with an infrared emission module, which can emit irradiation light, and the area surrounded by the irradiation light is recorded as the image acquisition area.

[0018] Preferably, an alarm module is also included. The processing solutions stored in the storage module are divided into safe operation processing solutions and dangerous operation processing solutions. During the process of image acquisition by the video recording device, when the solution analyzed by the analysis module is a dangerous operation processing solution, the alarm module can issue an alarm message.

[0019] Preferably, while the user is processing the scene according to the processing solution, the video recording device can continuously collect images, and when the collected images are inconsistent with the processing solution, the alarm module can issue a second alarm message.

[0020] Compared with the prior art, this application has at least the following beneficial effects:

[0021] When this system is implemented, by collecting images of the target area, with the help of the comparison module and the analysis module, it can display whether the equipment (parts) in the target area need maintenance or the maintenance method, and provide standardized guidance on whether the equipment needs maintenance and the maintenance operation, which solves the problem of the existing technology that mainly relies on the experience of maintenance personnel to operate. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more intuitively illustrate the prior art and the present application, exemplary drawings are given below. It should be understood that the specific shapes and structures shown in the drawings should not generally be regarded as limiting conditions for implementing the present application; for example, those skilled in the art are capable of easily making conventional adjustments or further optimizations to the addition / reduction / attribution division, specific shapes, positional relationships, connection methods, dimensional ratios, etc. of certain units (components) based on the technical concepts and exemplary drawings disclosed in the present application.

[0023] Figure 1 This is a schematic diagram of the structure of a wearable device capable of object recognition based on virtual reality in the present application;

[0024] Figure 2 This is a module diagram of a wearable device capable of object recognition based on virtual reality in the present application. DETAILED DESCRIPTION

[0025] The present application is further described below in detail through specific embodiments in conjunction with the accompanying drawings.

[0026] like Figure 1-2As shown, a wearable device capable of object recognition based on virtual reality includes a head-mounted shell, a video recording device, a hybrid display device, and a picture analysis and processing module, wherein the hybrid display device is arranged on a helmet-type display, and the video recording device and the picture analysis and processing module are both arranged in the head-mounted shell; the head-mounted shell can be worn on the head, making it convenient for users to use the device;

[0027] The video recording device can record the video images in front of the human eye, so that the user can collect images of the position that may need maintenance, and transmit the images to the image analysis and processing module. The hybrid display device is a transparent display screen structure (the color of the display screen is dark, and the transmittance is adjustable, so as to display the projected image), and the hybrid display device is located in front of the left eye / right eye. After analyzing and processing the image, the image analysis and processing module can project the analyzed and processed image onto the hybrid display device;

[0028] It also includes a data processing module, a comparison module, a storage module, an analysis module, an output module and a projection module;

[0029] The data processing module can perform image processing on the images collected by the video recording device. The image processing includes image noise reduction and feature extraction. Image noise reduction can effectively reduce the interference of other images on the target image, and after feature extraction, the type of device and the specific location of the image can be quickly judged.

[0030] The comparison module can compare the image processed by the data processing module with the image in the storage module, and determine the difference image between the collected image and the stored image. The acquisition of the difference image can determine the fault type of the equipment. The storage module is used to store images and processing solutions. The storage module stores images of the equipment when it is operating normally, as well as solutions when various problems occur in the equipment, so that they can be retrieved to guide the operation of maintenance personnel.

[0031] The analysis module can analyze the difference image and match the corresponding processing solution from the storage module according to the difference image. The output module can output the matched processing solution. The projection module can project the matched processing solution so that it is projected onto the hybrid display device.

[0032] When this system is implemented, by collecting images of the target area, with the help of the comparison module and the analysis module, it can display whether the equipment (parts) in the target area need maintenance or the maintenance method, and provide standardized guidance on whether the equipment needs maintenance and the maintenance operation, which solves the problem of the existing technology that mainly relies on the experience of maintenance personnel to operate.

[0033] As one of the implementation modes of the storage module, it also includes a wireless communication module. The storage module is a cloud database. The use of the cloud database can facilitate the updating of equipment failures and maintenance methods, and the memory capacity is larger. After the comparison module obtains the difference image, the wireless communication module can send the difference image to the cloud database. The analysis module is provided with an AI analysis model. The AI ​​analysis module can analyze the data in the cloud database according to the difference image and obtain the processing solution from the storage module. The AI ​​analysis module can quickly and accurately analyze the fault type and retrieve the optimal processing solution.

[0034] It also includes an eye tracking module, a tactile feedback module, an ambient light detection module and an audio collection module. The eye tracking module can collect user eye movement data, the tactile feedback module is arranged on the surface of the head-mounted shell, the ambient light detection module is used to collect the intensity of the ambient light, and the audio collection module can collect sound information.

[0035] When the eye tracking module captures the user observing the solution on the hybrid display unit, the projection module can scroll the projected solution according to the preset scrolling speed, and during the playback, when the user's eyes look in other directions, the solution played by the projection module can be paused, that is, the played solution stays on the hybrid display device and no longer scrolls, so that the user can perform maintenance operations while viewing the solution.

[0036] When the tactile feedback module detects a touch signal, the hybrid display device can change the transmittance and enhance / weaken the clarity of the projected image. That is, the transmittance of the hybrid display device can be adjusted by the user's touch, thereby facilitating the user's use in different scenarios.

[0037] The ambient light detection module can adjust the brightness of the hybrid display device according to the intensity of the ambient light. That is to say, the system can actively adjust the hybrid display device according to the intensity of the ambient light to enhance the user experience. Together with the tactile feedback module, it can meet the user's needs in various scenarios.

[0038] The audio acquisition module can acquire audio information of the user, and when the acquired audio information is consistent with the solution projected on the hybrid display device, the projected solution is played synchronously according to the audio speed of the user.

[0039] The video recording device is also provided with an infrared emission module, which can emit irradiation light, and the area surrounded by the irradiation light is recorded as the image acquisition area.

[0040] It also includes an alarm module. The processing solutions stored in the storage module are divided into safe operation processing solutions and dangerous operation processing solutions. When the video recording device is collecting images, when the solution analyzed by the analysis module is a dangerous operation processing solution, the alarm module can issue an alarm message.

[0041] When the user processes the scene according to the processing plan, the video recording device can continuously collect images. When the collected images are inconsistent with the processing plan, the alarm module can issue a second alarm message.

[0042] The alarm information 1 and the alarm information 2 are different alarm forms. For example, the alarm information 1 and the alarm information 2 may emit different sounds, vibrations or lights.

[0043] The technical features of the above embodiments may be arbitrarily combined (as long as there is no contradiction in the combination of these technical features). To make the description concise, not all possible combinations of the technical features in the above embodiments are described; these embodiments that are not explicitly written should also be considered to be within the scope of this specification.

Claims

1. A wearable device capable of object recognition based on virtual reality, characterized in that: It includes a head-mounted shell, a video recording device, a hybrid display device and a picture analysis and processing module, wherein the hybrid display device is arranged on a helmet-type display, and the video recording device and the picture analysis and processing module are both arranged in the head-mounted shell; The video recording device can record the video images in front of the human eye and transmit the images to the image analysis and processing module. The hybrid display device is a transparent display screen structure, and the hybrid display device is located in front of the left eye / right eye. After the image analysis and processing module analyzes and processes the images, it can project the analyzed and processed images onto the hybrid display device. It also includes a data processing module, a comparison module, a storage module, an analysis module, an output module and a projection module; The data processing module can perform image processing on the images collected by the video recording device, and the image processing includes image noise reduction and feature extraction. The comparison module can compare the image processed by the data processing module with the image in the storage module, and determine the difference image between the collected image and the stored image. The storage module is used to store images and processing solutions. The analysis module can analyze the difference image and match the corresponding processing solution from the storage module according to the difference image. The output module can output the matched processing solution. The projection module can project the matched processing solution so that it is projected onto the hybrid display device.

2. The object-recognition wearable device based on virtual reality according to claim 1, characterized in that: It also includes a wireless communication module, and the storage module is a cloud database. After the comparison module obtains the difference image, the wireless communication module can send the difference image to the cloud database. The analysis module is provided with an AI analysis model. The AI ​​analysis module can analyze the data in the cloud database according to the difference image and obtain a processing solution from the storage module.

3. The wearable device capable of object recognition based on virtual reality according to claim 2, characterized in that: It also includes an eye tracking module, a tactile feedback module, an ambient light detection module and an audio collection module. The eye tracking module can collect user eye movement data, the tactile feedback module is arranged on the surface of the head-mounted shell, the ambient light detection module is used to collect the intensity of the ambient light, and the audio collection module can collect sound information.

4. The object-recognition wearable device based on virtual reality according to claim 3, characterized in that: When the eye tracking module captures the user observing the solution on the hybrid display unit, the projection module can scroll the projected solution according to the preset scrolling speed, and during the playback process, when the user's eyes look in other directions, the solution played by the projection module can be paused, that is, the played solution stays on the hybrid display device and no longer scrolls.

5. The wearable device capable of object recognition based on virtual reality according to claim 3, characterized in that: When the tactile feedback module detects a touch signal, the hybrid display device can change the light transmittance to enhance / weaken the clarity of the projected image.

6. The object-recognition wearable device based on virtual reality according to claim 3, characterized in that: The ambient light detection module can adjust the brightness of the hybrid display device according to the intensity of the ambient light.

7. The wearable device capable of object recognition based on virtual reality according to claim 3, characterized in that: The audio acquisition module can acquire audio information of the user, and when the acquired audio information is consistent with the solution projected on the hybrid display device, the projected solution is played synchronously according to the audio speed of the user.

8. The wearable device capable of object recognition based on virtual reality according to claim 3, characterized in that: The video recording device is also provided with an infrared emission module, which can emit irradiation light, and the area surrounded by the irradiation light is recorded as the image acquisition area.

9. The wearable device capable of object recognition based on virtual reality according to claim 3, characterized in that: It also includes an alarm module. The processing solutions stored in the storage module are divided into safe operation processing solutions and dangerous operation processing solutions. When the video recording device is collecting images, when the solution analyzed by the analysis module is a dangerous operation processing solution, the alarm module can issue an alarm message.

10. The wearable device capable of object recognition based on virtual reality according to claim 3, characterized in that: When the user processes the scene according to the processing plan, the video recording device can continuously collect images. When the collected images are inconsistent with the processing plan, the alarm module can issue a second alarm message.

Images