VR game equipment display control method and system based on real-time images

Abstract

According to the VR game equipment display control method and system based on the real-time image, the shooting state of the shooting unit on the external environment is adjusted based on the wearing posture of the VR game equipment, the shot external environment image is rendered into the virtual environment image, and the virtual environment image and the external environment have the high compatibility matching degree; based on the real-time game scene attribute information, processing the virtual environment image into a virtual game scene image, determining a key image area and a non-key image area contained in the virtual game scene image, and performing image content identification on the virtual game scene image; and the projection state of the scene image is adjusted in a targeted manner based on the optimal game image watching area of the wearer, so that the key image area and the optimal game image watching area meet the preset relative position condition, and the image display verisimilitude of the VR game equipment and the watching immersion of the wearer are improved.

Description

Technical Field

[0001] The present invention relates to the field of VR display control, and particularly to a method and system for displaying and controlling a VR game device based on real-time images. Background Art

[0002] A head-mounted VR device can provide a three-dimensional virtual image for the wearer, and is widely used in VR game scenarios. After decoding the video source, the head-mounted VR device can project virtual images with parallax to the left and right eyes of the wearer, so that the wearer can view the corresponding three-dimensional virtual image. The video sources connected to the head-mounted VR device all transmit pre-recorded virtual images to it. Although this can provide virtual images in different content forms for the wearer, it also results in a poor match between the three-dimensional virtual image viewed by the wearer and the real environment image where the wearer is currently located, making the three-dimensional virtual image unable to be well integrated with the real environment, reducing the image display fidelity of the head-mounted VR device and the viewing immersion of the wearer. Summary of the Invention

[0003] The purpose of the present invention is to provide a method and system for displaying and controlling a VR game device based on real-time images. Based on the wearing posture of the VR game device, the shooting state of the camera unit of the VR game device for the external environment is adjusted, and the captured external environment image is rendered into a virtual environment image, so that the virtual environment image has a high compatibility and matching degree with the external environment; based on the real-time game scene attribute information, the virtual environment image is processed into a virtual game scene image, and the key image area and non-key image area included therein are determined, and the image content of the virtual game scene image is identified; also based on the best game image viewing area of the wearer, the projection state of the scene image is adjusted in a targeted manner, so that the key image area and the best game image viewing area meet the preset relative position conditions, improving the image display fidelity of the VR game device and the viewing immersion of the wearer.

[0004] The present invention is realized by the following technical solutions:

[0005] A method for displaying and controlling a VR game device based on real-time images includes:

[0006] Obtaining the wearing posture information of the VR game device, and based on the wearing posture information, adjusting the shooting state of the camera unit of the VR game device for the external environment; performing rendering and processing on the external environment image collected by the camera unit to obtain a virtual environment image;

[0007] Based on the real-time game scene attribute information of the VR game device, process the virtual environment image to form a virtual game scene image; identify the virtual game scene image to determine the key image area and non-key image area included in the virtual game scene image;

[0008] Obtain the line-of-sight state information of the wearer of the VR game device, and based on the line-of-sight state information, determine the best game image viewing area of the wearer; based on the best game image viewing area, adjust the projection state parameters of the virtual game scene image to the wearer, so that the key image area and the best game image viewing area meet the preset relative position conditions.

[0009] Optionally, obtain the wearing posture information of the VR game device, and based on the wearing posture information, adjust the shooting state of the camera unit of the VR game device for the external environment; perform rendering and processing on the external environment image collected by the camera unit to obtain a virtual environment image, including:

[0010] Perform a six-degree-of-freedom azimuth angle detection on the VR game device to obtain the attitude azimuth angle information of the VR game device at the wearer's head; based on the attitude azimuth angle information of the VR game device, adjust the shooting direction angle of the camera unit for the external environment, so that the camera unit captures an external environment image that meets the preset azimuth orientation conditions;

[0011] Perform pixel contour recognition processing on the external environment image to obtain the background area pixel contour feature information and non-background area pixel contour feature information of the external environment image; based on the background area pixel contour feature information and non-background area pixel contour feature information, perform contour edge sharpening rendering processing on the background area of the external environment image and perform contour edge blurring rendering processing on the non-background area of the external environment image to obtain a virtual environment image.

[0012] Optionally, performing a six-degree-of-freedom azimuth angle detection on the VR game device includes:

[0013] Step S1, using the following formula (1), based on the six-degree-of-freedom azimuth angles detected by two six-degree-of-freedom azimuth angle detection devices respectively, determine whether the six-degree-of-freedom azimuth angle detection device is working properly,

[0014]

[0015] In the above formula (1), E represents the judgment value of whether two six-degree-of-freedom spatial azimuth detection devices are working properly; θ1(a) represents the angle value detected by the first six-degree-of-freedom spatial azimuth detection device at the a-th azimuth; θ2(a) represents the angle value detected by the second six-degree-of-freedom spatial azimuth detection device at the a-th azimuth; α0 represents the preset maximum deviation threshold angle; || represents taking the absolute value.

[0016] If E = 1, it means that both six-degree-of-freedom spatial azimuth detection devices are working properly.

[0017] If E = 0, it means that the two six-degree-of-freedom spatial azimuth detection devices are not working properly.

[0018] Step S2, when both six-degree-of-freedom spatial azimuth detection devices are working properly, use the following formula (2) to merge the numerical values of the symmetric angle values according to the six-degree-of-freedom azimuth angles detected by the two six-degree-of-freedom spatial azimuth detection devices respectively.

[0019]

[0020] In the above formula (2), represents the angle value detected by the k-th six-degree-of-freedom spatial azimuth detection device at the a-th azimuth after merging the numerical values of the symmetric angle values, where k = 1 or 2; θ k (a) represents the angle value detected by the k-th six-degree-of-freedom spatial azimuth detection device at the a-th azimuth. represents the angle value detected by the k-th six-degree-of-freedom spatial azimuth detection device at the azimuth opposite to the a-th azimuth; H[θ k (a)] represents obtaining the sign of θ k (a). If θ k (a) is positive, the result of H[θ k (a)] is 1. If θ k (a) is negative, the result of H[θ k (a)] is -1.

[0021] Step S3, use the following formula (3) to obtain the final angle result value according to the angle values after merging the numerical values of the symmetric angle values detected by the two six-degree-of-freedom spatial azimuth detection devices respectively.

[0022]

[0023] In the above formula (3), represents the a-th final angle result value.

[0024] Optionally, based on the real-time game scene attribute information of the VR game device, the virtual environment image is processed to form a virtual game scene image; the virtual game scene image is recognized to determine the key image area and non-key image area included in the virtual game scene image, including:

[0025] Based on the real-time game scene type attribute information of the VR game device, several game object elements are selected from the game scene database, and the game object elements are loaded into the virtual environment image to obtain a virtual game scene image;

[0026] The distribution position of the game object elements in the virtual game scene image is recognized to determine the distribution density of the game object elements in the virtual game scene image; if the distribution density of the game object elements is greater than or equal to the preset density threshold, the area corresponding to the virtual game scene image is determined as the key image area; otherwise, the area corresponding to the virtual game scene image is determined as the non-key image area.

[0027] Optionally, the line-of-sight state information of the wearer of the VR game device is obtained, and based on the line-of-sight state information, the best game image viewing area of the wearer is determined; based on the best game image viewing area, the projection state parameters of the virtual game scene image to the wearer are adjusted so that the key image area and the best game image viewing area meet the preset relative position conditions, including:

[0028] The line-of-sight range information of each of the left and right eyes of the wearer of the VR game device is obtained, and the line-of-sight range information of each of the left and right eyes is compared to obtain the binocular overlapping line-of-sight range area of the wearer, which is used as the best game image viewing area of the wearer;

[0029] Based on the key image area and non-key image area included in the virtual game scene image, the screen position information of the key image area in the virtual game scene image is determined;

[0030] Based on the best game image viewing area and the screen position information, the projection direction and / or projection focal depth of the virtual game scene image to the wearer are adjusted so that the horizontal offset distance and vertical offset distance between the center of the key image area and the center of the best game image viewing area are both less than the preset distance threshold.

[0031] The VR game device display control system based on real-time images includes:

[0032] The camera adjustment and operation module is used to obtain the wearing posture information of the VR game device, and based on the wearing posture information, adjust the shooting state of the camera unit of the VR game device for the external environment;

[0033] An image rendering and processing module for rendering and processing the external environment image collected by the camera unit to obtain a virtual environment image;

[0034] A virtual game scene image generation module for processing the virtual environment image into a virtual game scene image based on the real-time game scene attribute information of the VR game device;

[0035] A virtual game scene image recognition module for recognizing the virtual game scene image to determine the key image area and non-key image area included in the virtual game scene image;

[0036] An optimal viewing area determination module for obtaining the line-of-sight state information of the wearer of the VR game device and determining the optimal game image viewing area of the wearer based on the line-of-sight state information;

[0037] A projection adjustment module for adjusting the projection state parameters of the virtual game scene image to the wearer based on the optimal game image viewing area, so that the key image area and the optimal game image viewing area meet the preset relative position conditions.

[0038] Optionally, the camera adjustment and operation module is used to obtain the wearing posture information of the VR game device and adjust the shooting state of the camera unit of the VR game device for the external environment based on the wearing posture information, including:

[0039] Performing a six-degree-of-freedom azimuth angle detection on the VR game device to obtain the attitude azimuth angle information of the VR game device at the wearer's head; based on the attitude azimuth angle information of the VR game device, adjusting the shooting direction angle of the camera unit for the external environment, so that the camera unit captures an external environment image that meets the preset azimuth orientation condition;

[0040] The image rendering and processing module is used to render and process the external environment image collected by the camera unit to obtain a virtual environment image, including:

[0041] Performing pixel contour recognition processing on the external environment image to obtain the background area pixel contour feature information and non-background area pixel contour feature information of the external environment image; based on the background area pixel contour feature information and non-background area pixel contour feature information, performing contour edge sharpening rendering processing on the background area of the external environment image and performing contour edge blurring rendering processing on the non-background area of the external environment image to obtain a virtual environment image.

[0042] Optionally, the virtual game scene image generation module is used to process the virtual environment image into a virtual game scene image based on the real-time game scene attribute information of the VR game device, including:

[0043] Based on the real-time game scene type attribute information of the VR game device, select several game object elements from the game scene database, and load the game object elements into the virtual environment image to obtain a virtual game scene image;

[0044] The virtual game scene image recognition module is used to recognize the virtual game scene image and determine the key image area and non-key image area included in the virtual game scene image, including:

[0045] Perform game object element distribution position recognition on the virtual game scene image to determine the distribution density of the game object elements in the virtual game scene image; if the distribution density of the game object elements is greater than or equal to the preset density threshold, determine the area corresponding to the virtual game scene image as the key image area; otherwise, determine the area corresponding to the virtual game scene image as the non-key image area.

[0046] Optionally, the best viewing area determination module is used to obtain the line-of-sight state information of the wearer of the VR game device, and based on the line-of-sight state information, determine the best game image viewing area of the wearer, including:

[0047] Obtain the field-of-view range information of each of the left and right eyes of the wearer of the VR game device, compare the field-of-view range information of each of the left and right eyes, and obtain the binocular overlapping field-of-view range area of the wearer, and use this as the best game image viewing area of the wearer;

[0048] The projection adjustment module is used to adjust the projection state parameters of the virtual game scene image to the wearer based on the best game image viewing area, so that the key image area and the best game image viewing area meet the preset relative position conditions, including:

[0049] Based on the key image area and non-key image area included in the virtual game scene image, determine the screen position information of the key image area in the virtual game scene image;

[0050] Based on the best game image viewing area and the screen position information, adjust the projection direction and / or projection focal depth of the virtual game scene image to the wearer, so that the horizontal offset distance and vertical offset distance between the center of the key image area and the center of the best game image viewing area are both less than the preset distance threshold.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] The display control method and system for VR game devices based on real-time images provided by this application adjust the shooting state of the camera unit of the VR game device for the external environment based on the wearing posture of the VR game device, and render the captured external environment images into virtual environment images, so that the virtual environment images have a high compatibility and matching degree with the external environment; based on the real-time game scene attribute information, process the virtual environment images into virtual game scene images, determine the key image areas and non-key image areas included therein, and perform image content recognition on the virtual game scene images; also, based on the best game image viewing area of the wearer, adjust the projection state of the scene images in a targeted manner, so that the key image areas and the best game image viewing area meet the preset relative position conditions, improving the image display realism of the VR game device and the viewing immersion of the wearer. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following-described drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts. Among them:

[0054] Figure 1 It is a flowchart showing the display control method for VR game devices based on real-time images provided by the present invention.

[0055] Figure 2 It is a structural diagram showing the display control system for VR game devices based on real-time images provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] To make the above objects, features, and advantages of this application more obvious and understandable, the following will provide a detailed description of the specific embodiments of this application in conjunction with the drawings. It can be understood that the specific embodiments described herein are only used to explain this application, rather than limiting this application. Additionally, it should be noted that for the sake of description, only parts related to this application are shown in the drawings, rather than all the structures. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

[0057] The term "comprising" and "having" in this application, as well as any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally further include steps or units not listed, or may optionally further include other steps or units inherent to these processes, methods, products, or devices.

[0058] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0059] Please refer to Figure 1 As shown, a method for displaying and controlling a VR game device based on real-time images according to an embodiment of the present application includes:

[0060] Obtain the wearing posture information of the VR game device, and based on the wearing posture information, adjust the shooting state of the camera unit of the VR game device for the external environment; perform rendering and processing on the external environment images collected by the camera unit to obtain virtual environment images;

[0061] Based on the real-time game scene attribute information of the VR game device, process the virtual environment images to form virtual game scene images; identify the virtual game scene images to determine the key image areas and non-key image areas included in the virtual game scene images;

[0062] Obtain the line-of-sight state information of the wearer of the VR game device, and based on the line-of-sight state information, determine the best game image viewing area of the wearer; based on the best game image viewing area, adjust the projection state parameters of the virtual game scene images to the wearer so that the key image areas and the best game image viewing area meet the preset relative position conditions.

[0063] Beneficial effects of the above embodiments. The display control method for a VR game device based on real-time images adjusts the shooting state of its camera unit for the external environment based on the wearing posture of the VR game device, and renders the captured external environment image into a virtual environment image, so that the virtual environment image has a high compatibility and matching degree with the external environment; based on the real-time game scene attribute information, processes the virtual environment image into a virtual game scene image, determines the key image area and non-key image area included therein, and performs image content recognition on the virtual game scene image; also adjusts the projection state of the scene image targeted based on the best game image viewing area of the wearer, so that the key image area and the best game image viewing area meet the preset relative position conditions, improving the image display realism of the VR game device and the viewing immersion of the wearer.

[0064] In another embodiment, obtain the wearing posture information of the VR game device, and based on the wearing posture information, adjust the shooting state of the camera unit of the VR game device for the external environment; perform rendering and processing on the external environment image collected by the camera unit to obtain a virtual environment image, including:

[0065] Perform a six-degree-of-freedom azimuth angle detection on the VR game device to obtain the attitude azimuth angle information of the VR game device at the wearer's head; based on the attitude azimuth angle information of the VR game device, adjust the shooting direction angle of the camera unit for the external environment so that the camera unit captures an external environment image that meets the preset azimuth orientation condition;

[0066] Perform pixel contour recognition processing on the external environment image to obtain the background area pixel contour feature information and non-background area pixel contour feature information of the external environment image; based on the background area pixel contour feature information and non-background area pixel contour feature information, perform contour edge sharpening rendering processing on the background area of the external environment image and perform contour edge blurring rendering processing on the non-background area of the external environment image to obtain a virtual environment image.

[0067] The beneficial effects of the above embodiments are as follows. The VR game device is worn on the head of the wearer and is used to project virtual images onto the eyes of the wearer, enabling the wearer to view three-dimensional virtual game scene images. An imaging unit is provided inside the VR game device for photographing the real environment where the wearer is currently located. If the VR game device is not correctly worn on the head of the wearer but is worn on the head in an inclined manner, the image of the external environment captured by the imaging unit will also be inclined. To ensure the correctness of the captured image of the external environment, a six-degree-of-freedom azimuth angle detection of the VR game device is performed to obtain the attitude azimuth angle information of the VR game device on the head of the wearer. Based on the attitude azimuth angle information of the VR game device, the shooting direction angle of the imaging unit for the external environment is adjusted so that the image of the external environment captured by the imaging unit is in a non-inclined state, improving the correctness of the image of the external environment. In addition, pixel contour recognition processing is performed on the image of the external environment to obtain the pixel contour feature information of the background area and the non-background area of the image of the external environment. The non-background area of the image of the external environment refers to the area where people and objects exist in the external environment, and the background area of the image of the external environment refers to all other areas except the non-background area. Based on the pixel contour feature information of the background area and the non-background area, contour edge sharpening rendering processing is performed on the background area of the image of the external environment and contour edge blurring rendering processing is performed on the non-background area of the image of the external environment to obtain a virtual environment image, which can highlight the background area part in the virtual environment image and improve the visual authenticity of the virtual game scene image.

[0068] In another embodiment, the six-degree-of-freedom azimuth angle detection of the VR game device includes:

[0069] Step S1, using the following formula (1), based on the six-degree-of-freedom azimuth angles detected by two six-degree-of-freedom azimuth angle detection devices respectively, determine whether the six-degree-of-freedom azimuth angle detection device is working properly.

[0070]

[0071] In the above formula (1), E represents the judgment value of whether two six-degree-of-freedom azimuth angle detection devices are working properly; θ1(a) represents the angle value of the a-th azimuth detected by the first six-degree-of-freedom azimuth angle detection device; θ2(a) represents the angle value of the a-th azimuth detected by the second six-degree-of-freedom azimuth angle detection device; α0 represents the preset maximum deviation threshold angle; || represents taking the absolute value.

[0072] If E = 1, it means that both six-degree-of-freedom azimuth angle detection devices are working properly.

[0073] If E = 0, it means that the two spatial six-degree-of-freedom azimuth detection devices are not working properly;

[0074] Step S2, when both of the two spatial six-degree-of-freedom azimuth detection devices are working properly, use the following formula (2) to perform numerical merging of symmetric angle values based on the six-degree-of-freedom azimuth angles detected by the two spatial six-degree-of-freedom azimuth detection devices respectively.

[0075]

[0076] In the above formula (2), represents the angle value of the a-th azimuth detected by the k-th spatial six-degree-of-freedom azimuth detection device after numerical merging of symmetric angle values, where k = 1 or 2; θ k (a) represents the angle value of the a-th azimuth detected by the k-th spatial six-degree-of-freedom azimuth detection device; represents the angle value of the azimuth opposite to the a-th azimuth detected by the k-th spatial six-degree-of-freedom azimuth detection device; H[θ k (a)] represents obtaining the sign of θ k (a), if θ k (a) is a positive number, then the result of H[θ k (a)] is 1, if θ k (a) is a negative number, then the result of H[θ k (a)] is -1;

[0077] Step S3, use the following formula (3) to obtain the final angle result value based on the angle values after numerical merging of symmetric angle values detected by the two spatial six-degree-of-freedom azimuth detection devices respectively.

[0078]

[0079] In the above formula (3), represents the a-th final angle result value.

[0080] The beneficial effects of the above embodiments are as follows. Using the above formula (1), based on the six-degree-of-freedom azimuth angles detected by two spatial six-degree-of-freedom azimuth detection devices respectively, it is determined whether the spatial six-degree-of-freedom azimuth detection device is working properly, thereby automatically detecting the device status, which reflects the intelligence of the system. Then, using the above formula (2), based on the six-degree-of-freedom azimuth angles detected by two spatial six-degree-of-freedom azimuth detection devices respectively, the numerical values of the symmetric angles are merged, so as to perform self-correction using the detection of its own azimuth and ensure the accuracy of independent detection. Finally, using the above formula (3), based on the angle values after the numerical values of the symmetric angles are merged according to the six-degree-of-freedom azimuth angles detected by two spatial six-degree-of-freedom azimuth detection devices respectively, the final angle result value is obtained. Thus, the two devices perform synchronous measurement and merge the measurement numerical values, which can make the result more accurate and ensure the reliability of subsequent control.

[0081] In another embodiment, based on the real-time game scene attribute information of the VR game device, the virtual environment image is processed to form a virtual game scene image; the virtual game scene image is recognized to determine the key image area and non-key image area included in the virtual game scene image, including:

[0082] Based on the real-time game scene type attribute information of the VR game device, several game object elements are selected from the game scene database, and the game object elements are loaded into the virtual environment image to obtain a virtual game scene image;

[0083] The distribution position of the game object elements in the virtual game scene image is recognized to determine the distribution density of the game object elements in the virtual game scene image; if the distribution density of the game object elements is greater than or equal to a preset density threshold, the area corresponding to the virtual game scene image is determined as the key image area; otherwise, the area corresponding to the virtual game scene image is determined as the non-key image area.

[0084] Based on the type attribute information of the game scene (such as the scene content type information corresponding to the game scene) displayed in real time by the VR game device, several game object elements are selected from the game scene database. The game object elements may include, but are not limited to, the people and / or objects displayed in the game scene. Then, the game object elements are loaded and merged into the virtual environment image to obtain a virtual game scene image containing the external environment background information and virtual object elements. Next, the distribution position of the game object elements in the virtual game scene image is recognized to determine the distribution density of the game object elements in the virtual game scene image. If the distribution density of the game object elements is greater than or equal to the preset density threshold, the area corresponding to the virtual game scene image is determined as the key image area; otherwise, the area corresponding to the virtual game scene image is determined as the non-key image area. In this way, the visual area of the virtual game scene image can be accurately distinguished, which is convenient for subsequent targeted projection of the image to the wearer.

[0085] In another embodiment, the line-of-sight state information of the wearer of the VR game device is obtained. Based on the line-of-sight state information, the best game image viewing area of the wearer is determined. Based on the best game image viewing area, the projection state parameters of the virtual game scene image to the wearer are adjusted so that the key image area and the best game image viewing area meet the preset relative position conditions, including:

[0086] The field-of-view range information of each of the left and right eyes of the wearer of the VR game device is obtained, and the field-of-view range information of each of the left and right eyes is compared to obtain the binocular overlapping field-of-view range area of the wearer, which is used as the best game image viewing area of the wearer.

[0087] Based on the key image area and the non-key image area included in the virtual game scene image, the screen position information of the key image area in the virtual game scene image is determined.

[0088] Based on the best game image viewing area and the screen position information, the projection direction and / or projection focal depth of the virtual game scene image to the wearer are adjusted so that the horizontal offset distance and the vertical offset distance between the center of the key image area and the center of the best game image viewing area are both less than the preset distance threshold.

[0089] The beneficial effects of the above embodiments are as follows: obtaining the field-of-view range information of each of the left and right eyes of the wearer of the VR game device, comparing the field-of-view range information of each of the left and right eyes to obtain the binocular overlapping field-of-view range area of the wearer, and using this as the best game image viewing area for the wearer. This facilitates accurately projecting the virtual image onto the best game image viewing area subsequently, enabling the wearer to obtain an optimal viewing visual experience. In addition, based on the key image area and non-key image area included in the virtual game scene image, the screen position information of the key image area in the virtual game scene image is determined, which can accurately locate the position of the key image area in the entire screen of the virtual game scene image. And based on the best game image viewing area and the screen position information, the projection direction and / or projection depth of the virtual game scene image towards the wearer are adjusted so that both the horizontal offset distance and the vertical offset distance between the center of the key image area and the center of the best game image viewing area are less than a preset distance threshold. In this way, the wearer can accurately view the key image area, improving the image display fidelity of the VR game device and the viewing immersion of the wearer.

[0090] Please refer to Figure 2 As shown in the figure, the VR game device display control system based on real-time images provided by an embodiment of the present application includes:

[0091] The camera adjustment and operation module is used to obtain the wearing posture information of the VR game device and adjust the shooting state of the camera unit of the VR game device for the external environment based on the wearing posture information;

[0092] The image rendering and processing module is used to perform rendering and processing on the external environment image collected by the camera unit to obtain a virtual environment image;

[0093] The virtual game scene image generation module is used to process the virtual environment image into a virtual game scene image based on the real-time game scene attribute information of the VR game device;

[0094] The virtual game scene image recognition module is used to recognize the virtual game scene image and determine the key image area and non-key image area included in the virtual game scene image;

[0095] The best viewing area determination module is used to obtain the line-of-sight state information of the wearer of the VR game device and determine the best game image viewing area of the wearer based on the line-of-sight state information;

[0096] The projection adjustment module is used to adjust the projection state parameters of the virtual game scene image towards the wearer based on the best game image viewing area so that the key image area and the best game image viewing area meet the preset relative position conditions.

[0097] Beneficial effects of the above embodiments: The display control system of the VR game device based on real-time images adjusts the shooting state of its camera unit for the external environment based on the wearing posture of the VR game device, and renders the captured external environment images into virtual environment images, so that the virtual environment images have a high compatibility and matching degree with the external environment; based on the real-time game scene attribute information, the virtual environment images are processed into virtual game scene images, and the key image areas and non-key image areas included therein are determined, and the image content of the virtual game scene images is recognized; also based on the best game image viewing area of the wearer, the projection state of the scene images is adjusted specifically, so that the key image areas and the best game image viewing areas meet the preset relative position conditions, improving the image display realism of the VR game device and the viewing immersion of the wearer.

[0098] In another embodiment, the camera adjustment and operation module is used to obtain the wearing posture information of the VR game device, and based on the wearing posture information, adjust the shooting state of the camera unit of the VR game device for the external environment, including:

[0099] Perform a six-degree-of-freedom azimuth angle detection on the VR game device to obtain the attitude azimuth angle information of the VR game device at the wearer's head; based on the attitude azimuth angle information of the VR game device, adjust the shooting direction angle of the camera unit for the external environment, so that the camera unit captures external environment images that meet the preset azimuth orientation conditions;

[0100] The image rendering and processing module is used to perform rendering and processing on the external environment images collected by the camera unit to obtain virtual environment images, including:

[0101] Perform pixel contour recognition processing on the external environment images to obtain the background area pixel contour feature information and non-background area pixel contour feature information of the external environment images; based on the background area pixel contour feature information and non-background area pixel contour feature information, perform contour edge sharpening rendering processing on the background area of the external environment images and perform contour edge blurring rendering processing on the non-background area of the external environment images to obtain virtual environment images.

[0102] The beneficial effects of the above embodiments are as follows. The VR game device is worn on the head of the wearer and is used to project virtual images onto the eyes of the wearer, enabling the wearer to view three-dimensional virtual game scene images. An imaging unit is provided inside the VR game device for photographing the real environment where the wearer is currently located. If the VR game device is not correctly worn on the head of the wearer but is worn on the head in an inclined manner, the images of the external environment captured by the imaging unit will also be inclined. To ensure the correctness of the captured images of the external environment, a six-degree-of-freedom azimuth angle detection of the VR game device is performed to obtain the attitude azimuth angle information of the VR game device on the head of the wearer. Based on the attitude azimuth angle information of the VR game device, the shooting direction angle of the imaging unit for the external environment is adjusted so that the corresponding images of the external environment captured by the imaging unit are in a non-inclined state, improving the correctness of the images of the external environment. In addition, pixel contour recognition processing is performed on the images of the external environment to obtain the pixel contour feature information of the background area and the non-background area of the images of the external environment. The non-background area of the images of the external environment refers to the area where people and objects exist in the external environment, and the background area of the images of the external environment refers to all other areas except the non-background area. Based on the pixel contour feature information of the background area and the non-background area, contour edge sharpening rendering processing is performed on the background area of the images of the external environment and contour edge blurring rendering processing is performed on the non-background area of the images of the external environment to obtain virtual environment images, which can highlight the background area part in the virtual environment images and improve the visual authenticity of the virtual game scene images.

[0103] In another embodiment, the virtual game scene image generation module is used to process the virtual environment image into a virtual game scene image based on the real-time game scene attribute information of the VR game device, including:

[0104] Based on the real-time game scene type attribute information of the VR game device, several game object elements are selected from the game scene database and the game object elements are loaded into the virtual environment image to obtain a virtual game scene image;

[0105] The virtual game scene image recognition module is used to recognize the virtual game scene image to determine the key image area and the non-key image area included in the virtual game scene image, including:

[0106] Perform game object element distribution position recognition on the virtual game scene image to determine the distribution density of the game object elements in the virtual game scene image. If the distribution density of the game object elements is greater than or equal to the preset density threshold, the area corresponding to the virtual game scene image is determined as the key image area; otherwise, the area corresponding to the virtual game scene image is determined as the non-key image area.

[0107] Based on the type attribute information of the game scene (such as the scene content type information corresponding to the game scene) displayed in real time by the VR game device, several game object elements are selected from the game scene database. The game object elements may include, but are not limited to, the people and / or objects displayed in the game scene. Then, the game object elements are loaded and merged into the virtual environment image to obtain a virtual game scene image containing external environmental background information and virtual object elements. Next, the distribution position of the game object elements in the virtual game scene image is recognized to determine the distribution density of the game object elements in the virtual game scene image. If the distribution density of the game object elements is greater than or equal to a preset density threshold, the area corresponding to the virtual game scene image is determined as a key image area; otherwise, the area corresponding to the virtual game scene image is determined as a non-key image area. In this way, the visual area of the virtual game scene image can be accurately distinguished, facilitating subsequent targeted projection of images to the wearer.

[0108] In another embodiment, the best viewing area determination module is used to obtain the line-of-sight state information of the wearer of the VR game device. Based on the line-of-sight state information, the best game image viewing area of the wearer is determined, including:

[0109] Obtain the field-of-view range information of each of the left and right eyes of the wearer of the VR game device, compare the field-of-view range information of each of the left and right eyes, and obtain the binocular overlapping field-of-view range area of the wearer, which is used as the best game image viewing area of the wearer;

[0110] The projection adjustment module is used to adjust the projection state parameters of the virtual game scene image to the wearer based on the best game image viewing area, so that the key image area and the best game image viewing area meet the preset relative position conditions, including:

[0111] Based on the key image area and the non-key image area included in the virtual game scene image, determine the screen position information of the key image area in the virtual game scene image;

[0112] Based on the best game image viewing area and the screen position information, adjust the projection direction and / or projection focal depth of the virtual game scene image to the wearer, so that the horizontal offset distance and the vertical offset distance between the center of the key image area and the center of the best game image viewing area are both less than a preset distance threshold.

[0113] The beneficial effects of the above embodiments are as follows: obtaining the field of view range information of each of the left and right eyes of the wearer of the VR game device, comparing the field of view range information of each of the left and right eyes to obtain the binocular overlapping field of view range area of the wearer, and using this as the best game image viewing area of the wearer, which is convenient for accurately projecting the virtual image onto the best game image viewing area subsequently, so that the wearer can obtain the optimal viewing visual experience. In addition, based on the key image area and the non-key image area included in the virtual game scene image, the position information of the key image area in the virtual game scene image is determined, so that the location of the key image area in the entire virtual game scene image can be accurately positioned. And based on the best game image viewing area and the position information, the projection direction and / or projection depth of the virtual game scene image towards the wearer are adjusted, so that the horizontal offset distance and the vertical offset distance between the center of the key image area and the center of the best game image viewing area are both less than the preset distance threshold, so that the wearer can accurately view the key image area, improving the image display realism of the VR game device and the viewing immersion of the wearer.

[0114] Generally speaking, the VR game device display control method and system based on real-time images adjust the shooting state of its camera unit for the external environment based on the wearing posture of the VR game device, and render the captured external environment image into a virtual environment image, so that the virtual environment image has a high compatibility and matching degree with the external environment; based on the real-time game scene attribute information, the virtual environment image is processed into a virtual game scene image, and the key image area and the non-key image area included therein are determined, and image content recognition is performed on the virtual game scene image; also based on the best game image viewing area of the wearer, the projection state of the scene image is adjusted targeted, so that the key image area and the best game image viewing area meet the preset relative position conditions, improving the image display realism of the VR game device and the viewing immersion of the wearer.

[0115] The above is only a specific embodiment of the present invention, and any improvement made on the premise of the present invention concept is regarded as the protection scope of the present invention.

Claims

1. A method for controlling the display of a VR game device based on real-time images, characterized in that, Including: Obtain the wearing posture information of the VR game device, and based on the wearing posture information, adjust the shooting state of the camera unit of the VR game device for the external environment; Render and process the external environment images collected by the camera unit to obtain virtual environment images; Based on the real-time game scene attribute information of the VR game device, process the virtual environment images to form virtual game scene images; identify the virtual game scene images to determine the key image areas and non-key image areas included in the virtual game scene images; Obtain the line-of-sight state information of the wearer of the VR game device, and based on the line-of-sight state information, determine the best game image viewing area of the wearer; based on the best game image viewing area, adjust the projection state parameters of the virtual game scene images to the wearer so that the key image areas and the best game image viewing area meet the preset relative position conditions.

2. The method for controlling the display of a VR game device based on real-time images according to claim 1, characterized in that: Obtain the wearing posture information of the VR game device, and based on the wearing posture information, adjust the shooting state of the camera unit of the VR game device for the external environment; Render and process the external environment images collected by the camera unit to obtain virtual environment images, including: Perform spatial six-degree-of-freedom azimuth angle detection on the VR game device to obtain the attitude azimuth angle information of the VR game device at the wearer's head; Based on the attitude azimuth angle information of the VR game device, adjust the shooting direction angle of the camera unit for the external environment so that the camera unit captures external environment images that meet the preset azimuth orientation conditions; Perform pixel contour recognition processing on the external environment images to obtain the background area pixel contour feature information and non-background area pixel contour feature information of the external environment images; based on the background area pixel contour feature information and non-background area pixel contour feature information, perform contour edge sharpening rendering processing on the background area of the external environment images and perform contour edge blurring rendering processing on the non-background area of the external environment images to obtain virtual environment images.

3. The method for controlling the display of a VR game device based on real-time images according to claim 2, characterized in that: Perform spatial six-degree-of-freedom azimuth angle detection on the VR game device, including: Step S1, use the following formula (1) to judge whether the spatial six-degree-of-freedom azimuth angle detection device is working properly according to the six-degree-of-freedom azimuth angles detected by two spatial six-degree-of-freedom azimuth angle detection devices, In the above formula (1), E represents the judgment value of whether two spatial six-degree-of-freedom azimuth angle detection devices are working properly; θ1(a) represents the angle value of the a-th azimuth detected by the first spatial six-degree-of-freedom azimuth angle detection device; θ2(a) represents the angle value of the a-th azimuth detected by the second spatial six-degree-of-freedom azimuth angle detection device; α0 represents the preset maximum deviation threshold angle; || represents taking the absolute value; If E = 1, it means that both spatial six-degree-of-freedom azimuth angle detection devices are working properly; If E = 0, it means that the two spatial six-degree-of-freedom azimuth angle detection devices are not working properly; Step S2, when both of the two six - degree - of - freedom spatial azimuth detection devices are working properly, use the following formula (2) to perform numerical merging of symmetric angle values based on the six - degree - of - freedom azimuth angles detected by the two six - degree - of - freedom spatial azimuth detection devices respectively. In the above formula (2), represents the angular value of the a-th azimuth detected by the k-th spatial six-degree-of-freedom azimuth detection device after the numerical combination of symmetric angular values, where k = 1 or 2; θ k (a) represents the angular value of the a-th azimuth detected by the k-th spatial six-degree-of-freedom azimuth detection device; represents the angular value of the azimuth opposite to the a-th azimuth detected by the k-th spatial six-degree-of-freedom azimuth detection device; H[θ k (a)] represents obtaining the sign of θ(a). If θ k (a) is a positive number, then the result of H[θ k (a)] is 1. If θ k (a) is a negative number, then the result of H[θ k (a)] is -1; Step S3, use the following formula (3) to obtain the final angle result value based on the angle values after numerical merging of symmetric angle values according to the six - degree - of - freedom azimuth angles detected by the two six - degree - of - freedom spatial azimuth detection devices respectively. In the above formula (3), represents the a-th final angle result value.

4. The method for controlling the display of a VR game device based on real-time images according to claim 1, characterized in that: Based on the real - time game scene attribute information of the VR game device, process the virtual environment image to form a virtual game scene image; identify the virtual game scene image to determine the key image area and non - key image area included in the virtual game scene image, including: Based on the real - time game scene type attribute information of the VR game device, select several game object elements from the game scene database and load the game object elements into the virtual environment image to obtain a virtual game scene image. Identify the distribution position of the game object elements in the virtual game scene image to determine the distribution density of the game object elements in the virtual game scene image; if the distribution density of the game object elements is greater than or equal to the preset density threshold, determine the area corresponding to the virtual game scene image as the key image area; otherwise, determine the area corresponding to the virtual game scene image as the non - key image area.

5. The method for controlling the display of a VR game device based on real-time images according to claim 1, characterized in that: Obtain the line - of - sight state information of the wearer of the VR game device, and based on the line - of - sight state information, determine the best game image viewing area of the wearer; based on the best game image viewing area, adjust the projection state parameters of the virtual game scene image to the wearer so that the key image area and the best game image viewing area meet the preset relative position conditions, including: Obtain the field - of - view range information of each of the left and right eyes of the wearer of the VR game device, compare the field - of - view range information of each of the left and right eyes to obtain the binocular overlapping field - of - view range area of the wearer, and use this as the best game image viewing area of the wearer. Based on the key image area and non - key image area included in the virtual game scene image, determine the screen position information of the key image area in the virtual game scene image. Based on the best game image viewing area and the screen position information, adjust the projection direction and / or projection focal depth of the virtual game scene image to the wearer so that both the horizontal offset distance and the vertical offset distance between the center of the key image area and the center of the best game image viewing area are less than the preset distance threshold.

6. A display control system for a VR game device based on real-time images, characterized in that, Including: A camera adjustment and operation module, which is used to obtain the wearing posture information of the VR game device and adjust the shooting state of the camera unit of the VR game device for the external environment based on the wearing posture information. An image rendering and processing module, which is used to perform rendering and processing on the external environment image collected by the camera unit to obtain a virtual environment image. A virtual game scene image generation module, which is used to process the virtual environment image into a virtual game scene image based on the real-time game scene attribute information of the VR game device; A virtual game scene image recognition module, which is used to recognize the virtual game scene image and determine the key image area and non-key image area included in the virtual game scene image; An optimal viewing area determination module, which is used to obtain the line-of-sight state information of the wearer of the VR game device and determine the optimal game image viewing area of the wearer based on the line-of-sight state information; A projection adjustment module, which is used to adjust the projection state parameters of the virtual game scene image to the wearer based on the optimal game image viewing area, so that the key image area and the optimal game image viewing area meet the preset relative position conditions.

7. The display control system of a VR game device based on real-time images according to claim 6, wherein: The camera adjustment and operation module is used to obtain the wearing posture information of the VR game device and adjust the shooting state of the camera unit of the VR game device to the external environment based on the wearing posture information, including: Performing a six-degree-of-freedom azimuth angle detection on the VR game device to obtain the posture azimuth angle information of the VR game device at the wearer's head; Based on the posture azimuth angle information of the VR game device, adjusting the shooting direction angle of the camera unit to the external environment, so that the camera unit shoots an external environment image that meets the preset azimuth orientation condition; The image rendering and processing module is used to perform rendering and processing on the external environment image collected by the camera unit to obtain a virtual environment image, including: Performing pixel contour recognition processing on the external environment image to obtain the background area pixel contour feature information and non-background area pixel contour feature information of the external environment image; Based on the background area pixel contour feature information and non-background area pixel contour feature information, performing contour edge sharpening rendering processing on the background area of the external environment image and performing contour edge blurring rendering processing on the non-background area of the external environment image to obtain a virtual environment image.

8. The display control system of a VR game device based on real-time images according to claim 6, wherein: The virtual game scene image generation module is used to process the virtual environment image into a virtual game scene image based on the real-time game scene attribute information of the VR game device, including: Based on the real-time game scene type attribute information of the VR game device, selecting a number of game object elements from the game scene database and loading the game object elements into the virtual environment image to obtain a virtual game scene image; The virtual game scene image recognition module is used to recognize the virtual game scene image and determine the key image area and non-key image area included in the virtual game scene image, including: Performing game object element distribution position recognition on the virtual game scene image to determine the game object element distribution density of the virtual game scene image; If the game object element distribution density is greater than or equal to the preset density threshold, determining the area corresponding to the virtual game scene image as the key image area; Otherwise, determining the area corresponding to the virtual game scene image as the non-key image area.

9. The display control system of a VR game device based on real-time images according to claim 6, wherein: The optimal viewing area determination module is used to obtain the line-of-sight state information of the wearer of the VR game device, and based on the line-of-sight state information, determine the optimal game image viewing area of the wearer, including: Obtain the field-of-view range information of each of the left and right eyes of the wearer of the VR game device, compare the field-of-view range information of each of the left and right eyes, and obtain the binocular overlapping field-of-view range area of the wearer, which is used as the optimal game image viewing area of the wearer; The projection adjustment module is used to adjust the projection state parameters of the virtual game scene image to the wearer based on the optimal game image viewing area, so that the key image area and the optimal game image viewing area meet the preset relative position conditions, including: Based on the key image area and the non-key image area included in the virtual game scene image, determine the screen position information of the key image area in the virtual game scene image; Based on the optimal game image viewing area and the screen position information, adjust the projection direction and / or projection focal depth of the virtual game scene image to the wearer, so that the horizontal offset distance and the vertical offset distance between the center of the key image area and the center of the optimal game image viewing area are both less than the preset distance threshold.

Images