Virtual reality device and image processing method

Abstract

Embodiments of the present application provide a virtual reality device and an image processing method. The virtual reality device includes a first camera and a second camera. When the field of view angle of the first camera is greater than the field of view angle of the second camera, the target number of pixels in the horizontal and vertical directions of the first camera is determined based on the field of view angles of the first camera and the second camera, and the effective number of pixels of the first camera in the horizontal and vertical directions. Based on the target number of pixels in the horizontal and vertical directions of the first camera, a target image is selected from the image captured by the first camera, and after pre-processing, the target image is sent to the first display screen of the virtual reality device for display. The image captured by the second camera is sent to the second display screen of the virtual reality device for display. This allows the first display screen and the second display screen to display two identical images, thereby improving the user's visual experience.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to a virtual reality device and an image processing method. Background Art

[0002] With the advancement of technology, virtual reality devices have gradually entered daily life, bringing people a new visual experience.

[0003] Currently, some VR devices feature two cameras. Due to manufacturing constraints, the field of view (FOV) of different camera modules can vary by ±3°. When both cameras are used simultaneously, the FOV can vary by up to 6°. This can cause objects to appear larger on one side than on the other when the same scene is presented to both eyes, negatively impacting the user experience. Summary of the Invention

[0004] The embodiments of the present application provide a virtual reality device and an image processing method, which can solve the technical problem that the sizes of objects captured by two cameras of the virtual reality device are different due to the difference in the field of view angles of the two cameras.

[0005] In some embodiments, the present application provides a virtual reality device, including:

[0006] The first camera and the second camera are respectively configured to capture images; wherein the field of view of the first camera is greater than the field of view of the second camera;

[0007] a first display screen and a second display screen, wherein the first display screen is configured to display an image captured by the first camera, and the second display screen is configured to display an image captured by the second camera;

[0008] A controller connected to the first camera, the second camera, the first display screen, and the second display screen, respectively, the controller being configured to:

[0009] Determining target numbers of pixels of the first camera in the horizontal direction and the vertical direction according to the field of view angles of the first camera and the second camera, and the effective numbers of pixels of the first camera in the horizontal direction and the vertical direction;

[0010] Selecting a target image from images captured by the first camera according to target pixel numbers of the first camera in the horizontal direction and the vertical direction, and preprocessing the target image;

[0011] The pre-processed target image is sent to the first display screen for display, and the image captured by the second camera is sent to the second display screen for display.

[0012] In a feasible implementation manner, the controller is configured to:

[0013] The target number of pixels P of the first camera in the horizontal direction is determined as follows: lx :

[0014]

[0015] The target number of pixels P of the first camera in the vertical direction is determined as follows: lw :

[0016]

[0017] Among them, P l represents the number of effective pixels of the first camera in the horizontal direction, P w Indicates the number of effective pixels of the first camera in the vertical direction, FOV max Indicates the field of view of the first camera, FOV min Indicates the field of view of the second camera.

[0018] In a feasible implementation manner, the controller is configured to:

[0019] determining an origin of the target image in an image captured by the first camera according to the effective number of pixels of the first camera in the horizontal direction and the vertical direction and the target number of pixels of the first camera in the horizontal direction and the vertical direction;

[0020] Based on the origin of the target image, the target image is selected from the images captured by the first camera.

[0021] In a feasible implementation manner, the controller is specifically configured to:

[0022] The position (Ox, Oy) of the origin of the target image in the image captured by the first camera is determined in the following manner:

[0023]

[0024] Among them, P l represents the number of effective pixels of the first camera in the horizontal direction, P w represents the number of effective pixels of the first camera in the vertical direction; P lx represents the target number of pixels of the first camera in the horizontal direction; Plw Indicates the target number of pixels in the vertical direction of the first camera.

[0025] In a feasible implementation manner, the controller is configured to:

[0026] After selecting a target image from the images captured by the first camera, the target image is preprocessed using a preset transformation matrix so that the number of pixels of the processed target image in the horizontal direction is equal to the effective number of pixels of the first camera in the horizontal direction, and the number of pixels of the processed target image in the vertical direction is equal to the effective number of pixels of the first camera in the vertical direction.

[0027] In a feasible implementation manner, the first camera and the second camera have the same shooting parameters, and the shooting parameters include a frame rate and an exposure time;

[0028] The controller is further configured to:

[0029] Simultaneously controlling the first camera and the second camera to shoot a preset reference object, wherein the preset reference object includes any one of a stopwatch and a marquee;

[0030] Acquire a first image captured by the first camera and a second image captured by the second camera;

[0031] adjusting shooting control parameters of the first camera and / or the second camera according to states of reference objects in the first image and the second image;

[0032] Return to the step of simultaneously controlling the first camera and the second camera to shoot the preset reference object until the state of the reference object in the first image most recently shot by the first camera is the same as the state of the reference object in the second image most recently shot by the second camera.

[0033] In a feasible implementation manner, the controller includes a first synchronization signal interface and a second synchronization signal interface, the first synchronization signal interface is connected to the first camera, and the second synchronization signal interface is connected to the second camera; the controller is configured to:

[0034] According to the status of the reference objects in the first image and the second image, adjust the synchronization signal sent by the controller to the first camera using the first synchronization signal interface, and / or adjust the synchronization signal sent by the controller to the second camera using the second synchronization signal interface.

[0035] In a feasible implementation manner, a power management chip is further included, wherein the power management chip is respectively connected to the first camera, the second camera, the first display screen, the second display screen and the controller;

[0036] The controller includes an image processor, a memory and a display controller;

[0037] The image processor is configured to receive image data captured by the first camera and the second camera, and to process the received image data in real time and then transmit the processed image data to the memory for storage;

[0038] The display controller is configured to extract the image data collected by the first camera and the second camera stored in the memory, and transmit the image data to the first display screen and the second display screen for display respectively.

[0039] In some embodiments, embodiments of the present application provide an image processing method applied to a virtual reality device, the virtual reality device including a first camera and a second camera, the field of view of the first camera being greater than the field of view of the second camera; the method comprising:

[0040] Determining target numbers of pixels of the first camera in the horizontal direction and the vertical direction according to the field of view angles of the first camera and the second camera, and the effective numbers of pixels of the first camera in the horizontal direction and the vertical direction;

[0041] Selecting a target image from images captured by the first camera according to target pixel numbers of the first camera in the horizontal direction and the vertical direction, and preprocessing the target image;

[0042] The pre-processed target image is sent to the first display screen of the virtual reality device for display, and the image captured by the second camera is sent to the second display screen of the virtual reality device for display.

[0043] In a feasible implementation manner, determining the target number of pixels of the first camera in the horizontal direction and the vertical direction based on the field of view angles of the first camera and the second camera and the effective number of pixels of the first camera in the horizontal direction and the vertical direction includes:

[0044] The target number of pixels P of the first camera in the horizontal direction is determined as follows: lx :

[0045]

[0046] The target number of pixels P of the first camera in the vertical direction is determined as follows:lw :

[0047]

[0048] Among them, P l represents the number of effective pixels of the first camera in the horizontal direction, P w Indicates the number of effective pixels of the first camera in the vertical direction, FOV max Indicates the field of view of the first camera, FOV min Indicates the field of view of the second camera.

[0049] In a feasible implementation manner, selecting a target image from images captured by the first camera according to target pixel numbers of the first camera in the horizontal direction and the vertical direction includes:

[0050] determining an origin of the target image in an image captured by the first camera according to the effective number of pixels of the first camera in the horizontal direction and the vertical direction and the target number of pixels of the first camera in the horizontal direction and the vertical direction;

[0051] Based on the origin of the target image, the target image is selected from the images captured by the first camera.

[0052] In a feasible implementation manner, determining the origin of the target image in the image captured by the first camera based on the effective number of pixels of the first camera in the horizontal direction and the vertical direction and the target number of pixels of the first camera in the horizontal direction and the vertical direction includes:

[0053] The position (Ox, Oy) of the origin of the target image in the image captured by the first camera is determined in the following manner:

[0054]

[0055] Among them, P l represents the number of effective pixels of the first camera in the horizontal direction, P w represents the number of effective pixels of the first camera in the vertical direction; P lx represents the target number of pixels of the first camera in the horizontal direction; P lw Indicates the target number of pixels in the vertical direction of the first camera.

[0056] In a feasible implementation manner, the preprocessing of the target image includes:

[0057] After selecting a target image from the images captured by the first camera, the target image is preprocessed using a preset transformation matrix so that the number of pixels of the processed target image in the horizontal direction is equal to the effective number of pixels of the first camera in the horizontal direction, and the number of pixels of the processed target image in the vertical direction is equal to the effective number of pixels of the first camera in the vertical direction.

[0058] In a feasible implementation, it further includes:

[0059] Simultaneously controlling the first camera and the second camera to shoot a preset reference object, wherein the preset reference object includes any one of a stopwatch and a marquee;

[0060] Acquire a first image captured by the first camera and a second image captured by the second camera;

[0061] adjusting shooting control parameters of the first camera and / or the second camera according to states of reference objects in the first image and the second image;

[0062] Return to the step of simultaneously controlling the first camera and the second camera to shoot the preset reference object until the state of the reference object in the first image most recently shot by the first camera is the same as the state of the reference object in the second image most recently shot by the second camera.

[0063] The virtual reality device and image processing method provided by the embodiment of the present application, when the field of view angle of the first camera of the virtual reality device is greater than the field of view angle of the second camera, the target number of pixels of the first camera in the horizontal direction and the vertical direction is determined according to the field of view angles of the first camera and the second camera, and the effective number of pixels of the first camera in the horizontal direction and the vertical direction; according to the target number of pixels of the first camera in the horizontal direction and the vertical direction, a target image is selected from the image captured by the first camera, and after pre-processing the target image, it is sent to the first display screen of the virtual reality device for display, and the image captured by the second camera is sent to the second display screen of the virtual reality device for display. Since the scene range covered by the above-mentioned target image is determined based on the field of view angles of the first camera and the second camera, the above-mentioned target image and the image captured by the second camera can cover the same scene, and after pre-processing the target image, the first display screen and the second display screen can display two exactly the same images, thereby being able to more realistically restore the surrounding scene, reduce dizziness, and effectively enhance the user's visual experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments of the present application or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0065] Figure 1 A schematic diagram of the structure of a virtual reality device provided in an embodiment of the present application;

[0066] Figure 2 A schematic diagram of the module structure of a virtual reality device provided in an embodiment of the present application;

[0067] Figure 3 Schematic diagram of the effective imaging area of ​​cameras in the horizontal direction with different field of view angles under normal circumstances;

[0068] Figure 4 Schematic diagram comparing the horizontal field of view angle and actual projection length of the first camera in an embodiment of the present application;

[0069] Figure 5 Schematic diagram of the position of the target image in the image captured by the first camera in an embodiment of the present application;

[0070] Figure 6 Schematic diagram of comparison of target images corresponding to the first camera and the second camera in an embodiment of the present application;

[0071] Figure 7 This is a schematic diagram of the synchronous control architecture of the camera in an embodiment of the present application;

[0072] Figure 8 This is a flow chart of synchronizing the completion time of the first camera and the second camera in an embodiment of the present application;

[0073] Figure 9 This is a schematic flow chart of the steps of an image processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. In addition, although the disclosure in this application is introduced according to one or more exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete implementation method separately.

[0075] It should be noted that the brief descriptions of terms in this application are only for the purpose of facilitating the understanding of the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.

[0076] In the specification and claims of this application, as well as in the accompanying drawings, the terms "first," "second," and the like are used to distinguish similar or similar objects or entities and are not necessarily intended to limit a particular order or precedence, unless otherwise noted. It should be understood that such terms are interchangeable where appropriate, e.g., embodiments of this application can be implemented in an order other than that shown or described in the drawings or descriptions.

[0077] In addition, the terms "comprises" and "comprising" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0078] The term "module" as used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0079] At present, some virtual reality devices have two cameras and two display screens. When the user is in use, the two display screens show the images of the left and right eyes respectively. After the user's two eyes obtain this different information, a three-dimensional picture can be generated in the mind, which can be applied to virtual driving, virtual cities, virtual games and other scenarios.

[0080] For example, refer to Figure 1 , Figure 1The following is a schematic diagram of the structure of a virtual reality device provided in an embodiment of the present application. In some embodiments, the virtual reality device includes a host 100, which is equipped with two cameras, namely a first camera 31 and a second camera 32, and two display screens, namely a first display screen 41 and a second display screen 42. The first camera 31 and the second camera 32 can be used to capture left-eye images and right-eye images, respectively, and then send the captured left-eye images and right-eye images to the first display screen 41 and the first display screen 42 for display.

[0081] Due to manufacturing constraints, the field of view (FOV) of different camera modules can vary by ±3°. When a VR device's two cameras are used simultaneously, the FOVs can vary by up to 6°. This can cause one side of the user's eyes to see objects larger than the other, resulting in mismatched images captured by the two cameras during image alignment, negatively impacting the user experience.

[0082] In order to avoid the negative impact of the camera's field of view, in some embodiments, only one camera is usually used for video capture, and the same captured video is then transmitted to display screens in front of the user's two eyes. Although this method can avoid the situation where one side of the object observed by the user's two eyes is larger and the other side is smaller, since the images seen by the user's two eyes are completely consistent and there is no parallax, the scene observed by the user is completely different from the real scene, and it usually causes dizziness and lacks a sense of three-dimensionality.

[0083] In order to solve the above technical problems, an embodiment of the present application provides a virtual reality device, which has two cameras. The two cameras can simulate the user's two eyes to synchronously capture images. At the same time, according to the difference in field of view angles between the two cameras, the image captured by one of the cameras is adjusted so that the two display screens corresponding to the two cameras can display two exactly the same images, thereby more realistically restoring the surrounding scene, reducing dizziness, and effectively improving the user's visual experience.

[0084] Reference Figure 2 , Figure 2 This is a schematic diagram of the module structure of a virtual reality device provided in an embodiment of the present application. Figure 2 As shown, the virtual reality device provided in the embodiment of the present application includes a power management chip 10, a controller 20, a first camera 31, a second camera 32, a first display screen 41, and a second display screen 42.

[0085] Among them, the power management chip 10 is used to convert the battery power into corresponding sub-power supplies according to the requirements of each module, and supply power to each module respectively. For example, the camera requires AVDD / DVDD / IOVDD power supplies, and the main controller 204 requires 3.3V / 1.8V / 1.2V / 0.9V power supplies.

[0086] The controller 20 includes an image processor 201 , a memory 202 , a display controller 203 , and a main controller 204 .

[0087] The image processor 201 is used to receive image data from the first camera 31 and the second camera 32 , perform real-time processing on the image data, and then transmit the processed image data to the memory 202 for storage.

[0088] The memory 202 is used to receive image data from the image processor 201 in real time and store the image data.

[0089] The display controller 203 is used to extract the image data to be displayed from the memory 202 and present the left and right images on the first display screen 41 and the second display screen 42 respectively as required.

[0090] The main controller 204 is used to control the operation of the entire system. For example, after power-on, the main controller 204 first initializes the minimum system. After completion, the image processor 201, memory 202, and display controller 203 run again. Then, the image sensor is configured through Reset / MCLK / I2C to complete the camera initialization. According to environmental changes, the camera's operating status is controlled in real time through I2C, and the display screen initialization is completed.

[0091] The first camera 31 is used to receive control instructions and configurations from the main controller 204, and complete image acquisition of the left perspective according to the corresponding configuration information and transmit it to the image processor 201; the second camera 32 is used to receive control instructions and configurations from the main controller 204, and complete image acquisition of the right perspective according to the corresponding configuration information and transmit it to the image processor 201.

[0092] The first display screen 41 is used to display images collected from the first camera 31 ; the second display screen 42 is used to display images collected from the second camera 32 .

[0093] In the embodiment of the present application, it is assumed that the first camera 31 and the second camera 32 are respectively located on the left and right sides of the virtual reality device after being assembled. If the field of view of the first camera 31 is larger than the field of view of the second camera 32, the controller 20 is configured as follows:

[0094] Based on the field of view angles of the first camera 31 and the second camera 32, and the effective number of pixels of the first camera 31 in the horizontal and vertical directions, the target number of pixels of the first camera 31 in the horizontal and vertical directions is determined; based on the target number of pixels of the first camera 31 in the horizontal and vertical directions, a target image is selected from the image captured by the first camera 31, and after preprocessing the target image, the preprocessed target image is sent to the first display screen 41 for display, and the image captured by the second camera 32 is sent to the second display screen 42 for display.

[0095] In order to better understand the embodiments of the present application, the horizontal direction is used as an example for explanation below:

[0096] Assume that the field of view of the first camera 31 is FOV max , the field of view of the second camera 32 is FOV min , FOV max FOV min .

[0097] Reference Figure 3 , Figure 3 Schematic diagram of the effective imaging area in the horizontal direction of cameras with different field of view angles under normal circumstances.

[0098] In the embodiment of the present application, the widths of the effective imaging areas of the first camera 31 and the second camera 32 in the horizontal direction are the same, which can be assumed to be L.

[0099] Reference Figure 4 , Figure 4 Schematic diagram comparing the horizontal field of view angle and actual projection length of the first camera in the embodiment of the present application.

[0100] exist Figure 4 In the figure, the incident light of the first person view (FPV) of the first camera 31 is shown as a solid line, and its actual projection length on the image sensor is L; the field of view of the first camera 31 adopts the same field of view as that of the second camera 32, that is, FOV min When the FPV incident light is shown as a dotted line, the actual projection length of the first camera 31 on the image sensor should be L x , and the following relationship exists:

[0101]

[0102] Can be further converted to:

[0103]

[0104] Since the pixels of image sensors are all the same size and are evenly distributed on the surface of the image sensor, the following relationship exists:

[0105]

[0106] Among them, P lx represents the target pixel number of the first camera 31 in the horizontal direction, P l Indicates the number of effective pixels of the first camera 31 in the horizontal direction.

[0107] Thus, the target pixel number P of the first camera 31 in the horizontal direction can be calculated. lx for:

[0108]

[0109] Substituting the above Lx into the equation, we can get:

[0110]

[0111] Similarly, the target number of pixels P of the first camera 31 in the vertical direction can be calculated lw for:

[0112]

[0113] Among them, P w Indicates the number of effective pixels of the first camera 31 in the vertical direction.

[0114] Based on the above description, the actual size of the target image corresponding to the first camera 31 is:

[0115] Px=[P lx ,P lw ]=[P l *FOV min / FOV max ,P w *FOV min / FOV max ]

[0116] In an embodiment of the present application, after determining the actual size of the above-mentioned target image, the target image is selected from the image captured by the first camera 31, and after pre-processing the selected target image, the pre-processed target image is sent to the first display screen 41 for display, and the image captured by the second camera 32 is sent to the second display screen 42 for display.

[0117] It can be understood that since the above-mentioned target image is determined based on the field of view of the first camera 31 and the second camera 32, the above-mentioned target image and the image captured by the second camera 32 can cover the same scene. After preprocessing the target image, the first display screen 41 and the second display screen 42 can display two exactly the same images, thereby being able to more realistically restore the surrounding scene, reduce dizziness, and effectively enhance the user's visual experience.

[0118] Based on the contents described in the above embodiments, in some embodiments, after determining the target number of pixels of the first camera 31 in the horizontal and vertical directions, the origin of the target image is determined in the image captured by the first camera 31 according to the effective number of pixels of the first camera 31 in the horizontal and vertical directions, and the target number of pixels of the first camera 31 in the horizontal and vertical directions; based on the origin of the target image, the target image is selected from the image captured by the first camera 31.

[0119] In a feasible implementation, the position (Ox, Oy) of the origin of the target image in the image captured by the first camera 31 can be determined in the following manner:

[0120]

[0121] Among them, P l represents the number of effective pixels of the first camera 31 in the horizontal direction, P w P represents the number of effective pixels of the first camera 31 in the vertical direction; lx represents the target pixel number of the first camera 31 in the horizontal direction; P lw Indicates the target number of pixels of the first camera 31 in the vertical direction.

[0122] In order to better understand the embodiments of the present application, refer to Figure 5 , Figure 5 Schematic diagram of the position of the target image in the image captured by the first camera in an embodiment of the present application.

[0123] In the embodiment of the present application, Ox=[(P l -P lx ) / 2,(P w -P lw ) / 2] as the origin, with P lx As the length, P lw is the width, and a target image is selected from the images captured by the first camera 31.

[0124] Reference Figure 6 , Figure 6 Schematic diagram of comparison of target images corresponding to the first camera and the second camera in the embodiment of the present application.

[0125] exist Figure 6 For the first camera 301, the target number of pixels in the horizontal direction is P lx , and for the second camera 302, its target number of pixels in the horizontal direction is P l This ensures that the images captured by the first camera 301 and the second camera 32 can cover the same scene, avoiding the problem that one side of the object observed by the user's eyes is larger and the other side is smaller.

[0126] Based on the contents described in the above embodiments, in some embodiments, after selecting a target image from the image captured by the first camera 31, the target image is preprocessed using a preset conversion matrix so that the number of pixels of the processed target image in the horizontal direction is equal to the effective number of pixels of the first camera 31 in the horizontal direction, and the number of pixels of the processed target image in the vertical direction is equal to the effective number of pixels of the first camera 31 in the vertical direction.

[0127] That is, the image size of the processed target image is P = [P l ,P w ], the first display screen 41 and the second display screen 42 can display two completely identical images, thereby more realistically restoring the surrounding scene, reducing dizziness, and effectively improving the user's visual experience.

[0128] Based on the content described in the above embodiments, due to the different data processing capabilities of the first camera 31 and the second camera 32, as well as differences in the mainboard wiring, the data exposure, processing and transmission processes will cause the two cameras to complete at different times, ultimately forming two different photos. If these two photos are displayed to the user's two eyes at the same time, the scenes in the two eyes will be inconsistent, affecting the user experience.

[0129] In some embodiments of the present application, the first camera 31 and the second camera 32 can be turned on first, and then the shooting parameters such as the frame rate and exposure time of the first camera 31 and the second camera 32 can be set to the same. Then, the completion time of the first camera 31 and the second camera 32 can be synchronized in the following manner.

[0130] Reference Figure 7 , Figure 7 Schematic diagram of the synchronous control architecture of the camera in an embodiment of the present application.

[0131] exist Figure 7 In the embodiment, the first camera 31 and the second camera 32 are both connected to the controller 20 via MIPI (Mobile Industry Processor Interface) and I2C bus.

[0132] MIPI is an open standard developed by the MIPI Alliance for mobile application processors. MIPI interfaces offer advantages such as high speed, large data transfer volumes, low power consumption, and excellent anti-interference capabilities.

[0133] In some embodiments, the Camera Serial Interface 2 (CSI-2) in MIPI can be used. The CSI-2 camera data transmission process uses data differential signals to transmit pixel values ​​in the video. At the same time, the CSI-2 transmission interface can be very flexibly simplified or expanded. For application scenarios with fewer interfaces, the CSI-2 interface can use only one set of differential data signal lines and one set of differential clock lines to complete the camera's data serial transmission process, thus reducing the load and meeting a certain transmission rate. For large-array CCD cameras, the CSI-2 interface can also expand its differential data lines to meet the high-speed requirements of parallel transmission of multiple sets of data lines.

[0134] In some embodiments, the controller 20 further includes a first synchronization signal interface and a second synchronization signal interface, the first synchronization signal interface is connected to the first camera 31 , and the second synchronization signal interface is connected to the second camera 32 .

[0135] The controller 20 can use the first synchronization signal interface to send a synchronization signal FSIN1 to the first camera 31, and use the second synchronization signal interface to send a synchronization signal FSIN2 to the second camera 32. The first camera 31 and the second camera 32 synchronously output the collected image data according to the received synchronization signal FSIN.

[0136] In the embodiment of the present application, by adjusting the synchronization signal FSIN1 and / or the synchronization signal FSIN2, the first camera 31 and the second camera 32 can simultaneously output images captured at the same time, thereby ensuring that the scenes viewed by the user's two eyes remain consistent.

[0137] Reference Figure 8 , Figure 8 This is a flow chart of synchronizing the completion time of the first camera and the second camera in an embodiment of the present application. The above method includes:

[0138] 801 . Simultaneously control the first camera 301 and the second camera 302 to shoot a preset reference object.

[0139] Optionally, the preset reference object may be any one of a stopwatch and a marquee.

[0140] 802 : Acquire a first image captured by the first camera 301 and a second image captured by the second camera 302 .

[0141] 803. Determine whether the states of the reference objects in the first image and the second image are the same. If not, execute 804; if so, execute 805.

[0142] 804. Adjust the shooting control parameters of the first camera 301 and / or the second camera 302 according to the states of the reference objects in the first image and the second image. Return to 801.

[0143] For example, assuming the reference object is a ticker tape, the ticker tape includes a plurality of light-emitting diodes, with adjacent diodes lighting up after a preset interval. In one feasible implementation, the number of lit diodes in the first and second images can be identified. If the number of lit diodes in the first image is greater than the number of lit diodes in the second image, the FSIN1 time of the first camera 301 should be advanced or the FSIN2 time of the second camera 302 should be delayed, so that the first and second cameras 301, 302 can output image data simultaneously.

[0144] 805 : Save the shooting control parameters of the first camera 301 and the second camera 302 .

[0145] The above method provided by the embodiment of the present application can make the images captured by the first camera 301 and the second camera 302 at the same time completely consistent, thereby being able to more realistically restore the surrounding scene, reduce dizziness, and effectively improve the user's visual experience.

[0146] Based on the content described in the above embodiments, an embodiment of the present application further provides an image processing method, which is applied to a virtual reality device. The virtual reality device includes a first camera and a second camera, wherein the field of view angle of the first camera is greater than the field of view angle of the second camera.

[0147] Reference Figure 9 , Figure 9 The following is a flow chart of steps of an image processing method provided in an embodiment of the present application. In one feasible implementation, the method includes:

[0148] S901: Determine target numbers of pixels of the first camera in the horizontal and vertical directions according to the field of view angles of the first camera and the second camera, and the effective numbers of pixels of the first camera in the horizontal and vertical directions.

[0149] S902: Select a target image from the images captured by the first camera according to the field of view of the first camera and the second camera, and the target number of pixels of the first camera in the horizontal direction and the vertical direction, and preprocess the target image.

[0150] S903: Based on the field of view of the first camera and the second camera, the first camera sends the pre-processed target image to the first display screen of the virtual reality device for display in the horizontal direction, and sends the image captured by the second camera to the second display screen of the virtual reality device for display.

[0151] The image processing method provided in the embodiment of the present application, since the scene range covered by the above-mentioned target image is jointly determined based on the field of view angles of the first camera and the second camera, the above-mentioned target image and the image captured by the second camera can cover the same scene, and after preprocessing the target image, the first display screen and the second display screen can display two exactly the same images, thereby being able to more realistically restore the surrounding scene, reduce dizziness, and effectively enhance the user's visual experience.

[0152] Based on the contents described in the above embodiments, in some embodiments, the target number of pixels P of the first camera in the horizontal direction can be determined in the following manner: lx :

[0153]

[0154] The target number of pixels P of the first camera in the vertical direction is determined as follows: lw :

[0155]

[0156] Among them, P l Indicates the number of effective pixels of the first camera in the horizontal direction, P w Indicates the number of effective pixels of the first camera in the vertical direction, FOV max Indicates the field of view of the first camera, FOV min Indicates the field of view of the second camera.

[0157] In a feasible implementation manner, selecting the target image from the images captured by the first camera according to the target number of pixels of the first camera in the horizontal direction and the vertical direction includes:

[0158] According to the effective number of pixels of the first camera in the horizontal and vertical directions, and the target number of pixels of the first camera in the horizontal and vertical directions, the origin of the target image is determined in the image captured by the first camera; based on the origin of the target image, the target image is selected from the image captured by the first camera.

[0159] In a feasible implementation, determining the origin of the target image in the image captured by the first camera based on the effective number of pixels of the first camera in the horizontal and vertical directions and the target number of pixels of the first camera in the horizontal and vertical directions includes:

[0160] Determine the position (Ox, Oy) of the origin of the target image in the image captured by the first camera as follows:

[0161]

[0162] Among them, P l Indicates the number of effective pixels of the first camera in the horizontal direction, P w Indicates the number of effective pixels of the first camera in the vertical direction; P lx Indicates the target number of pixels in the horizontal direction of the first camera; P lw Indicates the target number of pixels in the vertical direction of the first camera.

[0163] In a feasible implementation, preprocessing the target image includes:

[0164] After selecting a target image from the images captured by the first camera, the target image is preprocessed using a preset transformation matrix so that the number of pixels of the processed target image in the horizontal direction is equal to the effective number of pixels of the first camera in the horizontal direction, and the number of pixels of the processed target image in the vertical direction is equal to the effective number of pixels of the first camera in the vertical direction.

[0165] In a feasible implementation, it further includes:

[0166] Simultaneously controlling the first camera and the second camera to shoot a preset reference object, where the preset reference object includes any one of a stopwatch and a marquee;

[0167] Acquire a first image captured by a first camera and a second image captured by a second camera;

[0168] adjusting shooting control parameters of the first camera and / or the second camera according to the states of the reference objects in the first image and the second image;

[0169] Return to the step of simultaneously controlling the first camera and the second camera to shoot the preset reference object until the state of the reference object in the first image most recently shot by the first camera and the state of the reference object in the second image most recently shot by the second camera are the same.

[0170] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A virtual reality device, characterized in that: include: The first camera and the second camera are each configured to capture an image; wherein the field of view of the first camera is greater than the field of view of the second camera; and the width of the effective imaging area of ​​the first camera and the second camera in the horizontal direction is the same; a first display screen and a second display screen, wherein the first display screen is configured to display an image captured by the first camera, and the second display screen is configured to display an image captured by the second camera; A controller connected to the first camera, the second camera, the first display screen, and the second display screen, respectively, the controller being configured to: Determining target numbers of pixels of the first camera in the horizontal direction and the vertical direction according to the field of view angles of the first camera and the second camera, and the effective numbers of pixels of the first camera in the horizontal direction and the vertical direction; Selecting a target image from images captured by the first camera based on the target number of pixels in the horizontal and vertical directions of the first camera, and preprocessing the target image so that the number of pixels in the horizontal direction of the processed target image is equal to the effective number of pixels in the horizontal direction of the first camera, and the number of pixels in the vertical direction of the processed target image is equal to the effective number of pixels in the vertical direction of the first camera; The pre-processed target image is sent to the first display screen for display, and the image captured by the second camera is sent to the second display screen for display.

2. The virtual reality device according to claim 1, wherein: The controller is configured to: The target number of pixels P of the first camera in the horizontal direction is determined as follows: lx : The target number of pixels P of the first camera in the vertical direction is determined as follows: lw : Among them, P l represents the number of effective pixels of the first camera in the horizontal direction, P w Indicates the number of effective pixels of the first camera in the vertical direction, FOV max Indicates the field of view of the first camera, FOV min Indicates the field of view of the second camera.

3. The virtual reality device according to claim 1, wherein: The controller is configured to: determining an origin of the target image in an image captured by the first camera according to the effective number of pixels of the first camera in the horizontal direction and the vertical direction and the target number of pixels of the first camera in the horizontal direction and the vertical direction; Based on the origin of the target image, the target image is selected from the images captured by the first camera.

4. The virtual reality device according to claim 3, wherein: The controller is specifically configured to: The position (Ox, Oy) of the origin of the target image in the image captured by the first camera is determined in the following manner: Among them, P l represents the number of effective pixels of the first camera in the horizontal direction, P w represents the number of effective pixels of the first camera in the vertical direction; P lx represents the target number of pixels of the first camera in the horizontal direction; P lw Indicates the target number of pixels in the vertical direction of the first camera.

5. The virtual reality device according to claim 1, wherein: The first camera and the second camera have the same shooting parameters, which include frame rate and exposure time; The controller is further configured to: Simultaneously controlling the first camera and the second camera to shoot a preset reference object, wherein the preset reference object includes any one of a stopwatch and a marquee; Acquire a first image captured by the first camera and a second image captured by the second camera; adjusting shooting control parameters of the first camera and / or the second camera according to states of reference objects in the first image and the second image; Return to the step of simultaneously controlling the first camera and the second camera to shoot the preset reference object until the state of the reference object in the first image most recently shot by the first camera is the same as the state of the reference object in the second image most recently shot by the second camera.

6. The virtual reality device according to claim 5, characterized in that The controller includes a first synchronization signal interface and a second synchronization signal interface, the first synchronization signal interface is connected to the first camera, and the second synchronization signal interface is connected to the second camera; the controller is configured to: According to the status of the reference objects in the first image and the second image, adjust the synchronization signal sent by the controller to the first camera using the first synchronization signal interface, and / or adjust the synchronization signal sent by the controller to the second camera using the second synchronization signal interface.

7. The virtual reality device according to claim 1, wherein: It also includes a power management chip, which is connected to the first camera, the second camera, the first display screen, the second display screen and the controller respectively; The controller includes an image processor, a memory and a display controller; The image processor is configured to receive image data captured by the first camera and the second camera, and to process the received image data in real time and then transmit the processed image data to the memory for storage; The display controller is configured to extract the image data collected by the first camera and the second camera stored in the memory, and transmit the image data to the first display screen and the second display screen for display respectively.

8. An image processing method, characterized in that: Applied to a virtual reality device, the virtual reality device comprising a first camera and a second camera, wherein the field of view of the first camera is greater than the field of view of the second camera; The first camera and the second camera have the same width of effective imaging area in the horizontal direction; and the method includes: Determining target numbers of pixels of the first camera in the horizontal direction and the vertical direction according to the field of view angles of the first camera and the second camera, and the effective numbers of pixels of the first camera in the horizontal direction and the vertical direction; Selecting a target image from images captured by the first camera based on the target number of pixels in the horizontal and vertical directions of the first camera, and preprocessing the target image so that the number of pixels in the horizontal direction of the processed target image is equal to the effective number of pixels in the horizontal direction of the first camera, and the number of pixels in the vertical direction of the processed target image is equal to the effective number of pixels in the vertical direction of the first camera; The pre-processed target image is sent to the first display screen of the virtual reality device for display, and the image captured by the second camera is sent to the second display screen of the virtual reality device for display.

9. The method according to claim 8, characterized in that Also includes: Simultaneously controlling the first camera and the second camera to shoot a preset reference object, wherein the preset reference object includes any one of a stopwatch and a marquee; Acquire a first image captured by the first camera and a second image captured by the second camera; adjusting shooting control parameters of the first camera and / or the second camera according to states of reference objects in the first image and the second image; Return to the step of simultaneously controlling the first camera and the second camera to shoot the preset reference object until the state of the reference object in the first image most recently shot by the first camera is the same as the state of the reference object in the second image most recently shot by the second camera.

Images