Panoramic projection method, projection equipment and storage medium

Abstract

The invention relates to a panoramic projection method, projection equipment and a storage medium. The method comprises the following steps: acquiring panoramic content and attitude information, wherein the attitude information comprises attitude information received from a rotated control device; controlling the projection equipment to rotate according to the attitude information; determining a view field picture area from the panoramic content according to the attitude information; and projecting the picture content of the view field picture area to a projection plane corresponding to the projection equipment. By utilizing the method and the device, the projection equipment and the projected picture content can track and respond to the change of the viewing angle of the user, so that the viewing experience of the user on the panoramic content can be improved.

Description

Technical Field

[0001] The present application relates to the field of projection technology, and in particular to a panoramic projection method, a projection device and a storage medium. Background Art

[0002] Panoramic content technology brings immersive visual enjoyment to users. In related technologies, users watch panoramic content mainly in two ways, however, both of these methods have certain limitations.

[0003] The first way is to watch panoramic content through mobile phones, tablets and other devices. Due to the limitation of screen size, users often need to manually drag the screen to change the visual focus, which not only affects the smoothness of viewing, but also weakens the immersiveness of panoramic content.

[0004] The second way is to watch panoramic content by wearing special equipment such as virtual reality (VR) helmets and glasses. However, wearing these special equipment is often cumbersome, and wearing them for a long time may cause physical discomfort to users, such as dizziness and eye fatigue. In addition, the cost of these special equipment is relatively high and difficult to popularize. Summary of the invention

[0005] The present application discloses a panoramic projection method, a projection device and a storage medium, which can enable the projection device and the projected picture content to track and respond to changes in the user's viewing angle, thereby improving the user's viewing experience of the panoramic content.

[0006] In a first aspect, the present application relates to a panoramic projection method, comprising: acquiring panoramic content and posture information, wherein the posture information includes posture information received from a rotated control device, controlling the projection device to rotate according to the posture information so that the posture of the rotated projection device matches the posture of the rotated control device, determining a field of view image area from the panoramic content according to the posture information, rendering image content according to the field of view image area, and projecting the image content to a projection surface corresponding to the projection device, so that the field of view image area matches the viewing angle corresponding to the posture information. Optionally, the step of controlling the projection device to rotate according to the posture information includes: determining rotation information corresponding to a rotation mechanism of the projection device according to the posture information, and controlling the projection device to rotate through the rotation mechanism based on the rotation information.

[0007] Optionally, the panoramic content includes panoramic content and panoramic pictures, and the step of determining the field of view area from the panoramic content based on the posture information includes: determining a rotation matrix based on the posture information, determining initial three-dimensional coordinates of a spatial point within the field of view of the projection device, and transforming the initial three-dimensional coordinates based on the rotation matrix to obtain target three-dimensional coordinates of the spatial point, and determining the field of view area from the panoramic content based on the target three-dimensional coordinates of the spatial point.

[0008] Optionally, the above-mentioned step of determining the field of view picture area from the panoramic content based on the target three-dimensional coordinates of the spatial point includes: determining the spherical coordinates corresponding to the spatial point according to the target three-dimensional coordinates, determining the plane coordinates corresponding to the spatial point according to the spherical coordinates, and determining the pixel area corresponding to the plane coordinates in the panoramic content as the field of view picture area.

[0009] Optionally, before the step of projecting the picture content of the field of view picture area to the projection surface corresponding to the projection device, the step further includes: rendering the picture content according to the field of view picture area.

[0010] Optionally, the panoramic projection method further includes: if updated posture information is received from the control device, controlling the rotated projection device to rotate according to the updated posture information and determining an updated field of view image area from the panoramic content.

[0011] Optionally, the step of acquiring the posture information further includes: acquiring the user's voice information, and adjusting the projection device and / or the projected image based on recognition of the voice information.

[0012] Optionally, the panoramic projection method further includes: calculating correction information based on multi-sensor data of the projection device, and correcting the projected picture using the correction information.

[0013] In a second aspect, the present application further provides a projection device, comprising: a processor, a memory, and an application, wherein the application is stored in the memory and configured to be executed by the processor to implement the panoramic projection method.

[0014] In a third aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor in a projection device, the panoramic projection method is implemented.

[0015] The present application relates to a panoramic projection method, a projection device and a storage medium. Among them, the above method controls the projection device to rotate by receiving the posture information from the rotated control device, so that the posture of the rotated projection device matches the posture of the rotated control device, thereby ensuring that the viewing angle of the rotated projection device matches the user viewing angle indicated by the posture information. According to the posture information, the field of view screen area that matches the user viewing angle indicated by the posture information can be determined from the panoramic content. The screen content is rendered according to the field of view screen area, and the screen content is projected onto the projection surface, so that the projected screen content matches the user viewing angle indicated by the posture information. By matching the viewing angle of the projection device, the projected screen content and the user viewing angle indicated by the posture information, the projection device and the projected screen content can track and respond to changes in the user viewing angle in real time, without the user manually dragging the screen to change the visual focus and without wearing specific equipment, so that the user's viewing freedom, comfort, immersion, and fluency of the panoramic content can be improved. In addition, compared with specific equipment such as VR, the popularity of projection equipment is relatively high, so the problem of high viewing threshold of panoramic content can be effectively solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 It is a flow chart of the panoramic projection method provided in an embodiment of the present application.

[0018] Figure 2 This is a flowchart of a method for determining a field of view image area provided in one embodiment of the present application.

[0019] Figure 3 It is a schematic diagram of the structure of the projection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0021] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0022] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0023] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0024] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0025] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0026] In the related art, users watch panoramic contents mainly in two ways, however, both ways have certain limitations.

[0027] The first way is to watch panoramic content through mobile phones, tablets and other devices. Due to the limitation of screen size, users often need to manually drag the screen to change the visual focus, which not only affects the smoothness of viewing, but also weakens the immersiveness of panoramic content.

[0028] The second way is to watch panoramic content by wearing special equipment such as virtual reality (VR) helmets and glasses. However, wearing these special equipment is often cumbersome, and wearing them for a long time may cause physical discomfort to users, such as dizziness and eye fatigue. In addition, the cost of these special equipment is relatively high and difficult to popularize.

[0029] To solve the above problems, an embodiment of the present application provides a panoramic projection method, which enables the projection device and the projected picture content to track and respond to changes in the user's viewing angle, thereby improving the user's viewing experience of the panoramic content.

[0030] The panoramic projection method provided in the embodiments of the present application can be applied to one or more projection devices. For a projection device that needs to perform panoramic projection, the function for panoramic projection provided by the method of the present application can be directly integrated on the projection device, or run on the projection device in the form of a software development kit (SDK).

[0031] like Figure 1 , is a flowchart of a panoramic projection method provided by an embodiment of the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted. The panoramic projection method is applied to a projection device. For example, Figure 3 Projection equipment in.

[0032] S11, obtaining panoramic content and posture information.

[0033] In some embodiments of the present application, panoramic content includes but is not limited to: panoramic videos and panoramic pictures. A panoramic video is a video with a 360-degree perspective or a perspective close to 360 degrees taken by a camera such as a panoramic camera, and a panoramic picture is an image with a 360-degree perspective or a perspective close to 360 degrees taken by a camera such as a wide-angle lens or a fisheye lens.

[0034] In some embodiments of the present application, the projection device may obtain panoramic content through a variety of methods, and the present application does not limit the method for obtaining the panoramic content.

[0035] For example, the panoramic content can be obtained by user input, or the projection device can read the video data or image data under the preset path as the panoramic content, or the projection device can control the camera device to shoot the scene, and obtain the video data and image data of the 360-degree viewing angle shot by the camera device as the panoramic content. Among them, the preset path can be set in advance, and this application does not limit this.

[0036] In some embodiments of the present application, the projection device may obtain the posture information through a variety of methods, and the present application does not limit the method for obtaining the posture information.

[0037] Exemplarily, the projection device may receive data such as Euler angles and quaternions sent from the rotated control device as posture information.

[0038] Among them, the projection device is connected to the control device in communication, and this application does not limit the communication connection method. For example, the projection device can be connected to the control device in communication through wireless fidelity (Wi-Fi), Bluetooth (BT), etc. The control device includes but is not limited to: a remote control, a mobile phone, and a computer. The posture information can be obtained by detecting the control device through a sensor in the control device. For example, if the control device is a remote control, the remote control may include an inertial measurement unit (IMU), and the projection device may receive the posture information sent by the remote control after rotating 30 degrees to the left. The posture information can be obtained by performing posture detection on the remote control through the inertial measurement unit. Euler angles may include pitch angle (Pitch), roll angle (Roll), and yaw angle (Yaw), etc.

[0039] Exemplarily, the user may directly input corresponding gesture information through any physical button or virtual control set in the projection device.

[0040] The above examples of methods for acquiring posture information are only examples and are not limited thereto in practical applications.

[0041] In this embodiment, the projection device can establish communication connections with various types of control devices such as remote controls and mobile phones, thereby realizing diversified interactions with users. This diversified interaction method can not only improve the flexibility of interaction, but also fully meet the actual needs of users.

[0042] S12, controlling the projection device to rotate according to the posture information.

[0043] In some embodiments of the present application, the projection device may be connected to a rotating mechanism, wherein the rotating mechanism may be a pan / tilt head or the like.

[0044] For example, the pan-tilt head may be a multi-axis (e.g., dual-axis) structure, which enables the pan-tilt head to rotate in the horizontal and vertical directions, thereby driving the projection device to rotate in all directions to achieve omnidirectional and multi-angle projection. The pan-tilt head may be driven by one or more driving mechanisms, wherein the driving mechanism may be a motor, etc., and the motor may be a stepping motor or a brushless DC motor, etc.

[0045] In some embodiments of the present application, controlling the projection device to rotate according to the posture information includes: determining rotation information corresponding to a rotation mechanism of the projection device according to the posture information, and controlling the projection device to rotate through the rotation mechanism based on the rotation information.

[0046] The rotation information may include information such as the rotation speed, rotation direction and rotation angle of a driving mechanism (eg, a motor) of the rotating mechanism.

[0047] Exemplarily, if the posture information is obtained through a rotated control device, the projection device can detect the current posture information of the projection device through sensors such as an inertial measurement unit in the projection device, determine the posture difference between the posture information obtained from the control device and the projection device, and determine the rotation information based on the posture difference.

[0048] The projection device can control the rotation of the driving mechanism of the rotating mechanism through pulse width modulation (PWM) or a closed-loop control algorithm based on the rotation information, so that the rotating mechanism drives the projection device to rotate.

[0049] In this embodiment, the posture information obtained from the control device can indicate the user's viewing angle. By controlling the projection device to rotate through the posture information received from the rotated control device, the posture of the rotated projection device can be matched with the posture of the rotated control device, thereby ensuring that the viewing angle of the rotated projection device matches the user's viewing angle indicated by the posture information, thereby achieving synchronization between the projection device and the user's gestures.

[0050] S13, determining a field of view area from the panoramic content according to the posture information.

[0051] In some embodiments of the present application, the projection device determines the field of view image area from the panoramic content based on the posture information, including: determining the rotation matrix based on the posture information, determining the initial three-dimensional coordinates of the spatial point within the field of view of the projection device, and transforming the initial three-dimensional coordinates based on the rotation matrix to obtain the target three-dimensional coordinates of the spatial point, and determining the field of view image area from the panoramic content based on the target three-dimensional coordinates of the spatial point.

[0052] Among them, the rotation matrix R can be obtained by the rotation matrix around the X axis , rotation matrix around the Y axis And the rotation matrix around the Z axis For example, if the attitude information includes pitch angle, roll angle and yaw angle, the projection device can determine the rotation matrix based on the roll angle , Determine the rotation matrix based on the pitch angle And determine the rotation matrix based on the yaw angle . Determine the rotation matrix based on the roll angle , Determine the rotation matrix based on the pitch angle And determine the rotation matrix based on the yaw angle The method can refer to the conversion formula between Euler angle and rotation matrix in the related technology.

[0053] For example, the projection device can rotate the matrix , rotation matrix and the rotation matrix Perform multiplication operation to obtain the rotation matrix R.

[0054] The field of view of the projection device may be a spatial area projected or displayed by the projection device, and the spatial points within the field of view may be all coordinate points within the spatial area or boundary coordinate points of the spatial area.

[0055] Exemplarily, the projection device may perform a multiplication operation on the rotation matrix and the three-dimensional coordinates of each spatial point to obtain the target three-dimensional coordinates of each spatial point.

[0056] The projection device can utilize the conversion relationship between the spatial coordinate systems and the mapping relationship of the panoramic content to convert the target three-dimensional coordinates into coordinates corresponding to the panoramic content, thereby obtaining the field of view picture area.

[0057] In this embodiment, according to the gesture information, a field of view picture area matching the user viewing angle indicated by the gesture information can be determined from the panoramic content.

[0058] S14, projecting the image content of the field of view image area onto a projection surface corresponding to the projection device.

[0059] In some embodiments of the present application, before projecting the picture content of the field of view picture area onto the projection surface corresponding to the projection device, the projection device may render the picture content according to the field of view picture area.

[0060] The projection device may include a rendering engine, and the projection device may render the picture content through the rendering engine according to the field of view picture area.

[0061] The projection surface corresponding to the projection device can be the surface where the light emitted by the projection device is projected and the image is formed. The projection surface can be a wall or a screen, etc. This application does not limit the type of projection surface.

[0062] In this embodiment, the screen content is rendered according to the field of view screen area, and the screen content is projected onto the projection surface, so that the projected screen content can match the user's viewing angle indicated by the posture information. By matching the viewing angle of the projection device, the projected screen content, and the user's viewing angle indicated by the posture information, the projection device and the projected screen content can track and respond to changes in the user's viewing angle in real time, without the user having to manually drag the screen to change the visual focus or wearing specific equipment, thereby improving the user's viewing freedom, comfort, immersion, and smoothness of the panoramic content.

[0063] In other embodiments of the present application, the projection device can monitor and receive posture information from the control device in real time. If updated posture information is received from the control device, the projection device can control the rotated projection device to rotate according to the updated posture information, determine an updated field of view image area from the panoramic content, and project the updated field of view image onto the projection surface of the projection device.

[0064] The projection device may compare the posture information received from the control device with the posture information of the previous frame, and if the two posture information are different, determine that the received posture information is updated posture information.

[0065] In this embodiment, by real-time monitoring and receiving the posture information from the control device, the projection device can immediately respond to the user's operation intention and dynamically adjust the projection viewing angle, screen content, etc. This real-time monitoring and response mechanism can not only improve the smoothness and accuracy of the interaction, but also improve the user's viewing experience.

[0066] In other embodiments of the present application, the projection device can obtain the user's voice information, and adjust the projection device and / or the projected screen according to the recognition of the voice information. The projection device may include a voice recognition module, and the projection device may convert the user's voice information into text information through the voice recognition module, and the projection device may adjust the projection device and / or the projected screen through the text information.

[0067] Among them, the text information may include instructional text, content selection text, parameter adjustment text, device control text, etc. Instructive text can be used to control the playback status of the projected picture content. For example, the instructional text may include "pause playback", "play next page", etc. Content selection text can be used to instruct the projection device to play specific picture content. For example, content rotation text may include "play today's weather forecast", "play a documentary about the universe", etc. Parameter adjustment text can be used to adjust the display effect of the projected picture. For example, parameter adjustment text may include "increase brightness", "lower contrast", etc. Device control text can be used to adjust the viewing angle and direction of the projection device. For example, the device control text may include "rotate 90 degrees to the left", "tilt 15 degrees upward", etc.

[0068] Adjustment of the projection device may include adjustment of the projection direction and projection viewing angle of the projection device, etc. Adjustment of the projected image may include switching the image content, adjusting the brightness, contrast, color saturation, resolution, image size, image ratio, and image display mode, etc.

[0069] In some embodiments of the present application, it is considered that the projection device may generate trapezoidal distortion when rotating, thereby causing deformation of the image mapped to the projection surface. In order to solve the problem of image deformation caused by trapezoidal distortion, the projection device can calculate correction information based on the multi-sensor data of the projection device, and use the correction information to correct the projected image.

[0070] Among them, the multi-sensor data can be detected by sensors such as the inertial measurement unit and the time of flight (TOF) sensor in the projection device. The multi-sensor data may include information such as the attitude information of the projection device, the distance and angle between the projection device and the projection surface, etc. Among them, the attitude information of the projection device can be detected by the inertial measurement unit in the projection device, and the distance and angle between the projection device and the projection surface can be obtained by the time of flight sensor. The time of flight sensor emits a light beam (for example, infrared light or laser) to the projection surface and measures the time it takes for the reflected light beam to return to the projection device, and determines information such as the distance and angle between the projection device and the projection surface through the measured time.

[0071] Correction operations on the projected image may include image scaling and keystone correction.

[0072] Exemplarily, the projection device can calculate the degree of geometric deformation of the picture based on the posture information of the projection device, the distance and angle between the projection device and the projection surface, and other information. Based on the degree of geometric deformation, the projection device can calculate correction information such as the trapezoidal correction coefficient, picture scaling ratio, and displacement parameters.

[0073] In this embodiment, by correcting the projected image, not only can the image be kept in a standard rectangular display, but the clarity of the image can also be ensured, thereby improving the viewing experience of the user.

[0074] In the panoramic projection method provided in the embodiment of the present application, the projection device is controlled to rotate by the posture information received from the rotated control device, so that the posture of the rotated projection device can match the posture of the rotated control device, thereby ensuring that the viewing angle of the rotated projection device matches the user viewing angle indicated by the posture information. According to the posture information, the field of view screen area that matches the user viewing angle indicated by the posture information can be determined from the panoramic content. The screen content is rendered according to the field of view screen area, and the screen content is projected onto the projection surface, so that the projected screen content matches the user viewing angle indicated by the posture information. By matching the viewing angle of the projection device, the projected screen content and the user viewing angle indicated by the posture information, the projection device and the projected screen content can track and respond to changes in the user viewing angle in real time, without the user manually dragging the screen to change the visual focus and without wearing specific equipment, so that the user's viewing freedom, comfort, immersion, and fluency of the panoramic content can be improved. In addition, compared with specific devices such as VR, the popularity of projection devices is relatively high, so the problem of high viewing threshold of panoramic content can be effectively solved.

[0075] like Figure 2 FIG. 1 is a flow chart of a method for determining a field of view image area provided by an embodiment of the present application, comprising the following steps: S131, determining the spherical coordinates corresponding to the spatial point according to the target three-dimensional coordinates.

[0076] In some embodiments of the present application, the spherical coordinates of each spatial point may include longitude and latitude.

[0077] For example, if the target three-dimensional coordinates of the spatial point are expressed as (X, Y, Z), the calculation formulas for longitude and latitude in spherical coordinates can refer to the following formulas (1) to (2): ; (1) ; (2) in, Indicates longitude, Represents dimension.

[0078] S132, determining the plane coordinates corresponding to the space point based on the spherical coordinates.

[0079] In some embodiments of the present application, the plane coordinates corresponding to each spatial point may include a horizontal coordinate and a vertical coordinate.

[0080] For example, following the above embodiment, if the longitude in spherical coordinates is expressed as , latitude is expressed as , the calculation formulas for the horizontal and vertical coordinates in the plane coordinates can refer to the following formulas (3) to (4): ; (3) ; (4) in, represents the horizontal axis, Indicates the vertical coordinate.

[0081] S133: Determine a pixel area corresponding to the plane coordinates in the panoramic content as a field of view picture area.

[0082] In some embodiments of the present application, considering that the panoramic content is obtained by mapping the scene in the three-dimensional space onto a two-dimensional plane through projection methods such as equirectangular projection (ERP) and cube map projection (CMP), and the two-dimensional plane corresponds to plane coordinates, the projection device can determine the corresponding pixel area from the panoramic content as the field of view picture area based on the plane coordinates corresponding to the spatial point and information such as the resolution and pixel size of the panoramic content.

[0083] Among them, according to the plane coordinates corresponding to the spatial point and the resolution and pixel size of the panoramic content, the corresponding pixel area is determined from the panoramic content as the field of view picture area. Reference can be made to the descriptions of image mapping and projection, image cropping and scaling, spatial coordinate transformation, and panoramic image processing in related technologies.

[0084] like Figure 3 , which is a schematic diagram of the structure of the projection device provided in an embodiment of the present application.

[0085] The projection device 200 includes a projection unit 210 and a driving unit 220 driving the projection unit 210. The projection unit 210 may form an optical image and project the optical image onto an imaging medium SC.

[0086] The projection unit 210 includes a light source unit 211 , a light modulator 212 , and an optical system 213 . The driving unit 220 includes a light source driving unit 221 and a light modulator driving unit 222 .

[0087] The light source unit 211 may include a solid light source such as a light emitting diode (LED), a laser, a pump lamp, etc. The light source unit 211 may include optical elements such as a lens and a polarizing plate for improving the optical characteristics of the projected light, and a dimming element for adjusting the light flux.

[0088] The light source driving unit 221 can control the light source in the light source unit 211 to operate, including lighting and extinguishing, according to the instruction of the control unit 250 .

[0089] The light modulator 212 includes a display panel 215 , which may be a transmissive liquid crystal display (LCD) panel, a reflective liquid crystal on silicon (LCOS) panel, or a digital micromirror device (DMD).

[0090] The light modulator 212 is driven by a light modulator driving unit 222 , and the light modulator driving unit 222 is connected to the image processing unit 245 .

[0091] The image processing unit 245 inputs image data to the light modulator driving unit 222. The light modulator driving unit 222 converts the input image data into a data signal suitable for the operation of the display panel 215. The light modulator driving unit 222 applies a voltage to each pixel of each display panel 215 according to the converted data signal, and draws an image on the display panel 215.

[0092] The optical system 213 includes a lens or a reflector for forming an image of the incident image light PLA on the imaging medium SC. The optical system 213 may also include a zoom mechanism for magnifying or reducing the image projected on the imaging medium SC and a focus adjustment mechanism for adjusting the focus.

[0093] The projection device 200 further includes an operation unit 231, a signal receiving unit 233, an input interface 235, a storage unit 237, a data interface 241, an interface unit 242, a frame memory 243, an image processing unit 245, and a control unit 250. The input interface 235, the storage unit 237, the data interface 241, the interface unit 242, the image processing unit 245, and the control unit 250 can communicate data with each other via the internal bus 207.

[0094] The operation unit 231 can generate corresponding operation signals according to the operations of various buttons and switches on the housing surface of the projection device 200, and output them to the input interface 235. The input interface 235 includes a circuit that outputs the operation signals input from the operation unit 231 to the control unit 250.

[0095] After receiving a signal (such as an infrared signal or a Bluetooth signal) sent from the control device 5 (such as a remote controller), the signal receiving unit 233 can decode the received signal to generate a corresponding operation signal. The signal receiving unit 233 outputs the generated operation signal to the input interface 235. The input interface 235 outputs the received operation signal to the control unit 250.

[0096] The storage unit 237 may be a magnetic recording device such as a hard disk drive (HDD) or a storage device using a semiconductor storage element such as a flash memory. The storage unit 237 stores programs executed by the control unit 250, data processed by the control unit 250, image data, and the like.

[0097] The data interface 241 includes a connector and an interface circuit, and can be wired to other electronic devices 100. The data interface 241 can be a communication interface for performing communication with other electronic devices 100. The data interface 241 receives image data, sound data, etc. from other electronic devices 100. In this embodiment, the image data can be a content image.

[0098] The interface unit 242 is a communication interface for communicating with other electronic devices 100 according to the Ethernet standard. The interface unit 242 includes a connector and an interface circuit for processing a signal sent by the connector. The interface unit 242 is an interface substrate including a connector and an interface circuit and is connected to the main substrate of the control unit 250, which is a substrate on which a processor 253 and other components are mounted. The connector and the interface circuit constituting the interface unit 242 are mounted on the main substrate of the control unit 250. The interface unit 242 can receive setting information or instruction information sent by other electronic devices 100.

[0099] The control unit 250 includes a memory 251 and a processor 253 .

[0100] The memory 251 is a storage device that stores programs and data executed by the processor 253 in a non-volatile manner. The memory 251 is composed of semiconductor storage elements such as a magnetic storage device, a flash read-only memory (ROM), or other types of non-volatile storage devices. The memory 251 may also include a random access memory (RAM) that constitutes a work area of ​​the processor 253. The memory 251 stores data processed by the control unit 250 and a control program executed by the processor 253. When the data and control program stored in the memory 251 are executed by the processor 253, the panoramic projection method executed on the projection device 200 can be implemented.

[0101] The processor 253 may be composed of a single processor or a combination of multiple processing groups. The processor 253 executes a control program to control various parts of the projection device 200. For example, the processor 253 performs the corresponding interactive panoramic projection according to the operation signal generated by the operation unit 231, and outputs the parameters used in the interactive panoramic projection (such as the parameters for performing trapezoidal correction on the image) to the image processing unit 245. In addition, the processor 253 can control the light source in the light source unit 211 to light up, turn off or adjust the brightness by controlling the light source driving unit 221. The processor 253 provides computing and control capabilities. For example, the processor 253 is used to execute the data and programs stored in the memory 251 to implement the above-mentioned panoramic projection method.

[0102] The image processing unit 245 and the frame memory 243 may be formed by integrated circuits. Integrated circuits include large-scale integrated circuits (LSI), application-specific integrated circuits (ASIC), and programmable logic devices (PLD), wherein the PLD may include a field-programmable gate array (FPGA). The integrated circuit may also include a portion of an analog circuit, or a combination of a processor and an integrated circuit. The combination of a processor and an integrated circuit is called a microcontroller unit (MCU), a system on chip (SoC), a system LSI, a chipset, and the like.

[0103] The image processing unit 245 may store the image data received from the data interface 241 in the frame memory 243. The frame memory 243 includes a plurality of memory banks, each of which includes a storage capacity that can write image data of one frame. The frame memory 243 may be composed of a synchronous dynamic random access memory (SDRAM) or a dynamic random access memory (DRAM).

[0104] The image processing unit 245 can perform interactive panoramic projection on the image data stored in the frame memory 243, including resolution conversion, size adjustment, distortion correction, shape correction, digital zoom, image tone adjustment, image brightness adjustment, etc.

[0105] The image processing unit 245 can also convert the input frame frequency of the vertical synchronization signal into a drawing frequency and generate a vertical synchronization signal having the drawing frequency. The generated vertical synchronization signal is called an output synchronization signal. The image processing unit 245 then outputs the output synchronization signal to the optical modulator driving unit 222.

[0106] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the methods in the above-mentioned embodiments of the present application.

[0107] The computer-readable storage medium may be the internal memory of the projection device described in the above embodiment, for example, the hard disk or memory of the projection device. The computer-readable storage medium may also be an external storage device of the projection device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., provided on the projection device.

[0108] In some embodiments, the computer-readable storage medium may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the projection device, etc.

[0109] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0110] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0111] Although the subject matter has been described in the language of specific method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims

1. A panoramic projection method, applied to a projection device, characterized in that: The method comprises: Acquiring panoramic content and posture information, wherein the posture information includes posture information received from the rotated control device; Controlling the projection device to rotate according to the posture information; Determining a field of view image area from the panoramic content according to the posture information; Project the image content of the field of view image area onto the projection surface corresponding to the projection device.

2. The panoramic projection method according to claim 1, characterized in that: The controlling the projection device to rotate according to the posture information comprises: Determining rotation information corresponding to the rotation mechanism of the projection device according to the posture information; Based on the rotation information, the projection device is controlled to rotate through the rotation mechanism.

3. The panoramic projection method according to claim 1, characterized in that: Determining the field of view area from the panoramic content according to the posture information includes: Determine a rotation matrix according to the posture information; Determine the initial three-dimensional coordinates of a spatial point within the field of view of the projection device, and transform the initial three-dimensional coordinates according to the rotation matrix to obtain the target three-dimensional coordinates of the spatial point; Based on the target three-dimensional coordinates of the spatial point, the field of view picture area is determined from the panoramic content.

4. The panoramic projection method according to claim 3, characterized in that: The determining the field of view area from the panoramic content based on the target three-dimensional coordinates of the spatial point comprises: Determine the spherical coordinates corresponding to the spatial point according to the target three-dimensional coordinates; Determine the plane coordinates corresponding to the spatial point based on the spherical coordinates; A pixel area corresponding to the plane coordinates in the panoramic content is determined as the field of view picture area.

5. The panoramic projection method according to claim 1, characterized in that: Before projecting the picture content of the field of view picture area onto the projection surface corresponding to the projection device, the method further includes: The image content is rendered according to the field of view image area.

6. The panoramic projection method according to claim 1, characterized in that: The method further comprises: If updated posture information is received from the control device, the rotated projection device is controlled to rotate according to the updated posture information and an updated field of view image area is determined from the panoramic content.

7. The panoramic projection method according to claim 1, characterized in that: The method further comprises: Get the user's voice information; The projection device and / or the projected image are adjusted according to the recognition of the voice information.

8. The panoramic projection method according to claim 1, characterized in that: The method further comprises: Calculating correction information based on multi-sensor data of the projection device; The projected image is corrected using the correction information.

9. A projection device, characterized in that: The projection device comprises: processor; A memory; and an application, wherein the application is stored in the memory and is configured to be executed by the processor to implement the panoramic projection method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor in the projection device, the panoramic projection method according to any one of claims 1 to 8 is implemented.

Images