Intelligent device display method, projection system, and correction method

Abstract

Embodiments of the present application relate to the technical field of display, and provide an intelligent device display method, a projection system, and a correction method. The intelligent device display method provided in the present application comprises: an intelligent device displaying an operation interface; when a picture in the operation interface comprises a projection device main unit, displaying a virtual image of the projection device main unit in the operation interface, the virtual image being located at a preset target position; and when the position of the projection device main unit in the interface changes, the operation interface also indicating path information of a current position of the projection device main unit reaching the preset target position. The present application can improve the correction accuracy.

Description

Smart device display method, projection system and correction method

[0001] The present application claims priority to the Chinese patent application No. 2024113772992, filed on September 29, 2024 in the China Patent Office, and entitled "Smart device display method, projection system and correction method", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of display, in particular to a smart device display method, a projection system and a correction method. BACKGROUND

[0003] The in-mural correction refers to a process of accurately aligning a projection picture to the edge of a projection screen when using a projection device.

[0004] In the related art, the projection picture projected on the projection screen can be captured, the captured image can be analyzed based on an image processing algorithm, the deviation of the picture from the edge of the projection screen can be recognized, and then the parameters that need to be adjusted can be calculated to complete the in-mural correction.

[0005] However, during the implementation of the present application, the inventors found that at least the following problem existed in the related art: the above-mentioned method is affected by environmental factors, such as uneven light, environmental interference around the projection screen, etc., and the correction accuracy is low. TECHNICAL PROBLEM

[0006] One of the purposes of the embodiments of the present application is to provide a smart device display method, a projection system and a correction method, which can improve the correction accuracy. TECHNICAL SOLUTION

[0007] The technical solution adopted by the embodiments of the present application is:

[0008] In a first aspect, a smart device display method is provided, applied to a smart device, comprising:

[0009] displaying an operation interface;

[0010] when the picture in the operation interface includes a projection device host, displaying a virtual image of the projection device host in the operation interface, the position of the virtual image being located at a preset target position;

[0011] when the position of the projection device host in the operation interface changes, the operation interface also simultaneously indicates the path information of the current position of the projection device host to the preset target position.

[0012] The method provided in the embodiment can prompt a user to move the host of the projection device to a preset target position by displaying a virtual image of the host of the projection device, so that the correction operation, such as the curtain correction, the image correction, or the like, is implemented. The influence of light and the like can be avoided, and the accuracy of the correction is improved.

[0013] In a second aspect, a projection device is provided, comprising:

[0014] a host of the projection device, configured to receive an operation interface sent by the smart device and display the operation interface on a projection screen; the operation interface comprises an adjustment control;

[0015] the projection screen, configured to display the operation interface;

[0016] a remote control component, connected to the host of the projection device, configured to perform a touch operation on the adjustment control based on path information of a path from a current position of the host of the projection device to a preset target position indicated by the operation interface; a position of a virtual image of the host of the projection device is displayed in a picture in the operation interface at the preset target position; the preset target position is a position of the host of the projection device when a normal picture is displayed on the projection screen;

[0017] the host of the projection device is further configured to generate position adjustment information in response to the touch operation on the adjustment control, and send the position adjustment information to a driving device, so that the driving device moves the host of the projection device to the preset target position.

[0018] The device provided in the embodiment can display the operation interface on the projection screen by the host of the projection device, so that the host of the projection device is integrated with the position adjustment function, the control in the interface is touched by the remote control device, and then the position adjustment of the host of the projection device is implemented. Since the projection screen has a larger display area, the convenience of the user operation can be further improved. Especially for the non-initial installation scene, the position of the host of the projection device may not be greatly offset, and the interface displayed on the projection screen is close to normal, so that the display area can be increased, the operation convenience can be improved, and whether the correction is successful can be determined by observing the display effect of the displayed interface.

[0019] In a third aspect, a projection system is provided, comprising the smart device of the first aspect, the projection device of the second aspect, and a driving device.

[0020] In a fourth aspect, a correction method is provided, comprising:

[0021] Obtaining a target image of an actual use scene of a host of a projection device, generating a to-be-displayed image according to the target image; the target image comprises a projection screen of the projection device and a placing table for placing the host of the projection device; a virtual image of the host of the projection device is presented on a preset target position in the to-be-displayed image; the preset target position is a position of the host of the projection device when a normal picture is projected on the projection screen.

[0022] Displaying the to-be-displayed image, the to-be-displayed image is used to indicate that the host of the projection device is moved to the preset target position.

[0023] The method provided in the embodiment displays the AR picture with the virtual image in the preset target position in the operation interface, so as to prompt the user to move the host of the projection device to the preset target position, and correct operation, such as curtain correction, is realized. The influence of light and other conditions can be avoided, and the accuracy of correction is improved. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the implementation manners in the related art, the drawings needed to be used in the embodiment or the related art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art according to these drawings.

[0025] Fig. 1 is a schematic diagram of an operation scene between a display device and a control device according to an embodiment;

[0026] Fig. 2a exemplarily shows a configuration block diagram of the control device 100 according to an exemplary embodiment;

[0027] Fig. 2b exemplarily shows a configuration block diagram of the display device 200 according to an exemplary embodiment;

[0028] Fig. 3 is a flow diagram of a smart device display method provided by the present application;

[0029] Fig. 4a is a scene diagram for measuring scene parameters provided by the present application;

[0030] Fig. 4b is a scene diagram for measuring scene parameters provided by the present application;

[0031] Fig. 5 is an interface diagram of a smart device of a correction method provided by the present application;

[0032] Fig. 6 is an interface diagram of a smart device of a correction method provided by the present application;

[0033] Fig. 7 is an interface diagram of a smart device of a correction method provided by the present application;

[0034] FIG. 8 is a schematic diagram of an interface of a display device in a correction method provided by the present application;

[0035] FIG. 9a is a schematic diagram of a structure of a driving device in a correction method provided by the present application;

[0036] FIG. 9b is a schematic diagram of a position adjustment in a correction method provided by the present application;

[0037] FIG. 10 is a schematic diagram of a structure of a driving device in a correction method provided by the present application;

[0038] FIG. 11 is a schematic diagram of a flow of a correction method provided by the present application. Embodiments of the present application

[0039] For the purpose of making the objects, embodiments and advantages of the present application clearer, the following will describe the exemplary embodiments of the present application clearly and completely with reference to the accompanying drawings. Obviously, the described exemplary embodiments are only a part of the embodiments of the present application, but not all the embodiments.

[0040] It should be noted that the brief description of the terms in the present application is only for the convenience of understanding the following described embodiments, but is not intended to limit the embodiments of the present application. Unless otherwise specified, these terms should be understood according to their ordinary and general meanings.

[0041] In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclusive inclusion, for example, a product or device including a series of components does not have to be limited to the clearly listed components, but can include other components not clearly listed or inherent to these products or devices.

[0042] The display device provided by the embodiments of the present application can have various implementation forms, for example, can be a television, a smart television, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc. FIG. 1 and FIG. 2 are a specific embodiment of the display device of the present application.

[0043] FIG. 1 is a schematic diagram of an operation scene between the display device and the control device in the embodiments. As shown in FIG. 1, the user can operate the display device 200 through the smart device 300 or the control device 100.

[0044] In some embodiments, the control device 100 can be a remote controller, and the communication between the remote controller and the display device can be infrared protocol communication or Bluetooth protocol communication, or other short-distance communication, and the display device 200 can be controlled by the remote controller through wireless or wired communication. The user can input user instructions through the buttons on the remote controller, voice input, control panel input, etc., to control the display device 200.

[0045] In some embodiments, the smart device 300 (such as a smart device, a tablet, a computer, a notebook, etc.) can also be used to control the display device 200. For example, the display device 200 can be controlled by using an application running on the smart device.

[0046] In some embodiments, the display device can not receive instructions using the smart device or the control device described above, but can receive user control through touch or gesture, etc.

[0047] In some embodiments, the display device 200 can also be controlled in ways other than the control device 100 and the smart device 300, for example, the display device 200 can directly receive user voice instructions through a voice instruction acquisition module configured inside the display device 200, or the display device 200 can receive user voice instructions through a voice control device arranged outside the display device 200.

[0048] In some embodiments, the display device 200 also communicates with the server 400. The display device 200 can be connected through a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 can provide various content and interaction to the display device 200. The server 400 can be a cluster or multiple clusters, and can include one or more types of servers.

[0049] FIG. 2a shows a configuration block diagram of the control device 100 according to an exemplary embodiment. As shown in FIG. 2a, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, a power supply. The control device 100 can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, and act as an intermediary between the user and the display device 200.

[0050] As shown in FIG. 2b, the display device 200 includes a tuner and demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and at least one of a user interface.

[0051] The controller in some embodiments includes a processor, a video processor, an audio processor, a graphic processor, a RAM, a ROM, a first interface to an n-th interface for input / output.

[0052] The display 260 includes a display screen component for presenting a picture, and a driving component for driving the image display, a component for receiving an image signal originated from the output of the controller, and a component for displaying video content, image content, and a menu operation interface, and a user operation UI interface.

[0053] The display 260 can be a projection display, and can also be a projection device and a projection screen.

[0054] The communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example, the communicator can include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 can establish transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

[0055] The user interface can be used to receive control signals of the control device 100 (such as an infrared remote controller, etc.).

[0056] The detector 230 is used to collect signals of external environment or interaction with the outside. For example, the detector 230 includes a light receiver for collecting ambient light intensity, or the detector 230 includes an image collector such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures, or the detector 230 includes a sound collector such as a microphone, etc., for receiving external sound.

[0057] The external device interface 240 can include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or data high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It can also be a composite input / output interface formed by the above multiple interfaces.

[0058] The tuning demodulator 210 receives broadcast television signals through wired or wireless reception, and demodulates audio and video signals and EPG data signals from a plurality of wireless or wired broadcast television signals.

[0059] In some embodiments, the controller 250 and the tuning demodulator 210 can be located in different split devices, i.e., the tuning demodulator 210 can also be in an external device of the main body device where the controller 250 is located, such as an external set-top box, etc.

[0060] The controller 250 controls the operation of the display apparatus and responds to a user's manipulation by storing various software control programs in the memory. The controller 250 controls the overall operation of the display apparatus 200. For example, in response to receiving a user command for selecting a UI object displayed on the display 260, the controller 250 can perform an operation related to the object selected by the user command.

[0061] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a RAM (Random Access Memory), a ROM (Read-Only Memory), a first to an n-th interface for input / output, a communication bus, etc.

[0062] The user can input a user command through a graphic user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphic user interface (GUI). Alternatively, the user can input a user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through a sensor.

[0063] The "user interface" is a medium interface for interaction and information exchange between an application program or an operating system and a user, and it implements conversion between an internal form of information and a form acceptable to the user. A commonly used form of the user interface is a graphic user interface (GUI), which refers to a user interface related to computer operation displayed in a graphic manner. It can be an icon, a window, a control, etc. displayed on a display screen of an electronic device, and the control can include an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a widget, etc.

[0064] The "keystone correction" refers to a process of accurately aligning a projection screen to the edge of a projection screen when using a projector, a laser TV, etc.

[0065] In the related art, when performing the curtain-in correction, a built-in or external camera is usually used to capture the picture projected on the projection screen. Through image processing algorithms, the captured image is analyzed to identify the deviation of the picture from the edge of the projection screen, including position offset, inclination angle, etc. Based on the analysis result, the parameters that need to be adjusted are calculated, such as focal length, inclination angle, displacement amount, etc. The projection device automatically adjusts according to the calculated parameters, so that the projection picture is accurately aligned with the edge of the projection screen. However, the above-mentioned method is easily affected by environmental factors, such as uneven lighting, environmental interference around the projection screen, etc., which may result in poor correction effect or adjustment failure.

[0066] To solve the above technical problems, the present application has found that the preset target position (i.e. the position of the projection device host that can project a normal picture on the projection screen) can be displayed in the image of the actual use scene obtained in real time by using the augmented reality (AR) virtual display, and the projection device host is controlled to move to the preset target position, thereby realizing the correction operation, such as the curtain-in correction, and making the correction more accurate.

[0067] The following describes how the laser television, projector and other display devices realize the correction operation, such as the curtain-in correction, taking the smart device as an example.

[0068] The technical solutions of the present application will be described in detail below in combination with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0069] FIG. 3 is a flowchart of a display method of a smart device provided by the present application, applied to a smart device. As shown in FIG. 3, the method comprises the following steps:

[0070] 301, display an operation interface.

[0071] The execution subject of the present embodiment can be a smart device such as a mobile phone or a tablet computer.

[0072] 302, when the picture in the operation interface includes the projection device host, display a virtual image of the projection device host in the operation interface, and the position of the virtual image is located at the preset target position.

[0073] Specifically, the picture in the operation interface can be a picture of an actual use scene of the projector host, can include the projector host, can further include a placement table for placing the projector host, and can further include a projection screen. In an application scenario, for example, in a scenario where a preset target position has been determined in advance, the display timing of the virtual image of the projector host in the operation interface can be when the picture in the operation interface includes the projector host. In another application scenario, for example, in a scenario where the preset target position is determined for the first time, the display timing of the virtual image of the projector host in the operation interface can be when the picture in the operation interface includes the complete projection screen and the complete projector host. The preset target position can be determined according to the projection screen and the like, and it is convenient to determine whether the correction is performed and the execution effect by observing whether the picture display in the projection screen is normal.

[0074] In this embodiment, the projector host can be a projector that projects a picture to a projection screen, for example, a host of a laser television or a projector. The actual use scene of the projector host usually includes a projection screen for displaying the projected picture, a placement platform of the projector host, and the like. The target image can be an image of a real environment obtained by real-time shooting of the actual use scene.

[0075] In some embodiments, after the actual use scene of the projector host is shot to obtain the target image, there is a preset target position in the table surface in the target image for placing the projector host, which can enable the projector host to project a normal picture. The preset target position is displayed differently in the target image to obtain and display the to-be-displayed image. Further, the relative positional relationship between the current position of the projector host and the preset target position can be determined based on the to-be-displayed image, so that the projector host can be moved from the current position to the preset target position, so that the projector host can project a normal picture on the projection screen, and the in-mural correction is completed.

[0076] The target image only needs to include the light outlet position and the projector host, and can not include elements such as the projection screen, because the purpose is to move the projector host based on the light outlet position.

[0077] In some embodiments, the operation interface is displayed, including: obtaining a target image obtained by shooting an actual use scene of a projector host; the actual use scene includes a projection screen of the projector and a placement table for placing the projector host.

[0078] The operation interface is generated according to the target image.

[0079] Specifically, a target image of an actual use scene of a host of a projection device is acquired, a to-be-displayed image is generated according to the target image, and then an operation interface is generated according to the to-be-displayed image; the target image includes a projection screen of the projection device and a placement table for placing the host of the projection device; a virtual image of the host of the projection device is presented on a preset target position in the to-be-displayed image; the preset target position is a position of the host of the projection device when a normal picture is projected on the projection screen.

[0080] The method provided in this embodiment can obtain a live image by shooting a use scene including a projection screen and a placement table of a host of a projection device, so as to process the image, add a virtual image of the host of the projection device to a preset target position to generate an operation interface, and guide a user to move the host of the projection device reasonably, thereby improving correction accuracy.

[0081] In some embodiments, displaying the virtual image of the host of the projection device in the operation interface includes: acquiring the preset target position; and displaying the virtual image of the host of the projection device in a picture in the operation interface based on the preset target position.

[0082] Specifically, after an actual use scene of a host of a projection device is determined, for example, after positions of a projection screen and a placement table of the host of the projection device are fixed. A theoretical value of a position of the host of the projection device that can project a normal picture on the projection screen, that is, a preset target position, can be calculated according to a projection mode of the host of the projection device (for example, whether it is a front projection or a side projection), the position of the projection screen, a height of the placement table and other parameters. After the preset target position is acquired, a virtual image of the host of the projection device can be displayed on the preset target position in a target image based on AR technology, to obtain a to-be-displayed image.

[0083] The virtual image of the host of the projection device can be a solid model. The solid model can be designed according to an appearance of the host of the projection device, or can be a simplified model, for example, a minimum circumscribed polyhedron with one face, for example, a cuboid.

[0084] The correction method provided in this embodiment can first determine a preset target position, and then integrate a virtual image of a host of a projection device into the preset target position in a target image, so that a to-be-displayed image more accurately reflects a pose to which the host of the projection device needs to be moved, that is, a picture in an operation interface more accurately reflects the pose to which the host of the projection device needs to be moved, thereby improving correction accuracy and convenience.

[0085] In some embodiments, there are various ways to acquire the preset target position.

[0086] In an implementation, the projection screen and the placement table of the host of the projection device are usually not moved. In order to improve efficiency, the last time of installation or the preset target position used in the first time of installation can be directly called in the current non-first installation scenario. Specifically, the preset target position is obtained by obtaining the preset target position determined in advance from the preset storage position.

[0087] Specifically, after the first installation of the host of the projection device, the calculated preset target position can be stored in the preset storage position, so that the preset target position can be directly called in subsequent installation. If the position of the projection screen or the placement table changes, the preset target position can be recalculated, and the calculated preset target position can replace the preset target position determined in advance stored in the preset storage position.

[0088] The correction method provided in the embodiment saves the process of re-photographing and calculation by directly obtaining the preset target position determined in advance on the premise that the preset target position has been calculated, and improves the efficiency of correction.

[0089] In another implementation, the method further includes determining a mapping relationship between a display picture coordinate system of a display screen of the smart device, a camera coordinate system of a camera of the smart device, and a real world coordinate system.

[0090] Specifically, taking a mobile phone as an example of an execution subject. The calibration of the camera coordinate system of the camera of the mobile phone, the display picture coordinate system of the display projection screen, and the real world coordinate system of the real environment can be completed in the first installation or reinstallation, so that the three coordinate systems are calibrated, and it is ensured that no matter how the camera moves, the displayed still elements remain stable and do not move with the movement of the camera, and a more real augmented reality experience is achieved.

[0091] In some embodiments, obtaining the preset target position can include: determining a scene parameter based on the mapping relationship; determining a position of an outlet of the host of the projection device when a normal picture is projected on the projection screen based on the scene parameter; and determining the preset target position based on the position of the outlet of the host of the projection device.

[0092] Specifically, after the mapping relationship between the three coordinate systems is determined, the scene parameter can be determined based on the mapping relationship, that is, the relative position between the projection screen and the placement table obtained by the camera is converted into a relative position in the real environment through the mapping relationship, that is, the scene parameter can represent the relative position relationship between the projection screen and the placement table in the real environment. Further, the position coordinates of the outlet of the host of the projection device can be calculated based on the scene parameter, and finally the preset target position can be determined based on the position coordinates.

[0093] It should be understood that when the projection screen is initially installed or re-installed, since the installation position can not be determined or can change greatly, the display picture coordinate system of the display screen of the smart device, the camera coordinate system of the camera of the smart device and the real world coordinate system need to be calibrated to obtain the mapping relationship between the three coordinate systems, and then the scene parameters are determined based on the mapping relationship, and finally the position of the light outlet of the projection device host is calculated through the scene parameters to determine the preset target position.

[0094] It should also be understood that in subsequent daily correction, since the installation position has been fixed, the preset target position determined during installation can be directly read. Of course, since the projection device host or the placement table and the like can change due to use or other reasons, the preset target position can be determined again based on the mapping relationship at this time to determine the real-time preset target position, thereby ensuring the real-time and accuracy of the determination of the preset target position. It should be noted that in daily correction, since the installation position has been fixed, that is, the display picture coordinate system of the display screen of the smart device, the camera coordinate system of the camera of the smart device and the real world coordinate system have been calibrated, so the mapping relationship determined during installation can be directly used to obtain the current scene parameters, and then the real-time light outlet position is determined, thereby improving the efficiency of determining the preset target position.

[0095] The correction method provided in the embodiment can improve the calculation accuracy of the preset target position, and then improve the correction accuracy, by mapping the three coordinate systems and calculating the light outlet position based on the determined scene parameters, and then obtaining the preset target position.

[0096] In some embodiments, determining the mapping relationship between the display picture coordinate system of the display screen of the smart device, the camera coordinate system of the camera of the smart device and the real world coordinate system can include: obtaining a first image obtained by photographing an actual use scene of the projection device host; the first image includes a preset calibration object located in the actual use scene; determining the mapping relationship between the display picture coordinate system of the display screen of the smart device, the camera coordinate system of the camera of the smart device and the real world coordinate system according to the first image.

[0097] Specifically, the first image is obtained by photographing the actual use scene of the projection device host through the camera connected with the processor; the first image includes a preset calibration object located in the actual use scene; the first image is displayed through the display screen; and the mapping relationship between the display picture coordinate system of the display screen of the smart device, the camera coordinate system of the camera of the smart device and the real world coordinate system can include: determining the mapping relationship between the display picture coordinate system of the display screen of the smart device, the camera coordinate system of the camera of the smart device and the real world coordinate system according to the first image.

[0098] The calibration object can be a black-and-white checkerboard or a marker with a specific pattern, or can be an object fixedly placed in an actual use scenario.

[0099] Specifically, taking a mobile phone as an example, the process of mapping three coordinate systems is introduced as follows: a calibration object with a known size and position in the real world is selected, which is clearly visible in the camera view of the mobile phone. The calibration object should be placed in a position that can be observed by the camera. An image of the calibration object is captured using the camera, and the display position of the calibration object on the display device is recorded. At the same time, the physical size and position of the calibration object in the real world are measured. Using the position information of the calibration object in different coordinate systems, the coordinate transformation matrix can be calculated. These matrices describe the rotation and translation relationship from one coordinate system to another. Common calibration methods can include Zhang Zhengyou calibration method, which estimates the camera's intrinsic and extrinsic matrix by capturing multiple view images of the calibration object. Using the calculated transformation matrix, points in the camera coordinate system can be mapped to the display picture coordinate system and the real world coordinate system. This step ensures the spatial correspondence between virtual elements and real world elements. Thus, regardless of how the camera moves, the displayed content can be correctly positioned and rendered according to the real world coordinate system. Optionally, the calibration process can be iterated multiple times to optimize the transformation matrix and reduce errors. In practical applications, factors such as lens distortion, nonlinear transformation, etc. can be considered and corrected accordingly to improve the accuracy of calibration, and thus improve the accuracy of correction.

[0100] The method provided by the embodiment can ensure that regardless of how the camera moves, the displayed content can be correctly positioned and rendered according to the real world coordinate system, thereby improving the accuracy of correction.

[0101] In some embodiments, based on the mapping relationship, the scene parameters can be determined, which can include: obtaining a second image obtained by shooting the actual use scenario of the projection device host; the second image includes a projection screen and a placement table for placing the projection device host; determining the scene parameters of the actual use scenario according to the second image and the mapping relationship; the scene parameters include at least one of the following: the relative position relationship between the projection screen and the placement table, the size of the projection screen, and the size of the placement table.

[0102] Specifically, in the scene parameter determination process, the second image can be obtained by shooting the actual use scene of the host of the projection device through the camera; the second image includes a projection screen and a placing table for placing the host of the projection device; the scene parameters of the actual use scene are determined according to the second image and the mapping relationship, that is, the points in the camera coordinate system are mapped into the display picture coordinate system and the real world coordinate system through the mapping relationship, so as to convert the scene parameters in the second image into the scene parameters of the actual use scene; the scene parameters include at least one of the following: the relative position relationship between the projection screen and the placing table, the size of the projection screen, and the size of the placing table.

[0103] In the embodiment, the scene parameters can include the placing height of the host of the projection device, the size of the projection screen, the center point height of the projection screen, the width of the placing table, and the horizontal distance from the projection screen, etc. Different use scenes will measure different scene parameter values. As shown in FIG. 4a, h1 is the center point height of the projection screen, and h2 is the placing height of the host of the projection device. As shown in FIG. 4b, v1 is the horizontal distance from the projection screen. Optionally, in order to facilitate calculation, the second image can include a calibration object, that is, a marker.

[0104] The method provided in the embodiment can obtain the scene parameters by analyzing the image containing the projection screen and the placing table, and can determine the preset target position based on the internal relationship between the scene parameters and the preset target position, so as to avoid the influence of light and other conditions.

[0105] In some embodiments, in order to improve the sense of participation of the user and improve the operability, a user graphical interface can be used to realize the interaction between the user and the mobile phone, and then the curtain correction is completed. The operation interface includes a photographing measurement control; the second image obtained by shooting the actual use scene of the host of the projection device can include: in response to the touch operation on the photographing measurement control, the second image is obtained by shooting the actual use scene of the host of the projection device.

[0106] It should be understood that the operation interface further includes a calculation display control. When the preset target position is determined according to the second image, the position of the light outlet of the host of the projection device when a normal picture is projected on the projection screen can be determined in response to the touch operation on the calculation display control according to the scene parameters; the preset target position is determined according to the position of the light outlet of the host of the projection device.

[0107] Specifically, the operation interface can include a first correction interface, the first correction interface is displayed on the display screen; the operation interface includes a photographing measurement control and a calculation display control; when the preset target position is determined, the following operations can be included: in response to a touch operation on the photographing measurement control, the camera is controlled to capture the actual use scene of the projection device host to obtain a second image; the scene parameters of the actual use scene are determined according to the second image; in response to a touch operation on the calculation display control, the position of the light outlet of the projection device host when a normal picture is projected on the projection screen is determined according to the scene parameters; the preset target position is determined according to the position of the light outlet of the projection device host; and the display screen is controlled to display the preset target position in the operation interface.

[0108] For example, as shown in FIG. 5, after the application program is started, the interface 51 shown in FIG. 2a is displayed on the mobile phone. The interface 51 includes a photographing measurement control 501 and a calculation display control 502. The controls 501 and 502 can be displayed in the form of icons, words, symbols, pictures, etc., or in the form of a combination of multiple forms. The display parameters such as the color and size of the controls 501 and 502 are not limited in this embodiment. For example, as shown in FIG. 5, the controls 501 and 502 are displayed in the form of words. The application program receives a touch operation (such as a click operation or a long press operation) on the control 501. In response to the touch operation, the mobile phone can display a photographing interface. The photographing interface includes a photographing control. The camera of the mobile phone is aimed at the actual use scene. The second image is obtained by receiving a touch operation on the photographing control. The scene parameters of the actual use scene are determined according to the second image. The application program receives a touch operation (such as a click operation or a long press operation) on the control 502. In response to the touch operation, the position 504 of the light outlet of the projection device host is determined according to the scene parameters. The preset target position in the target image is determined according to the position 504 of the light outlet. The preset target position is marked and displayed. As shown in FIG. 5, the dashed auxiliary line is also displayed to highlight the preset target position and assist the user in understanding the rationality and accuracy of the determination of the preset target position.

[0109] It should be noted that when the coordinate calibration is performed, the photographing measurement control can also be used. In response to a touch operation on the photographing measurement control, the camera is controlled to capture the actual use scene of the projection device host to obtain a first image. The first image includes the preset calibration object in the actual use scene. After application processing, the mapping relationship between the display picture coordinate system of the display screen of the smart device, the camera coordinate system of the camera of the smart device, and the real world coordinate system is determined, so that the smart device is synchronized with the real world. After the calibration is completed, the application can prompt that the synchronization is successful in the operation interface.

[0110] The method provided in the embodiment reduces the learning cost of the user and improves the participation of the user by providing the user with a UI interface, guiding the user to capture an image to measure a scene parameter and calculate and display a preset target position.

[0111] In some embodiments, the operation interface includes a model import control; displaying the virtual image of the projector host in the picture in the operation interface can include: in response to a touch operation on the model import control, displaying a model selection interface; the model selection interface includes a plurality of candidate models; the plurality of candidate models include a virtual image corresponding to the projector host; in response to a touch operation on the virtual image corresponding to the projector host in the plurality of candidate models, displaying the virtual image of the projector host in the picture in the operation interface based on the preset target position.

[0112] Specifically, the operation interface includes a second correction interface, and the second correction interface is displayed on the display screen; the operation interface includes a model import control; the virtual image of the projector host is fused with the target image based on the preset target position to obtain a to-be-displayed image, which can include: in response to a touch operation on the model import control, controlling the display screen to display a model selection interface; the model selection interface includes a plurality of candidate models; the plurality of candidate models include a virtual image corresponding to the projector host; in response to a touch operation on the virtual image corresponding to the projector host in the plurality of candidate models, the virtual image of the projector host is set on the preset target position in the target image to obtain the to-be-displayed image.

[0113] For example, after the preset target position is determined, as shown in FIG. 5, the interface 51 can further include a model import control 503, and a touch operation on the model import control 503 is received. In response to the touch operation, a model selection interface can be displayed. The interface can include a plurality of candidate models. The plurality of candidate models can be corresponding models of a plurality of possible models of the projector host inferred by the application program based on the characteristics of the projector host in the captured image, or can be models corresponding to the model information input by the user. The embodiment is not limited in this regard. A selection operation on the plurality of candidate models is received, and in response to the selection operation, a virtual image corresponding to the projector host is determined, and the light outlet of the virtual image is aligned with the light outlet corresponding to the preset target position, so as to add the virtual image of the projector host to the target image to obtain a to-be-displayed image. As shown in FIG. 6, the to-be-displayed image shown in the interface 61 includes a virtual image (red dashed rectangular model) of the projector host and a real image of the projector host.

[0114] The method provided in the embodiment improves the convenience of operation and reduces the learning cost of the user by guiding the user to select a virtual image corresponding to the projector host in the form of a UI interface.

[0115] 303、When the position of the projection device host in the operation interface changes, the operation interface also simultaneously indicates path information of the current position of the projection device host to the preset target position.

[0116] Specifically, the projection device host displays a virtual image of the projection device host in the operation interface in real time at the preset target position, so that the user moves the position of the projection device host based on the virtual image, so as to move the projection device host to the preset target position. In order to give the user more clear and obvious guidance, the operation interface can also display path information between the current position of the projection device and the preset target position. The path information can be path guide lines and the like between the current position of the projection device and the preset target position, and can also be movement direction indication information such as left and right.

[0117] In some embodiments, after displaying the virtual image of the projection device host in the operation interface, the method can further include: receiving position adjustment information determined based on the path information indicated by the operation interface; and sending the position adjustment information to the driving device to move the projection device host to the preset target position.

[0118] Specifically, the display screen displays the to-be-displayed image. Since the to-be-displayed image displays both the virtual image of the projection device host, representing the preset target position where the projection device host should be, and the real image of the projection device host, representing the current position of the projection device host, the relative positional relationship between the current position of the projection device host and the preset target position can be clearly understood by observing the to-be-displayed image, and then the projection device host can be moved from the current position to the preset target position based on the to-be-displayed image.

[0119] The method provided in this embodiment displays the virtual image of the projection device host to prompt the user to move the projection device host to the preset target position, so as to realize correction operations such as curtain correction. The influence of light and the like can be avoided, and the accuracy of correction can be improved.

[0120] In some embodiments, the projection device host can be moved to the preset target position in multiple ways. In one implementable manner, the projection device host can be manually moved so that the projection device host in the to-be-displayed image coincides with the virtual image.

[0121] In another implementable manner, the control module can be used to control the driving assembly to adjust the position of the projection device host.

[0122] Specifically, in some embodiments, the position adjustment can be controlled by the processor, so that the execution subject of the embodiment, such as a mobile phone, has both AR picture processing function and position adjustment function. Specifically, after displaying the virtual image of the projection device host, the method further includes: receiving position adjustment information determined based on path information indicated by the operation interface; and sending the position adjustment information to the driving device to move the projection device host to the preset target position.

[0123] In the embodiment, the position adjustment information can be input based on the UI interface provided by the mobile phone, or can be input based on the remote control device associated with the projection device host.

[0124] The method provided by the embodiment controls the driving device to adjust the position of the projection device host, improves the intelligent level, and is more accurate and convenient than manual displacement.

[0125] In some embodiments, in order to facilitate user operation and more intuitively guide the user, a UI interface can be provided to receive the position adjustment information input by the user. Specifically, the operation interface can be displayed on the display screen. The operation interface includes an adjustment area. The adjustment area includes an adjustment control. The position adjustment information determined based on the path information indicated by the operation interface is received, including: in response to the touch operation of the user on the adjustment control based on the path information indicated by the operation interface, generating the position adjustment information.

[0126] For example, taking a mobile phone as an example, as shown in FIG. 7, the interface 71 includes an adjustment area 701, and the adjustment area 701 includes an adjustment control 702. The touch operation on the control 702 is received, and in response to the touch operation, the forward, backward, left, right, left and right lifting, rotation and other controls of the projection device host can be realized, so as to move the projection device host to the preset target position, so that the entity of the projection device host in the to-be-displayed image coincides with the virtual image.

[0127] The method provided by the embodiment improves the convenience of operation by displaying the adjustment control in the form of a UI interface.

[0128] In some embodiments, the operation interface further includes a display area. The display area is used to display the to-be-displayed image.

[0129] For example, as shown in FIG. 7, the interface 71 further includes a display area 703. The display area 703 is used to display the to-be-displayed image in real time, so that the user can observe the to-be-displayed image to know the distance between the projection device host and the preset target position in real time.

[0130] It is considered that although the picture displayed on the projection screen by the projection device host may be offset or distorted before the entry correction is completed, the initial device placement of the projection device host can be close to the preset target position according to experience, and thus the degree of distortion and offset of the displayed picture should be insufficient to destroy the integrity and accuracy of the displayed content. Therefore, the position adjustment function can be set in the processor of the projection device host.

[0131] The method provided in the embodiment can improve operation convenience by simultaneously displaying the to-be-displayed image and the adjustment control, so that the user can observe the to-be-displayed image while adjusting the position of the projection device host, and efficiency is improved.

[0132] In some embodiments, on the basis of the above-mentioned embodiments, for example, on the basis of the embodiment shown in FIG. 3, after step 303, the method can further include: generating, by the projection device host, an operation interface, and displaying the operation interface on the projection screen; the operation interface includes the adjustment control. The operation interface is displayed on the projection screen. The adjustment control is controlled by the remote control component connected to the projection device host based on the path information of the projection device host from the current position to the preset target position indicated by the operation interface; the position of the virtual image of the projection device host displayed in the preset target position in the picture in the operation interface is the position of the projection device host when a normal picture is displayed on the projection screen. The position adjustment information is generated by the projection device host in response to the touch operation on the adjustment control, and is sent to the driving device, so that the driving device controls the projection device host to move to the preset target position.

[0133] For example, as shown in FIG. 8, the to-be-displayed image displayed in the mobile phone presents a projection screen, and the interface 81 is displayed in the projection screen. The interface 81 includes the adjustment control 801, and the touch operation on the control 801 is received. In response to the touch operation, the forward, backward, left, right, left-right lifting, rotation and other controls of the projection device host can be realized, so that the projection device host is moved to the preset target position, and the entity of the projection device host in the to-be-displayed image is overlapped with the virtual image.

[0134] In the embodiment, the to-be-displayed image can be obtained by the method in the above-mentioned embodiments, for example, the embodiment shown in FIG. 3. The to-be-displayed image can be displayed by the smart device such as the mobile phone or the tablet computer. The to-be-displayed image can also be sent to the projection device host by the smart device, and the to-be-displayed image and the operation interface are fused by the projection device host and are displayed on the projection screen. This is convenient for the user to observe the to-be-displayed image while adjusting the position of the projection device host.

[0135] The correction method provided in the embodiment can enable the projection device host to integrate a position adjustment function by projecting an operation interface to the projection screen display, so that the position adjustment of the projection device host is realized by the touch operation of the control in the interface by the remote control device. Since the projection screen has a larger display area, the convenience of user operation can be further improved. Especially for the scene of non-initial installation, the position offset of the projection device host can be small, and the interface displayed by the projection screen is close to normal, so that the display area can be increased and the operation convenience can be improved, and whether the correction is successful can be determined by observing the display effect of the displayed interface.

[0136] To realize the position adjustment of the projection device host, the embodiment provides a driving device. The driving device comprises: a processor configured to receive position adjustment information sent by a smart device or a display device, and generate a driving signal according to the position adjustment information; the position adjustment information is determined based on a to-be-displayed image; a virtual image of the projection device host is presented on a preset target position in the to-be-displayed image; the preset target position is a position of the projection device host when a normal picture is projected on the projection screen; and a driving component connected with the processor, configured to drive the projection device host to move to the preset target position according to the driving signal.

[0137] The driving device provided in the embodiment can improve the accuracy of correction, improve the participation of users, and avoid the influence of light and other conditions by receiving the position adjustment information determined based on the to-be-displayed image and driving the projection device host to move based on the position adjustment information.

[0138] It should be noted that the driving device can be a separate device, or can be a sub-component of the projection device. The specific setting can be made according to actual needs.

[0139] In some embodiments, the driving device is a pan-tilt head, and the driving component comprises: a base; a driving piece arranged on the base, comprising: a moving control shaft and a moving platform; the moving platform is used for placing the projection device host; and the moving control shaft is used for driving the projection device host to move to the preset target position according to the driving signal.

[0140] Exemplarily, as shown in FIG. 9a, the holder includes a base 901, a moving control shaft 902 and a moving platform 903. The moving platform 903 is used to place the host of the projection device. In actual application, the host of the projection device can be placed on the moving platform 903 structure of the holder, and the position of the moving platform 903 is adjusted by controlling the moving control shaft 902 of the holder, so as to realize the change of the host of the projection device in four directions of front and back, left and right, up and down and horizontal rotation, and finally move the host of the projection device to the preset target position. As shown in FIG. 9b, the host of the projection device (for example, the host of the laser television) is placed on the holder, and the position adjustment of the host of the projection device is realized by moving the holder, so as to move the host of the projection device to the preset target position.

[0141] The driving device provided in the embodiment can realize the adjustment in multiple modes such as rotation, and improve the adjustment efficiency, by adjusting the position of the host of the projection device by the holder.

[0142] In some embodiments, the driving device is a moving caster, and the moving caster includes: a caster arranged at the bottom of the host of the projection device, and used to drive the host of the projection device to move to the preset target position according to the driving signal.

[0143] Exemplarily, as shown in FIG. 10, the driving assembly of the moving caster includes a caster 1001 arranged at the bottom of the host of the projection device 1002, and the host of the projection device 1002 is moved by controlling the caster 1001. The front and back, left and right and up and down changes of the host of the projection device 1002 can be realized by controlling the caster 1001.

[0144] The driving device provided in the embodiment can reduce the cost, by adjusting the position of the host of the projection device by the caster.

[0145] In some embodiments, considering the detachability between the host of the projection device and the moving caster, the moving caster can be arranged at the bottom of the placing platform, and the host of the projection device is detachably placed on the placing platform, so that the user can detach the host of the projection device and the moving caster, and improve the assembly convenience. The driving device is a moving caster, and the moving caster includes: a placing platform used to place the host of the projection device; and a caster arranged at the bottom of the placing platform and used to drive the host of the projection device to move to the preset target position according to the driving signal.

[0146] The application also provides a projection device including the driving device in the above embodiments and the host of the projection device connected with the driving device.

[0147] The application also provides a projection system including the smart device and the projection device in the above embodiments.

[0148] The following takes the projection system as an execution subject as an example to describe how the laser television, the projector and other display devices implement the correction operation, such as the curtain correction.

[0149] Fig. 11 is a flowchart of a correction method provided by the present application. As shown in Fig. 11, the method comprises the following steps:

[0150] 1101. Displaying an operation interface by the smart device.

[0151] 1102. When the picture in the operation interface comprises the projection device host, displaying a virtual image of the projection device host in the operation interface, and the virtual image is located at a preset target position.

[0152] 1103. When the position of the projection device host in the operation interface changes, the operation interface also simultaneously indicates the path information from the current position of the projection device host to the preset target position.

[0153] The steps 1101 to 1103 in the embodiment are similar to the steps 301 to 303 in the above-described embodiment, and will not be described here again.

[0154] 1104. Controlling the driving device to move the projection device host, and obtaining updated position information.

[0155] 1105. Synchronously displaying the updated position information in the operation interface of the smart device.

[0156] Specifically, under the guidance of the virtual image of the projection device host displayed in the operation interface of the smart device and the path information from the current position of the projection device host to the preset target position, the driving device can be controlled to move the position of the projection device host, and in the moving process, the projection device host is displayed in the corresponding position in the operation interface based on the updated position information of the projection device host.

[0157] It should be noted that how to move the projection device host by the driving device can refer to the above-described embodiments, especially the description of the driving device embodiment, which will not be described here again.

[0158] The correction method provided by the embodiment can make the user control the driving device to move the projection device host to the preset target position by displaying the virtual image of the projection device host and the path information from the current position of the projection device host to the preset target position, and thus implements the correction operation, such as the curtain correction, the image correction and other correction processes. The influence of the light and other conditions can be avoided, and the accuracy of the correction can be improved.

[0159] The correction device provided by the present application comprises a display module.

[0160] The display module is used for displaying an operation interface of the intelligent device.

[0161] The display module is further configured to display a virtual image of the host computer of the projection device when the screen in the operation interface comprises the host computer of the projection device, and the virtual image is located at a preset target position.

[0162] When the position of the host computer of the projection device in the operation interface changes, the operation interface further simultaneously indicates path information of the host computer of the projection device from a current position to the preset target position.

[0163] The correction device provided in the embodiments of the present application can execute the correction method in the method embodiments, and the implementation principle and technical effects are similar, which will not be described here. It should be noted that the division of the modules is only a schematic, and the division of the modules and the naming of the modules are not limited.

[0164] The present application also provides a computer readable storage medium, which can include: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various storage program codes. Specifically, the computer readable storage medium stores program instructions, and the program instructions are used for the method in the above embodiments.

[0165] The present application also provides a program product, which includes execution instructions stored in a readable storage medium. At least one control module of the display device can read the execution instructions from the readable storage medium, and the at least one control module executes the execution instructions to make the display device implement the handwriting erasing method provided in the various embodiments.

[0166] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

[0167] In order to facilitate explanation, the above description has been made in combination with specific embodiments. However, the above exemplary discussion is not intended to exhaust or limit the embodiments to the specific forms disclosed above. According to the above teaching, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different modified embodiments suitable for specific use considerations.

Claims

1. A smart device display method, characterized by, Applied to smart devices, including: Display the user interface; When the screen in the operation interface includes the main unit of the projection device, a virtual image of the main unit of the projection device is displayed in the operation interface, and the virtual image is located at a preset target position; When the position of the projection device host changes within the operation interface, the operation interface also simultaneously indicates the path information from the current position of the projection device host to the preset target position.

2. The method of claim 1, wherein, The step of displaying the virtual image of the projection device host in the operation interface includes: Obtain the preset target location; Based on the preset target position, a virtual image of the projection device host is displayed on the screen within the operation interface.

3. The method of claim 2, wherein, The method further includes: Determine the mapping relationship between the display screen coordinate system of the smart device, the camera coordinate system of the smart device, and the real-world coordinate system; The process of obtaining the preset target location includes: Based on the mapping relationship, the scene parameters are determined; The location of the light output port of the projection device host is determined based on the scene parameters when a normal image is projected onto the projection screen. The preset target position is determined based on the location of the light output port of the projection device host.

4. The method of claim 3, wherein, Determining the mapping relationship between the display screen coordinate system of the smart device, the camera coordinate system of the smart device, and the real-world coordinate system includes: A first image is obtained by capturing a photograph of the actual usage scenario of the projection device host; the first image includes a preset calibration object located in the actual usage scenario; Based on the first image, determine the mapping relationship between the display screen coordinate system of the smart device, the camera coordinate system of the smart device, and the real-world coordinate system.

5. The method of claim 3, wherein, The determination of scene parameters based on the mapping relationship includes: A second image is obtained by capturing a photograph of the actual usage scenario of the projection device host; the second image includes the projection screen and a stand for placing the projection device host; The scene parameters of the actual use scenario are determined based on the second image and the mapping relationship; the scene parameters include at least one of the following: the relative positional relationship between the projection screen and the placement table, the size of the projection screen, and the size of the placement table.

6. The method of claim 5, wherein, The user interface includes a photo measurement control; acquiring a second image obtained by taking a photo of the actual usage scenario of the projection device host includes: In response to a touch operation on the photo measurement control, a second image is obtained by capturing the actual usage scenario of the projection device host.

7. The method of claim 2, wherein, The user interface includes a model import control; displaying the virtual image of the projection device host in the screen of the user interface includes: In response to a touch operation on the model import control, a model selection interface is displayed; the model selection interface includes multiple candidate models; the multiple candidate models include a virtual image corresponding to the projection device host; In response to a touch operation on the virtual image corresponding to the projection device host in the plurality of candidate models, a virtual image of the projection device host is displayed in a picture in the operation interface based on the preset target position.

8. The method of claim 2, wherein, The preset target position is obtained from a preset storage position. The preset target position is obtained from a preset storage position.

9. The method of claim 1, wherein, The operation interface is displayed. An actual use scene of the projection device host is photographed to obtain a target image; the actual use scene includes a projection screen of the projection device and a placement table on which the projection device host is placed. The operation interface is generated based on the target image.

10. The method according to any one of claims 1 to 9, characterized in that, After the virtual image of the projection device host is displayed in the operation interface, the following steps are further included: Position adjustment information determined based on path information indicated by the operation interface is received. The position adjustment information is sent to a driving device to move the projection device host to the preset target position.

11. The method of claim 10, wherein, The operation interface includes an adjustment region; the adjustment region includes an adjustment control; the position adjustment information determined based on the path information indicated by the operation interface is received, including: In response to a touch operation on the adjustment control based on the path information indicated by the operation interface, position adjustment information is generated.

12. A projection apparatus, characterized by comprising: The projection device includes: A projection device host receives an operation interface sent by a smart device and displays the operation interface on a projection screen; the operation interface includes an adjustment control; The projection screen displays the operation interface; A remote control component connected to the projection device host performs a touch operation on the adjustment control based on path information of the projection device host from a current position to a preset target position indicated by the operation interface; a picture in the operation interface displays a virtual image of the projection device host at the preset target position; the preset target position is a position of the projection device host when a normal picture is displayed on the projection screen; The projection device host generates position adjustment information in response to a touch operation on the adjustment control and sends the position adjustment information to a driving device to move the projection device host to the preset target position.

13. A projection system, characterized by The driving device, the smart device of any one of claims 1-11, and the projection device of claim 12. The driving device, the smart device of any one of claims 1-11, and the projection device of claim 12.

14. A correction method characterized by, An operation interface is displayed by the smart device. When a picture in the operation interface includes a projection device host, a virtual image of the projection device host is displayed in the operation interface; the virtual image is located at a preset target position. When the position of the projection device host in the operation interface changes, the operation interface also indicates path information of the projection device host from a current position to the preset target position. A driving device moves the projection device host to obtain updated position information. The updated position information is synchronously displayed on the operation interface of the smart device. ​

Images