Projection control method and device and storage medium

Abstract

The invention discloses a control method and device of projection equipment and a storage medium. The method comprises the following steps: displaying a region selection page; the area selection page comprises a frame area of at least one object of a first shot image, and the first shot image is an image obtained by shooting the projection area; when an area selection operation is detected, displaying a material selection page corresponding to the first frame area; the first frame area is a frame area selected from at least one frame area based on the position of an area selection operation point, or a preset area with the area selection operation point as the center; under the condition that a material selection operation is detected, selecting a first material from one or more materials of the material selection page; projecting the obtained target material in a projection area corresponding to the first frame area based on multiple of the first material, the coordinate of the first material and the coordinate of the first frame area; the target material comprises a first material or a second material, and the second material is a material obtained after the first material is processed.

Description

Technical Field

[0001] The present application relates to control technology, and in particular to a projection control method, device and storage medium. Background Art

[0002] Currently, projection equipment is widely used in various situations, such as office briefings, film and television playback, installation art, etc. However, the projection equipment on the market usually does not have screen touch operation, and it is impossible to realize touch function and interact with objects in the physical world on ordinary general-purpose projection equipment. Summary of the invention

[0003] The present application hopes to provide a projection control method, device and storage medium.

[0004] The technical solution of this application is implemented as follows:

[0005] In a first aspect, a projection control method is provided, the method comprising:

[0006] Displaying an area selection page; wherein the area selection page includes a frame area of ​​at least one object in a first captured image, wherein the first captured image is an image obtained by capturing a projection area;

[0007] When an area selection operation is detected, a material selection page corresponding to a first frame area is displayed; wherein the first frame area is a frame area selected from at least one frame area based on the position of the area selection operation point, or a preset area centered on the area selection operation point;

[0008] When a material selection operation is detected, selecting a first material from one or more materials on the material selection page;

[0009] Based on a first material, the coordinates of the first material and multiple coordinates of the first border area, the obtained target material is projected into a projection area corresponding to the first border area; wherein the target material includes the first material or the second material, and the second material is the material after processing the first material.

[0010] In the above solution, projecting the obtained target material into the projection area corresponding to the first frame area based on the first material, the coordinates of the first material and the coordinates of the first frame area includes:

[0011] Displaying the first material in the first frame area;

[0012] The first material is projected into a projection area corresponding to the first frame area based on the first material, the coordinates of the first material and the coordinates of the first frame area.

[0013] In the above solution, projecting the obtained target material into the projection area corresponding to the first frame area based on the first material, the coordinates of the first material and the coordinates of the first frame area includes:

[0014] Processing the first material based on the aspect ratio of the first border area and / or the display resolution of the projection device to obtain the second material;

[0015] Determine, based on the display resolution and the resolution of the second material, coordinates of the second material in a projection coordinate system when the center coordinates of the projected display are aligned with the center coordinates of the second material;

[0016] Based on the second material, the coordinates of the second material and the coordinates of the first frame area, the second material is projected into a projection area corresponding to the first frame area.

[0017] In the above solution, the processing of the first material based on the aspect ratio of the first border area and / or the display resolution of the projection device to obtain the second material includes:

[0018] When the first frame area and the first material have different aspect ratios, the first material is cropped to obtain a third material;

[0019] When the resolution width of the third material is greater than the display resolution width, or the resolution height of the third material is greater than the display resolution height, the third material is scaled to obtain the second material.

[0020] In the above scheme, the method further comprises:

[0021] When there is at least one frame area to be selected in the at least one frame area, a frame area closest to the region selection operation point is selected from the at least one frame area to be selected as the first frame area; wherein the frame area to be selected is a frame area whose distance from the region selection operation point is less than or equal to a preset distance threshold;

[0022] Alternatively, when the frame area to be selected does not exist in the at least one frame area, a preset area of ​​the area selection operation point is used as the first frame area;

[0023] Alternatively, when target identification information is received, the border area indicated by the target identification information is used as the first border area; wherein different border areas are marked by different identification information.

[0024] In the above scheme, the method further comprises:

[0025] Acquire the first captured image and the second captured image; wherein the first captured image is an image obtained by the terminal photographing the projection area, and the second captured image is an image obtained by the camera of the projection device photographing the projection area; or, the first captured image is an image obtained by the camera of the projection device photographing the projection area, and the second captured image is an image obtained by the terminal photographing the projection area;

[0026] Performing feature matching on the first captured image and the second captured image to determine a first conversion relationship between the terminal coordinate system and the camera coordinate system of the projection device;

[0027] Determine a second transformation relationship between the terminal coordinate system and the projection coordinate system based on the transformation relationship between the camera coordinate system and the projection coordinate system and the first transformation relationship;

[0028] Based on the second conversion relationship and the coordinates of the first frame area in the terminal coordinate system, the coordinates of the first frame area in the projection coordinate system are determined.

[0029] The method described in the above scheme also includes:

[0030] Acquire a feature map and a third captured image projected by the projection device; wherein the third captured image is an image obtained by a terminal capturing the feature map;

[0031] Performing feature matching on the feature map and the third captured image to determine a second conversion relationship between the terminal coordinate system and the projection coordinate system;

[0032] Based on the second conversion relationship and the coordinates of the first frame area in the terminal coordinate system, the coordinates of the first frame area in the projection coordinate system are determined.

[0033] In the above scheme, the method further comprises:

[0034] The first material or the second material is stored in a projection device.

[0035] In a second aspect, a projection control device is provided, the device comprising:

[0036] A display unit, configured to display an area selection page; wherein the area selection page includes a frame area of ​​at least one object in a first captured image, wherein the first captured image is an image obtained by the terminal capturing the projection area;

[0037] The display unit is further configured to display a material selection page corresponding to the first frame area when a region selection operation is detected; wherein the first frame area is a frame area selected from at least one frame area based on the position of the region selection operation point, or a preset area centered on the region selection operation point;

[0038] A selection unit, configured to select a first material from one or more materials on the material selection page when a material selection operation is detected;

[0039] A control unit is used to project the obtained target material into a projection area corresponding to the first border area based on the first material, the coordinates of the first material and multiple coordinates of the first border area; wherein the target material includes the first material or the second material, and the second material is the material after processing the first material.

[0040] According to a third aspect, a projection control device is provided, comprising: a processor and a memory configured to store a computer program that can be run on the processor, wherein the processor is configured to execute the steps of the method according to the first aspect when running the computer program.

[0041] According to a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program implements the steps of the method according to the first aspect when executed by a processor.

[0042] The present application provides a projection control method, device and storage medium, which capture a projection area to obtain a first captured image, display an area selection page including a border area of ​​at least one object in the first captured image, select a first border area on the area selection page, select a first material from a material selection page corresponding to the first border area, and project the first material or a second material processed from the first material into a projection area corresponding to the first border area, thereby realizing touch operation on the projection area of ​​the projection device and interaction with objects in the object world in the projection area. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a schematic diagram of a first flow chart of a projection control method in an embodiment of the present application;

[0044] Figure 2 A schematic diagram of a projection screen projected by a projection device exemplified in an embodiment of the present application;

[0045] Figure 3 (a) is a schematic diagram of a first captured image exemplified in an embodiment of the present application;

[0046] Figure 3 (b) is a first schematic diagram of an area selection page exemplified in an embodiment of the present application;

[0047] Figure 4 (a) is a second schematic diagram of an area selection page exemplified in an embodiment of the present application;

[0048] Figure 4 (b) is a schematic diagram of a material selection page used as an example in an embodiment of the present application;

[0049] Figure 5 A first schematic diagram of a projection screen and a projection screen captured by a terminal in an embodiment of the present application;

[0050] Figure 6 This is a second flow chart of the projection control method in the embodiment of the present application;

[0051] Figure 7 This is a schematic diagram of a third flow chart of the projection control method in an embodiment of the present application;

[0052] Figure 8 This is a schematic diagram of a fourth process of the projection control method in an embodiment of the present application;

[0053] Fig. 9 This is a fifth flow chart of the projection control method in the embodiment of the present application;

[0054] Fig.10 This is a sixth flow chart of the projection control method in the embodiment of the present application;

[0055] Fig.11 A second schematic diagram of a projection screen and a projection screen captured by a terminal in an embodiment of the present application;

[0056] Fig.12 A schematic diagram of the structure of the projection control device in an embodiment of the present application;

[0057] Fig.13 It is a schematic diagram of the composition structure of the projection control device in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing this embodiment and are not intended to limit this application.

[0060] In the following description, references to “some embodiments,” “this embodiment,” “this embodiment,” and examples, etc., describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

[0061] If similar descriptions of "first / second" appear in the application documents, the following instructions are added. In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the present embodiment described here can be implemented in an order other than that illustrated or described here.

[0062] In this embodiment, the term "and / or" is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, object A and / or object B may represent three situations: object A exists alone, object A and object B exist at the same time, and object B exists alone.

[0063] The embodiment of the present application provides a projection control method, Figure 1 Schematic diagram of the first process of the projection control method in the embodiment of the present application. Figure 1 As shown, the projection control method may include:

[0064] Step 101: displaying an area selection page; wherein the area selection page includes a frame area of ​​at least one object in a first captured image, and the first captured image is an image obtained by capturing a projection area.

[0065] The projection area refers to the area that the projection light of the projection device can illuminate. The projection area may include at least one object, and the first captured image is obtained by capturing at least one object in the projection area. The projection area may also include a projection picture (such as a projection image) projected by the projection device and including at least one object. Figure 2 As shown), a first captured image is obtained by capturing a projection image of the projection area.

[0066] Step 102: When an area selection operation is detected, a material selection page corresponding to the first border area is displayed; wherein the first border area is a border area selected from at least one border area based on the position of the area selection operation point, or a preset area centered on the area selection operation point.

[0067] Step 103: When a material selection operation is detected, a first material is selected from one or more materials on the material selection page.

[0068] Step 104: Based on the first material, the coordinates of the first material and multiple coordinates of the first border area, project the obtained target material into the projection area corresponding to the first border area; wherein the target material includes the first material or the second material, and the second material is the material after processing the first material.

[0069] In a feasible implementation, the executor of the projection control method may be a processor of a projection device. Based on this, the projection device displays an area selection page; when the projection device detects an area selection operation, it displays a material selection page corresponding to the first frame area; when the projection device detects a material selection operation, it selects a first material from one or more materials on the material selection page; the projection device projects the obtained target material into a projection area corresponding to the first frame area based on the first material, the coordinates of the first material, and the coordinates of the first frame area; wherein the target material includes the first material or the second material, and the second material is the material processed from the first material. In this way, through the projection device, touch operation of the projection area and interaction with objects in the object world in the projection area are realized.

[0070] In another feasible implementation, the execution subject of the projection control method may be a processor of a terminal. Based on this, the terminal displays an area selection page; when the terminal detects an area selection operation, it displays a material selection page corresponding to the first frame area; when the terminal detects a material selection operation, it selects a first material from one or more materials on the material selection page; based on the first material, the coordinates of the first material, and the coordinates of the first frame area, the terminal controls the projection device to project the obtained target material into the projection area corresponding to the first frame area; wherein the target material includes the first material or the second material, and the second material is the material processed from the first material. In this way, through the terminal, touch operation of the projection area of ​​the projection device and interaction with objects in the object world in the projection area are realized.

[0071] In some embodiments, step 101 may include: acquiring the first captured image;

[0072] Performing frame recognition on the first captured image to obtain a frame area of ​​at least one object in the first captured image;

[0073] The area selection page including a frame area of ​​at least one object in the first captured image is displayed.

[0074] In a feasible implementation, the projection device acquires a first captured image, performs border recognition on the first captured image based on a border recognition algorithm, obtains a border area of ​​at least one object in the first captured image, and displays an area selection page including the border area of ​​at least one object in the first captured image.

[0075] In some embodiments, acquiring the first captured image includes:

[0076] The projection device captures at least one object in the projection area to obtain the first captured image.

[0077] In some embodiments, acquiring the first captured image includes:

[0078] When the projection device projects a projection picture including at least one object, the projection device captures the projection area to obtain the first captured image.

[0079] In another feasible implementation, the terminal obtains the first captured image, performs frame recognition on the first captured image based on a frame recognition algorithm, obtains a frame area of ​​at least one object in the first captured image, and displays a region selection page including the frame area of ​​at least one object in the first captured image. Figure 3 (a) shows the area selection page. Figure 3 As shown in (b), the area selection page includes the border areas of three picture frames, the border areas of three pillows, the border area of ​​the sofa, the border area of ​​the vase and the border area of ​​the table (the border areas are marked with thick lines).

[0080] In some embodiments, acquiring the first captured image includes:

[0081] The terminal captures at least one object in the projection area to obtain the first captured image.

[0082] In some embodiments, acquiring the first captured image includes:

[0083] The terminal sends a control instruction to the projection device, so that the projection device responds to the control instruction and projects a projection picture including at least one object;

[0084] The terminal captures the projection area to obtain the first captured image.

[0085] The control instruction is used to instruct the projection device to project a projection picture including at least one object. The control instruction can be an instruction generated by triggering a single key or a combination of keys.

[0086] Exemplarily, the user manually clicks to select the first border area (eg Figure 4(a) The user manually selects the border area of ​​the middle picture frame on the terminal. When the terminal detects the area selection operation, it selects a first border area from at least one border area based on the position of the area selection operation point, or a preset area centered on the area selection operation point as the first border area. When the terminal detects the area selection operation, it displays the material selection page corresponding to the first border area (for example, Figure 4 (b) The part indicated by the dotted box.

[0087] like Figure 5 As shown, the user manually clicks on the circular area on the area selection page on the terminal to determine that the frame area including the circular area is the first frame area.

[0088] In some embodiments, the method further comprises:

[0089] When there is at least one border area to be selected among the at least one border area, the border area closest to the area selection operation point is selected from the at least one border area to be selected as the first border area; wherein the border area to be selected is a border area whose distance from the area selection operation point is less than or equal to a preset distance threshold.

[0090] Specifically, multiple distances between multiple points on a second border area and an area selection operation point are determined, wherein the second border area is any one of at least one border area; the shortest distance among the multiple distances is determined; when the shortest distance is determined to be less than or equal to a preset distance threshold, the second border area is used as a border area to be selected; at least one border area to be selected is obtained; based on at least one shortest distance corresponding to at least one border area to be selected, a border area to be selected that is closest to the area selection operation point is selected from at least one border area to be selected as the first border area.

[0091] In some embodiments, the method further comprises:

[0092] When the frame area to be selected does not exist in the at least one frame area, a preset area of ​​the area selection operation point is used as the first frame area.

[0093] Specifically, multiple distances between multiple points on the second border area and the area selection operation point are determined, wherein the second border area is any one of the at least one border area; the shortest distance among the multiple distances is determined; the shortest distance corresponding to each border area in the at least one border area is determined, and when the distance between the points and the area selection operation point is greater than a preset distance threshold, it means that there is no border area to be selected in the at least one border area, and the preset area centered on the area selection operation point is used as the first border area.

[0094] In some embodiments, the method further comprises:

[0095] When a target identifier is received, the border area indicated by the target identifier is used as the first border area; wherein different border areas are marked by different identification information.

[0096] In the embodiment of the present application, by marking at least one border area, the first border area can be quickly determined according to the target identifier, thereby improving the recognition efficiency of the border area.

[0097] For example, the border area is marked by a digital identifier, the number 1 marks the border area 1, the number 2 marks the border area 2, the number 3 marks the border area 3, and so on.

[0098] In an embodiment of the present application, the area selection page may further include an input box, through which an identifier is input, and after the terminal receives the identifier, the first border area is determined based on the identifier; for example, when the number 2 is received, border area 2 is used as the first border area. Alternatively, the terminal includes a voice unit, and by inputting a voice identifier, after the terminal receives the voice identifier through the voice unit, the first border area is determined based on the voice identifier. The embodiments of the present application include but are not limited to these two methods.

[0099] The process of determining the first border area in the embodiment of the present application can be applied to a projection device or a terminal.

[0100] In some embodiments, the material types of the material on the material selection page include first local material, second local material and network material; wherein the first local material is the local material of the electronic photo album, and the second local material is a pre-established local material related to the area type of the first border area.

[0101] In the embodiment of the present application, the material selection page includes three types of materials:

[0102] Type 1: The second local material related to the area type of the first border area. For example, a home scene can be established in advance, with different area types, such as pillows, picture frames, vases, etc. on the sofa, and each type corresponds to multiple materials, such as a pillow on the sofa corresponds to multiple pillow images, a picture frame corresponds to multiple home decoration painting images, a vase corresponds to multiple plant images, and so on.

[0103] Type 2: The first local material of the electronic photo album. In this way, the user can jump to the album by clicking the album button and then select the material he likes from it.

[0104] Type 3: Internet web material Here, the web material of the corresponding type may be presented according to the area type of the first frame area.

[0105] In some embodiments, step 104 may include Figure 6 Steps shown:

[0106] Step 601: Display a first material in a first frame area.

[0107] The first material may be aligned and displayed in the first frame area manually, or in other ways, which are not limited here.

[0108] Step 602: Based on the first material, the coordinates of the first material and the coordinates of the first frame area, project the first material into a projection area corresponding to the first frame area.

[0109] In a feasible implementation method, the first material is displayed in the first border area on the terminal, and the terminal sends the first material, the coordinates of the first material and the coordinates of the first border area to the projection device, so that the projection device projects the first material in the projection area corresponding to the first border area based on the first material, the coordinates of the first material and the coordinates of the first border area.

[0110] Specifically, the terminal sends the first material, the coordinates of the first material in the terminal coordinate system, and the coordinates of the first border area in the terminal coordinate system to the projection device, so that the projection device converts the coordinates of the first material in the terminal coordinate system into the coordinates of the first material in the projection coordinate system, and converts the coordinates of the first border area in the terminal coordinate system into the coordinates of the first border area in the projection coordinate system based on the second conversion relationship between the terminal coordinate system and the projection coordinate system. Then, the projection device projects the first material in the projection area corresponding to the first border area based on the first material, the coordinates of the first material in the projection coordinate system, and the coordinates of the first border area in the projection coordinate system.

[0111] Alternatively, the terminal converts the coordinates of the first material in the terminal coordinate system into the coordinates of the first material in the projection coordinate system, and converts the coordinates of the first border area in the terminal coordinate system into the coordinates of the first border area in the projection coordinate system based on a second conversion relationship between the terminal coordinate system and the projection coordinate system. The terminal then sends the first material, the coordinates of the first material in the projection coordinate system, and the coordinates of the first border area in the projection coordinate system to the projection device, so that the projection device projects the first material in the projection area corresponding to the first border area based on the first material, the coordinates of the first material in the projection coordinate system, and the coordinates of the first border area in the projection coordinate system.

[0112] In another feasible implementation, the first material is displayed in a first border area on a projection device, and the projection device projects the first material into a projection area corresponding to the first border area based on the first material, the coordinates of the first material in a projection coordinate system, and the coordinates of the first border area in the projection coordinate system.

[0113] In some embodiments, step 104 may also include Figure 7 Steps shown:

[0114] Step 701: Process the first material based on the aspect ratio of the first border area and / or the display resolution of the projection device to obtain the second material.

[0115] Step 702: Based on the display resolution and the resolution of the second material, determine the coordinates of the second material in the projection coordinate system when the center coordinates of the projected display are aligned with the center coordinates of the second material.

[0116] Step 703: Based on the second material, the coordinates of the second material and the coordinates of the first frame area, project the second material into the projection area corresponding to the first frame area.

[0117] In a feasible implementation, the projection device determines the aspect ratio of the first border area, and processes the first material based on the aspect ratio of the first border area and / or the display resolution to obtain the second material; the projection device determines the coordinates of the second material in the projection coordinate system when the center coordinates of the projected display are aligned with the center coordinates of the second material based on the display resolution and the resolution of the second material; the projection device projects the second material into the projection area corresponding to the first border area based on the second material, the coordinates of the second material in the projection coordinate system, and the coordinates of the first border area in the projection coordinate system.

[0118] In another feasible implementation, the terminal determines the aspect ratio of the first border area, processes the first material based on the aspect ratio of the first border area and / or the display resolution of the projection device (which may be pre-set or sent by the projection device) to obtain the second material; the terminal determines the coordinates of the second material in the projection coordinate system when the center coordinates of the projected display are aligned with the center coordinates of the second material based on the display resolution and the resolution of the second material; the terminal sends the second material, the coordinates of the second material in the projection coordinate system, and the coordinates of the first border area in the terminal coordinate system to the projection device, so that the projection device converts the coordinates of the first border area in the terminal coordinates into the coordinates of the first border area in the projection coordinate system based on the second conversion relationship between the terminal coordinate system and the projection coordinate system, and then sets the coordinates of the second material in the projection coordinate system as the coordinates of the first border area in the projection coordinate system, so as to automatically align the second material in the projection area corresponding to the first border area.

[0119] Alternatively, the terminal determines the aspect ratio of the first border area, and processes the first material based on the aspect ratio of the first border area and / or the display resolution of the projection device (which may be pre-set or sent by the projection device) to obtain the second material; the terminal determines the coordinates of the second material in the projection coordinate system when the center coordinates of the projected display are aligned with the center coordinates of the second material based on the display resolution and the resolution of the second material; the terminal converts the coordinates of the first border area in the terminal coordinates into the coordinates of the first border area in the projection coordinate system, and then based on a second conversion relationship between the terminal coordinate system and the projection coordinate system, sends the second material, the coordinates of the second material in the projection coordinate system, and the coordinates of the first border area in the projection coordinate system to the projection device, so that the projection device sets the coordinates of the second material in the projection coordinate system as the coordinates of the first border area in the projection coordinate system, so as to automatically align the second material in the projection area corresponding to the first border area.

[0120] In some embodiments, step 701 may include:

[0121] When the first frame area and the first material have different aspect ratios, the first material is cropped to obtain a third material;

[0122] When the resolution width of the third material is greater than the display resolution width, or the resolution height of the third material is greater than the display resolution height, the third material is scaled to obtain the second material.

[0123] For example, since the aspect ratio of the first frame area is different from that of the first material, the first material needs to be cropped according to the aspect ratio of the first frame area during automatic alignment display. For example, the aspect ratio of the first frame area is 1:1.2, and the first material is cropped according to the aspect ratio of the first frame area based on the important content of the first material to obtain the third material.

[0124] Next, because the resolution width of the third material is greater than the display resolution width, or the resolution height of the third material is greater than the display resolution height, the third material needs to be scaled based on the display resolution to obtain the second material.

[0125] For example, the display resolution of the projection device is 1920*1080, and the resolution of the third material is 1000*1200. Since the resolution height of the third material is greater than the display resolution height, the third material is scaled based on the display resolution 1920*1080 to obtain the scaled second material. Here, when Dw and Dh represent the display resolution width and the display resolution height, respectively, and Iw and Ih represent the resolution width and resolution height of the third material after cropping, then Iw'=Dh / Ih*Iw=1080 / 1200*1000=900, Iw' represents the resolution width of the second material, and the resolution height Ih' of the second material is 1080, that is, the resolution of the second material is 900*1080.

[0126] Dw and Dh represent the display resolution width and the display resolution height respectively, Iw' and Ih' represent the resolution width and the resolution height respectively of the second material. When the center coordinates of the projected display are aligned with the center coordinates of the second material, the coordinates of the second material in the projection coordinate system include: lower left corner point (Dw / 2-Iw' / 2, Dh / 2-Ih' / 2), lower right corner point (Dw / 2+Iw' / 2, Dh / 2-Ih' / 2), upper right corner point (Dw / 2+Iw' / 2, Dh / 2+Ih' / 2), and upper left corner point (Dw / 2-Iw' / 2, Dh / 2+Ih' / 2).

[0127] Based on this, the coordinates of the second material in the projection coordinate system include: lower left corner point (1920 / 2-900 / 2, 1080 / 2-1080 / 2), lower right corner point (1920 / 2+900 / 2, 1080 / 2-1080 / 2), upper right corner point (1920 / 2+900 / 2, 1080 / 2+1080 / 2), and upper left corner point (1920 / 2-900 / 2, 1080 / 2+1080 / 2).

[0128] If the coordinates in the first frame area in the terminal coordinate system are assumed to be (x1, y1), (x2, y2), (x3, y3) and (x4, y4), then based on the conversion relationship H between the terminal coordinate system and the projection coordinate system, the coordinates of the first frame area in the projection coordinate system are determined to be (X1, Y1), (X2, Y2), (X3, Y3) and (X4, Y4), where (Xi, Yi, 1) = (xi, yi, 1) * H. Finally, the coordinates of the second material in the projection coordinate system are set to the coordinates of the first frame area in the projection coordinate system to automatically align the second material in the projection area corresponding to the first frame area.

[0129] In some embodiments, the method further comprises:

[0130] When the first border area has the same aspect ratio as the first material, no cropping operation is performed on the first material.

[0131] In some embodiments, the method further comprises:

[0132] When the resolution width of the third material is less than or equal to the display resolution width, or the resolution height of the third material is less than or equal to the display resolution height, no scaling operation is performed on the third material.

[0133] In some embodiments, the method further comprises:

[0134] The first material or the second material is stored in a projection device.

[0135] In a feasible implementation, the projection device stores the first material or the second material, so that the material can be automatically loaded when the projection device is turned on next time.

[0136] In another feasible implementation, the terminal application automatically uploads the first material or the second material to the projection device for storage, so that the material can be automatically loaded when the terminal application is closed or the projection device is turned on next time.

[0137] In the embodiment of the present application, when the coordinates of the first frame area are the coordinates of the first frame area in the projection coordinate system, the process of determining the coordinates of the first frame area in the projection coordinate system includes: Figure 8 Steps shown:

[0138] Step 801: Acquire a first captured image and a second captured image.

[0139] The projection area refers to the area that the projection light of the projection device can illuminate. The projection area may include at least one object, and the first captured image is obtained by capturing at least one object in the projection area. The projection area may also include a projection picture including at least one object projected by the projection device.

[0140] Step 802: Perform feature matching on the first captured image and the second captured image to determine a first conversion relationship between the terminal coordinate system and the camera coordinate system of the projection device.

[0141] Step 803: Based on the conversion relationship between the camera coordinate system and the projection coordinate system and the first conversion relationship, determine a second conversion relationship between the terminal coordinate system and the projection coordinate system.

[0142] Here, the first conversion relationship h between the terminal coordinate system and the camera coordinate system of the projection device is due to the conversion relationship h1 between the camera coordinate system and the projection coordinate system. Therefore, the second conversion relationship H=h*h1 between the terminal coordinate system and the projection coordinate system.

[0143] That is, without projecting the feature map, the second conversion relationship between the terminal coordinate system and the projection coordinate system is determined based on the first captured image and the second captured image.

[0144] Step 804: Determine the coordinates of the first frame area in the projection coordinate system based on the second conversion relationship and the coordinates of the first frame area in the terminal coordinate system.

[0145] In the embodiment of the present application, the determination of the second conversion relationship can be performed on the projection device or on the terminal.

[0146] In a feasible implementation, the projection device acquires a first captured image and a second captured image; the projection device performs feature matching on the first captured image and the second captured image to determine a first conversion relationship between the terminal coordinate system and the camera coordinate system of the projection device; the projection device determines a second conversion relationship between the terminal coordinate system and the projection coordinate system based on the conversion relationship between the camera coordinate system and the projection coordinate system and the first conversion relationship. The first captured image is an image obtained by the projection device when capturing the projection area, the second captured image is an image obtained by the terminal when capturing the projection area, and the second captured image acquired by the projection device is sent by the terminal.

[0147] Based on this, when the executor of the projection control method is the processor of the terminal, the terminal sends the coordinates of the first border area in the terminal coordinate system to the projection device, and the projection device determines the coordinates of the first border area in the projection coordinate system based on the second conversion relationship and the coordinates of the first border area in the terminal coordinate system; or, the terminal first determines the coordinates of the first border area in the projection coordinate system based on the second conversion relationship (the second conversion relationship may be sent by the projection device) and the coordinates of the first border area in the terminal coordinate system, and then sends the coordinates of the first border area in the projection coordinate system to the projection device.

[0148] Correspondingly, the terminal sends the coordinates of the first material in the terminal coordinate system to the projection device, and the projection device determines the coordinates of the first material in the projection coordinate system based on the second conversion relationship and the coordinates of the first material in the terminal coordinate system; or, the terminal first determines the coordinates of the first material in the projection coordinate system based on the second conversion relationship (the second conversion relationship may be sent by the projection device) and the coordinates of the first material in the terminal coordinate system, and then sends the coordinates of the first material in the projection coordinate system to the projection device.

[0149] In another feasible implementation, the terminal acquires the first and second captured images; the terminal performs feature matching on the first and second captured images to determine the first conversion relationship between the terminal coordinate system and the camera coordinate system of the projection device; the terminal determines the second conversion relationship between the terminal coordinate system and the projection coordinate system based on the conversion relationship between the camera coordinate system and the projection coordinate system and the first conversion relationship. The first captured image is an image obtained by the terminal capturing the projection area, the second captured image is an image obtained by the camera of the projection device capturing the projection area, and the second captured image acquired by the terminal is sent by the projection device.

[0150] Based on this, when the executor of the projection control method is the processor of the terminal, the terminal first determines the coordinates of the first border area in the projection coordinate system based on the second conversion relationship and the coordinates of the first border area in the terminal coordinate system, and then sends the coordinates of the first border area in the projection coordinate system to the projection device; or, the terminal sends the coordinates of the first border area in the terminal coordinate system to the projection device, and the projection device determines the coordinates of the first border area in the projection coordinate system based on the second conversion relationship (the second conversion relationship may be sent by the terminal) and the coordinates of the first border area in the terminal coordinate system.

[0151] Correspondingly, the terminal first determines the coordinates of the first material in the projection coordinate system based on the second conversion relationship and the coordinates of the first material in the terminal coordinate system, and then sends the coordinates of the first material in the projection coordinate system to the projection device; or, the terminal sends the coordinates of the first material in the terminal coordinate system to the projection device, and the projection device determines the coordinates of the first material in the projection coordinate system based on the second conversion relationship (the second conversion relationship may be sent by the terminal) and the coordinates of the first material in the terminal coordinate system.

[0152] In the embodiment of the present application, when the coordinates of the first frame area are the coordinates of the first frame area in the projection coordinate system, the process of determining the coordinates of the first frame area in the projection coordinate system may also include: Fig. 9 Steps shown:

[0153] Step 901: Acquire a feature map projected by a projection device and a third captured image; wherein the third captured image is an image obtained by capturing the feature map by a terminal.

[0154] Step 902: Perform feature matching on the feature map and the third captured image to determine a second conversion relationship between the terminal coordinate system and the projection coordinate system.

[0155] That is, in the case of projecting the feature map, the second conversion relationship between the terminal coordinate system and the projection coordinate system is determined based on the feature map and the third captured image.

[0156] Step 903: Determine the coordinates of the first frame area in the projection coordinate system based on the second conversion relationship and the coordinates of the first frame area in the terminal coordinate system.

[0157] In a feasible implementation manner, the terminal receives a feature map sent by the projection device, and the terminal determines a second conversion relationship between the terminal coordinate system and the projection coordinate system based on the feature map and the third captured image.

[0158] Based on this, when the executor of the projection control method is the processor of the terminal, the terminal first determines the coordinates of the first border area in the projection coordinate system based on the second conversion relationship and the coordinates of the first border area in the terminal coordinate system, and then sends the coordinates of the first border area in the projection coordinate system to the projection device; or, the terminal sends the coordinates of the first border area in the terminal coordinate system to the projection device, and the projection device determines the coordinates of the first border area in the projection coordinate system based on the second conversion relationship (the second conversion relationship may be sent by the terminal) and the coordinates of the first border area in the terminal coordinate system.

[0159] Correspondingly, the terminal first determines the coordinates of the first material in the projection coordinate system based on the second conversion relationship and the coordinates of the first material in the terminal coordinate system, and then sends the coordinates of the first material in the projection coordinate system to the projection device; or, the terminal sends the coordinates of the first material in the terminal coordinate system to the projection device, and the projection device determines the coordinates of the first material in the projection coordinate system based on the second conversion relationship (the second conversion relationship may be sent by the terminal) and the coordinates of the first material in the terminal coordinate system.

[0160] In another feasible implementation manner, the projection device receives the third captured image sent by the terminal, and the projection device determines a second conversion relationship between the terminal coordinate system and the projection coordinate system based on the feature map and the third captured image.

[0161] Based on this, when the executor of the projection control method is the processor of the terminal, the terminal first determines the coordinates of the first border area in the projection coordinate system based on the second conversion relationship (the second conversion relationship may be sent by the projection device) and the coordinates of the first border area in the terminal coordinate system, and then sends the coordinates of the first border area in the projection coordinate system to the projection device; or, the terminal sends the coordinates of the first border area in the terminal coordinate system to the projection device, and the projection device determines the coordinates of the first border area in the projection coordinate system based on the second conversion relationship and the coordinates of the first border area in the terminal coordinate system.

[0162] Correspondingly, the terminal first determines the coordinates of the first material in the projection coordinate system based on the second conversion relationship (the second conversion relationship may be sent by the projection device) and the coordinates of the first material in the terminal coordinate system, and then sends the coordinates of the first material in the projection coordinate system to the projection device; or, the terminal sends the coordinates of the first material in the terminal coordinate system to the projection device, and the projection device determines the coordinates of the first material in the projection coordinate system based on the second conversion relationship and the coordinates of the first material in the terminal coordinate system.

[0163] In the embodiment of the present application, based on the first material, the coordinates of the first material and the coordinates of the first frame area, after projecting the obtained target material into the projection area corresponding to the first frame area, the method may include: Fig.10 One or more of the following steps:

[0164] Step 1001: When a drag operation is detected, a target material is dragged from a first position to a second position based on a drag instruction.

[0165] Step 1002: When a zoom operation is detected, zoom the target material based on the zoom instruction.

[0166] Step 1003: When a long press operation is detected, jump to the corresponding content based on the long press instruction.

[0167] In a feasible implementation, when the user does not need to display the target material in the projection area corresponding to the first frame area, the user can manually drag the target material on the projection device side, so that when the projection device detects the dragging operation, it responds to the dragging instruction and drags the target material from the first position to the second position. The user can also manually scale the target material on the projection device side, so that when the projection device detects the scaling operation, it responds to the scaling instruction and enlarges or reduces the target material. The user can also manually long press the target material on the projection device side, so that when the projection device detects the long press operation, it responds to the long press instruction and jumps to the corresponding option content.

[0168] In another feasible implementation, when the user does not need to display the target material in the projection area corresponding to the first frame area, the user can manually drag the target material in the terminal. In this way, when the terminal detects the dragging operation, the dragging instruction is sent to the projection device, and the projection device responds to the dragging instruction and drags the target material from the first position to the second position. The user can also manually zoom the target material in the terminal. In this way, when the terminal detects the zooming operation, the zooming instruction is sent to the projection device, and the projection device responds to the zooming instruction to enlarge or reduce the target material. The user can also manually long press the target material in the terminal. In this way, when the terminal detects the long press operation, the long press instruction is sent to the projection device, and the projection device responds to the long press instruction and jumps to the corresponding option content.

[0169] For example, Fig.11 As shown, when the user does not need to display the target material in the projection area corresponding to the first frame area, after the user drags or scales the two images on the display page of the terminal, the projection screen of the projection device will synchronize the two images after dragging or scaling, and its content and position are consistent with the two images on the display page on the terminal. That is, the touch operation of the projection screen projected by the projection device is realized through the terminal.

[0170] In order to implement the method of the embodiment of the present application, based on the same inventive concept, the embodiment of the present application further provides a projection control device, Fig.12 Schematic diagram of the structure of the projection control device in the embodiment of the present application. Fig.12 As shown, the projection control device 120 includes:

[0171] The display unit 1201 is used to display an area selection page; wherein the area selection page includes a frame area of ​​at least one object in a first captured image, and the first captured image is an image obtained by capturing a projection area;

[0172] The display unit 1201 is further configured to display a material selection page corresponding to a first frame area when a region selection operation is detected; wherein the first frame area is a frame area selected from at least one frame area based on the position of the region selection operation point, or a preset area centered on the region selection operation point;

[0173] A selection unit 1202, configured to select a first material from one or more materials on the material selection page when a material selection operation is detected;

[0174] The control unit 1203 is used to project the obtained target material into the projection area corresponding to the first border area based on the first material, the coordinates of the first material and multiple coordinates of the first border area; wherein the target material includes the first material or the second material, and the second material is the material after processing the first material.

[0175] In an embodiment of the present application, a projection area is photographed to obtain a first captured image, an area selection page including a border area of ​​at least one object in the first captured image is displayed, the first border area on the area selection page is selected, the first material on the material selection page corresponding to the first border area is selected, and the first material or the second material processed from the first material is projected into the projection area corresponding to the first border area. In this way, touch operation on the projection area of ​​the projection device and interaction with objects in the object world in the projection area are implemented.

[0176] In some embodiments, the control unit 1203 is further used to display the first material in the first border area; based on the first material, the coordinates of the first material and the coordinates of the first border area, project the first material in the projection area corresponding to the first border area.

[0177] In some embodiments, the control unit 1203 is further configured to process the first material based on the aspect ratio of the first frame area and / or the display resolution of the projection device to obtain the second material;

[0178] Determine, based on the display resolution and the resolution of the second material, coordinates of the second material in a projection coordinate system when the center coordinates of the projected display are aligned with the center coordinates of the second material;

[0179] Based on the second material, the coordinates of the second material and the coordinates of the first frame area, the second material is projected into a projection area corresponding to the first frame area.

[0180] In some embodiments, the control unit 1203 is further configured to, when the aspect ratio of the first frame area and the first material is different, crop the first material to obtain a third material;

[0181] When the resolution width of the third material is greater than the display resolution width, or the resolution height of the third material is greater than the display resolution height, the third material is scaled to obtain the second material.

[0182] In some embodiments, the selection unit 1202 is further configured to select, when there is at least one frame region to be selected in the at least one frame region, a frame region closest to the region selection operation point from the at least one frame region to be selected as the first frame region; wherein the frame region to be selected is a frame region whose distance from the region selection operation point is less than or equal to a preset distance threshold;

[0183] Alternatively, when the frame area to be selected does not exist in the at least one frame area, a preset area of ​​the area selection operation point is used as the first frame area;

[0184] Alternatively, when target identification information is received, the border area indicated by the target identification information is used as the first border area; wherein different border areas are marked by different identification information.

[0185] In some embodiments, the control unit 1203 is further used to obtain the first captured image and the second captured image; wherein the first captured image is an image obtained by the terminal photographing the projection area, and the second captured image is an image obtained by the camera of the projection device photographing the projection area; or, the first captured image is an image obtained by the camera of the projection device photographing the projection area, and the second captured image is an image obtained by the terminal photographing the projection area;

[0186] Performing feature matching on the first captured image and the second captured image to determine a first conversion relationship between the terminal coordinate system and the camera coordinate system of the projection device;

[0187] Determine a second transformation relationship between the terminal coordinate system and the projection coordinate system based on the transformation relationship between the camera coordinate system and the projection coordinate system and the first transformation relationship;

[0188] Based on the second conversion relationship and the coordinates of the first frame area in the terminal coordinate system, the coordinates of the first frame area in the projection coordinate system are determined.

[0189] In some embodiments, the control unit 1203 is further configured to obtain a feature map and a third captured image projected by the projection device; wherein the third captured image is an image obtained by the terminal capturing the feature map;

[0190] Performing feature matching on the feature map and the third captured image to determine a second conversion relationship between the terminal coordinate system and the projection coordinate system;

[0191] Based on the second conversion relationship and the coordinates of the first frame area in the terminal coordinate system, the coordinates of the first frame area in the projection coordinate system are determined.

[0192] In some embodiments, the storage unit is used to store the first material or the second material in the projection device.

[0193] The present application also provides another projection control device. Fig.13 Schematic diagram of the structure of the projection control device in the embodiment of the present application. Fig.13 As shown, the projection control device 130 includes: a processor 1301 and a memory 1302 configured to store a computer program that can be run on the processor;

[0194] The processor 1301 is configured to execute the method steps in the aforementioned embodiment when running a computer program.

[0195] Of course, in practical applications, Fig.13 As shown, the various components in the projection control device 130 are coupled together via a bus system 1303. It is understood that the bus system 1303 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 1303 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig.13 Various buses are labeled as bus system 1303.

[0196] In practical applications, the processor may be at least one of an application-specific integrated circuit (ASIC), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a controller, a microcontroller, and a microprocessor. It is understandable that for different devices, the electronic device used to implement the functions of the processor may also be other, and the embodiments of the present application do not specifically limit this.

[0197] The above-mentioned memory can be a volatile memory (volatile memory), such as a random access memory (RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD); or a combination of the above-mentioned types of memory, and provide instructions and data to the processor.

[0198] In an exemplary embodiment, the present application also provides a computer-readable storage medium for storing a computer program.

[0199] Optionally, the computer-readable storage medium can be applied to any one of the methods in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the processor in each method in the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0200] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0201] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0202] In addition, all functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units. A person of ordinary skill in the art may understand that all or part of the steps of implementing the above method embodiments may be completed by hardware related to program instructions, and the above program may be stored in a computer-readable storage medium, which, when executed, executes the steps of the above method embodiments; and the above storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0203] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0204] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0205] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0206] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A projection control method, characterized in that: The method comprises: Displaying an area selection page; wherein the area selection page includes a frame area of ​​at least one object in a first captured image, wherein the first captured image is an image obtained by capturing a projection area; When an area selection operation is detected, a material selection page corresponding to a first frame area is displayed; wherein the first frame area is a frame area selected from at least one frame area based on the position of the area selection operation point, or a preset area centered on the area selection operation point; When a material selection operation is detected, selecting a first material from one or more materials on the material selection page; Based on a first material, the coordinates of the first material and multiple coordinates of the first border area, the obtained target material is projected into a projection area corresponding to the first border area; wherein the target material includes the first material or the second material, and the second material is the material after processing the first material.

2. The method according to claim 1, characterized in that The step of projecting the obtained target material into a projection area corresponding to the first frame area based on the first material, the coordinates of the first material, and the coordinates of the first frame area includes: Displaying the first material in the first frame area; The first material is projected into a projection area corresponding to the first frame area based on the first material, the coordinates of the first material and the coordinates of the first frame area.

3. The method according to claim 1, characterized in that The step of projecting the obtained target material into a projection area corresponding to the first frame area based on the first material, the coordinates of the first material, and the coordinates of the first frame area includes: Processing the first material based on the aspect ratio of the first border area and / or the display resolution of the projection device to obtain the second material; Determine, based on the display resolution and the resolution of the second material, coordinates of the second material in a projection coordinate system when the center coordinates of the projected display are aligned with the center coordinates of the second material; Based on the second material, the coordinates of the second material and the coordinates of the first frame area, the second material is projected into a projection area corresponding to the first frame area.

4. The method according to claim 3, characterized in that: The processing of the first material based on the aspect ratio of the first border area and / or the display resolution of the projection device to obtain the second material includes: When the first frame area and the first material have different aspect ratios, the first material is cropped to obtain a third material; When the resolution width of the third material is greater than the display resolution width, or the resolution height of the third material is greater than the display resolution height, the third material is scaled to obtain the second material.

5. The method according to claim 1, characterized in that The method further comprises: When there is at least one frame area to be selected in the at least one frame area, a frame area closest to the region selection operation point is selected from the at least one frame area to be selected as the first frame area; wherein the frame area to be selected is a frame area whose distance from the region selection operation point is less than or equal to a preset distance threshold; Alternatively, when the frame area to be selected does not exist in the at least one frame area, a preset area of ​​the area selection operation point is used as the first frame area; Alternatively, when target identification information is received, the border area indicated by the target identification information is used as the first border area; wherein different border areas are marked by different identification information.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Acquire the first captured image and the second captured image; wherein the first captured image is an image obtained by the terminal photographing the projection area, and the second captured image is an image obtained by the camera of the projection device photographing the projection area; or, the first captured image is an image obtained by the camera of the projection device photographing the projection area, and the second captured image is an image obtained by the terminal photographing the projection area; Performing feature matching on the first captured image and the second captured image to determine a first conversion relationship between the terminal coordinate system and the camera coordinate system of the projection device; Determine a second transformation relationship between the terminal coordinate system and the projection coordinate system based on the transformation relationship between the camera coordinate system and the projection coordinate system and the first transformation relationship; Based on the second conversion relationship and the coordinates of the first frame area in the terminal coordinate system, the coordinates of the first frame area in the projection coordinate system are determined.

7. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Acquire a feature map and a third captured image projected by the projection device; wherein the third captured image is an image obtained by a terminal capturing the feature map; Performing feature matching on the feature map and the third captured image to determine a second conversion relationship between the terminal coordinate system and the projection coordinate system; Based on the second conversion relationship and the coordinates of the first frame area in the terminal coordinate system, the coordinates of the first frame area in the projection coordinate system are determined.

8. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: The first material or the second material is stored in a projection device.

9. A projection control device, characterized in that: The device comprises: A display unit, configured to display an area selection page; wherein the area selection page includes a frame area of ​​at least one object in a first captured image, wherein the first captured image is an image obtained by capturing a projection area; The display unit is further configured to display a material selection page corresponding to the first frame area when a region selection operation is detected; wherein the first frame area is a frame area selected from at least one frame area based on the position of the region selection operation point, or a preset area centered on the region selection operation point; A selection unit, configured to select a first material from one or more materials on the material selection page when a material selection operation is detected; A control unit is used to project the obtained target material into a projection area corresponding to the first border area based on the first material, the coordinates of the first material and multiple coordinates of the first border area; wherein the target material includes the first material or the second material, and the second material is the material after processing the first material.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 8 are implemented.

Images