Projection method and device, equipment and storage medium

Abstract

The invention discloses a projection method and device, equipment and a storage medium, and the method comprises the steps: obtaining a shot image collected for a current projection picture projected in a current projection region, and intercepting a source image of the current projection picture; obtaining a preset color brightness conversion relation, and converting the source image into a color conversion image consistent with the shot image in size based on the preset color brightness conversion relation; determining whether a projection obstacle exists in the current projection area or not based on the color difference between the color conversion image and the shot image; and under the condition that the projection obstacle exists in the current projection area, determining a target projection area avoiding the position of the projection obstacle, and performing projection based on the target projection area.

Description

Technical Field

[0001] The present application relates to the field of projection technology, and in particular to a projection method, device, equipment and storage medium. Background Art

[0002] Projectors can be conveniently used in many occasions. For example, people usually directly project the projected image of the projector onto a wall for demonstration and playback, that is, the wall is used as a display screen for projection without installing a display screen dedicated to projection display.

[0003] However, in actual situations, there may be objects on the wall that affect the projection effect, such as non-white switches on the wall, or patterned wallpaper, etc. In the prior art, for this application scenario, it is generally necessary to project a functional image and then achieve projection alignment based on the functional image, resulting in a poor projection experience and low efficiency of projection alignment. Summary of the invention

[0004] The present application hopes to provide a projection method, device, equipment and storage medium.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] An embodiment of the present application provides a projection method, which includes obtaining a captured image of a current projection screen projected on a current projection area, and intercepting a source image of the current projection screen; obtaining a preset color-brightness conversion relationship, and based on the preset color-brightness conversion relationship, converting the source image into a color conversion image that is consistent with the size of the captured image; determining whether there is a projection obstacle in the current projection area based on a color difference between the color conversion image and the captured image; and if the projection obstacle exists in the current projection area, determining a target projection area that avoids the location of the projection obstacle, and performing projection based on the target projection area.

[0007] The present application provides a projection device, including:

[0008] An acquisition module, used to acquire a captured image of a current projection picture projected on a current projection area, and to intercept a source image of the current projection picture;

[0009] A conversion module, used to obtain a preset color-brightness conversion relationship, and based on the preset color-brightness conversion relationship, convert the source image into a color-converted image having the same size as the captured image;

[0010] a determination module, configured to determine whether there is a projection obstacle in the current projection area based on a color difference between the color conversion image and the captured image;

[0011] The projection module is used to determine a target projection area avoiding the location of the projection obstacle when the projection obstacle exists in the current projection area, and perform projection based on the target projection area.

[0012] An embodiment of the present application provides a projection device, the projection device comprising: a processor, a memory and a communication bus;

[0013] The communication bus is used to realize the communication connection between the processor and the memory;

[0014] The processor is used to execute the computer program stored in the memory to implement the above-mentioned projection method.

[0015] An embodiment of the present application provides a computer-readable storage medium, which stores one or more computer programs. The one or more computer programs can be executed by one or more processors to implement the above-mentioned projection method.

[0016] The embodiments of the present application provide a projection method, device, equipment and storage medium, which obtain a captured image of a current projection screen projected on a current projection area, and intercept a source image of the current projection screen; obtain a color-brightness conversion relationship between the source image and the captured image, and based on the color-brightness conversion relationship, convert the source image into a color-converted image of the same size as the captured image; determine whether there is a projection obstacle in the current projection area based on the color difference between the color-converted image and the captured image; if there is a projection obstacle in the current projection area, determine a target projection area that avoids the location of the projection obstacle, and perform projection based on the target projection area. The technical solution provided by the present application does not require an additional projection function image, and directly determines the color-converted image of the source image based on a preset color-brightness conversion relationship, and then based on the color difference between the color-converted image and the captured image, it can determine whether there is a projection obstacle in the current projection area, and if there is a projection obstacle area in the projection area, plan an optimal target projection area for projection, thereby improving the efficiency of projection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a projection method provided in an embodiment of the present application;

[0018] Figure 2 An exemplary process for obtaining a preset color brightness conversion relationship provided in an embodiment of the present application is shown in FIG. Figure 1 ;

[0019] Figure 3 An exemplary process for obtaining a preset color brightness conversion relationship provided in an embodiment of the present application is shown in FIG. Figure 2 ;

[0020] Figure 4 An exemplary process for determining a target mapping relationship provided in an embodiment of the present application is shown in FIG. Figure 1 ;

[0021] Figure 5 An exemplary process for determining a target mapping relationship provided in an embodiment of the present application is shown in FIG. Figure 2 ;

[0022] Figure 6 An exemplary process for determining a target mapping relationship provided in an embodiment of the present application is shown in FIG. Figure 3 ;

[0023] Figure 7 An exemplary process for determining a target mapping relationship provided in an embodiment of the present application is shown in FIG. Figure 4 ;

[0024] Figure 8 A schematic diagram of an exemplary process for determining a preset color brightness conversion relationship provided in an embodiment of the present application;

[0025] Fig. 9 A schematic diagram of an exemplary process of determining whether there is an obstacle in the projection area provided in an embodiment of the present application;

[0026] Fig.10 A schematic diagram of an exemplary process of determining a target projection area provided in an embodiment of the present application;

[0027] Fig.11 A schematic diagram of a flow chart of an exemplary projection method provided in an embodiment of the present application;

[0028] Fig.12 A schematic diagram of the structure of a projection device provided in an embodiment of the present application;

[0029] Fig.13 A schematic diagram of the structure of a projection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical solution in the present application will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present application. It is to be understood that the specific embodiments described herein are only used to explain the related application, rather than to limit the application. It should also be noted that, for ease of description, only the parts related to the related application are shown in the accompanying drawings.

[0031] The present application embodiment provides a projection method, which is implemented by a projection device, such as Figure 1 As shown, the steps S101 to S104 are included:

[0032] Step S101 : acquiring a captured image of a current projection screen projected on a current projection area, and capturing a source image of the current projection screen.

[0033] In an embodiment of the present application, the projection device is an electronic device with a projection function. The projection device projects a fixed projection area, that is, projects the current projection screen to the current projection area, and then triggers the obstacle avoidance function, suspends the content update of the current projection screen, and captures the captured image of the current projection screen projected on the current projection area, and captures the source image of the current projection screen. Among them, the means of triggering the obstacle avoidance function can be manually pressing the obstacle avoidance button set on the projection device, a fixed gesture or voice instruction set on the projection device screen, etc., of course, it can also be other triggering methods. The specific triggering method can be set according to the actual situation and application requirements, and this application is not limited to this.

[0034] Step S102: Obtain a preset color-brightness conversion relationship, and based on the preset color-brightness conversion relationship, convert the source image into a color-converted image having the same size as the captured image.

[0035] In an embodiment of the present application, after acquiring the captured image and the source image, the projection device can acquire a preset color-brightness conversion relationship M based on the source image and the captured image. After the projection device acquires the color-brightness conversion relationship, the source image can be converted into a color conversion image I having the same size as the captured image using the color-brightness conversion relationship M.

[0036] For example, the conversion method is shown in formula (1):

[0037] C(x, y) = C(X, Y) * M' (1);

[0038] Wherein, M is a preset color brightness conversion matrix, M' is the inverse of M, C(x, y) represents the RGB value of the pixel point (x, y) in the color conversion image I, and C(X, Y) represents the RGB value of the pixel point (X, Y) in the source image.

[0039] Step S103: determining whether there is a projection obstacle in the current projection area based on the color difference between the color conversion image and the captured image.

[0040] In an embodiment of the present application, the projection device determines whether there is a projection obstacle in the current projection area based on the color difference between the color conversion image and the captured image, in order to achieve projection when there is an obstacle in the current projection area.

[0041] Step S104: when there is a projection obstacle in the current projection area, determine a target projection area that avoids the location of the projection obstacle, and perform projection based on the target projection area.

[0042] In an embodiment of the present application, the projection device can obtain the current projection picture projected on the current projection area, and detect whether there is a projection obstacle in the current projection area based on the current projection picture, so as to determine a target projection area that can avoid the location of the projection obstacle when there is a projection obstacle in the current projection area, so as to perform projection based on the target projection area.

[0043] Exemplarily, the projection device performs projection based on the target projection area by converting the target projection area into optomechanical image coordinates, which is the target position for obstacle avoidance, and then projects the projection picture based on the optomechanical image coordinates.

[0044] Compared with the prior art, which requires projecting a functional image and then realizing projection alignment based on the functional image, in the present application, there is no need to additionally project a functional image. The color conversion image of the source image is determined directly based on a preset color-brightness conversion relationship, and then based on the color difference between the color conversion image and the captured image, it can be determined whether there is a projection obstacle in the current projection area. In the case where there is a projection obstacle area in the projection area, an optimal target projection area is planned for projection, thereby improving the efficiency of projection.

[0045] In some embodiments, the projecting device may preset the color brightness conversion relationship in executing step S102, including the following steps S201 to S202:

[0046] Step S201: Determine a target mapping relationship between a source image and a captured image.

[0047] In an embodiment of the present application, the projection device determines a target mapping relationship between the source image and the captured image based on each pixel point on the source image and the captured image.

[0048] Step S202: determining a color brightness conversion relationship based on a target mapping relationship.

[0049] In the embodiment of the present application, after acquiring the target mapping relationship, the projection device can determine the color brightness conversion relationship based on the target mapping relationship.

[0050] In some embodiments, the projection device may perform the above step S201 by including the following steps S301 to S302:

[0051] Step S301: Obtain the wall information of the projection wall where the current projection area is located.

[0052] In an embodiment of the present application, the projection device obtains the wall information of the projection wall where the current projection area is located. The wall information can be a solid color background, such as a standard white wall; or other patterned backgrounds, such as patterned wallpaper on the wall; or other wall information.

[0053] Step S302: Select a preset color-brightness conversion relationship that matches the wall information from a preset color-brightness conversion relationship database.

[0054] In an embodiment of the present application, a preset color-brightness conversion relationship database stores preset color-brightness conversion relationships corresponding to different pre-calibrated wall information. In this way, after the projection device obtains the wall information of the current projection wall, it can select the preset color-brightness conversion relationship that matches the current projection wall from the preset color-brightness conversion relationship, and then determine whether there is a projection obstacle in the current projection area based on the preset color-brightness conversion relationship.

[0055] In some embodiments, the projection device may perform the following steps S401 to S404 after performing the above step S201:

[0056] Step S401: using the Sobel operator, respectively determine the gradient value of each pixel in the source image and the captured image.

[0057] In an embodiment of the present application, the projection device can use the Sobel operator to calculate the gradient value of each pixel in the source image and the gradient value of each pixel in the captured image; the specific method of calculating the gradient value is consistent with the existing calculation method and will not be repeated here.

[0058] Step S402: Filter out pixel points corresponding to the local maximum gradient value from the source image using the first preset window to obtain a corresponding first pixel point set.

[0059] In the embodiment of the present application, the first preset window may be a w*w window, and of course may be a window of other sizes. The specific first preset window size may be set according to actual conditions, and the present application does not limit this.

[0060] In an embodiment of the present application, the projection device uses a first preset window to divide the source image into multiple image areas of the same size as the first preset window, and then selects pixel points corresponding to the maximum gradient value from each image area as elements in the first pixel point set.

[0061] Step S403: Filter out pixel points corresponding to the local maximum gradient value from the captured image using the second preset window to obtain a corresponding second pixel point set.

[0062] In an embodiment of the present application, the second preset window may be the same as the first preset window. Of course, the second preset window may also be a multiple of the first preset window, such as 2w*2w, or other sizes. The specific second preset window size may be set according to actual conditions, and the present application does not limit this.

[0063] In an embodiment of the present application, the projection device uses a second preset window to divide the captured image into multiple image areas of the same size as the second preset window, and then selects pixel points corresponding to the maximum gradient value from each image area as elements in the second pixel point set.

[0064] Step S404: updating the initial mapping relationship based on the first pixel point set and the second pixel point set to obtain a target mapping relationship; the initial mapping relationship is a pre-calibrated mapping relationship between the camera and the optical machine.

[0065] In an embodiment of the present application, the initial mapping relationship is a pre-calibrated mapping relationship between a camera and an optical machine. After obtaining the first pixel point set and the second pixel point set, the projection device can update the initial mapping relationship based on the pixels included in the first pixel point set and the second pixel point set, in order to obtain a target mapping relationship that is more suitable for the current application scenario.

[0066] In an embodiment of the present application, the projection device may perform the above step S404 by including the following steps S501 to S504:

[0067] Step S501: using the initial mapping relationship, determine that each pixel point included in the first pixel point set is mapped to a corresponding pixel point in the captured image, and obtain a third pixel point set corresponding to the first pixel point set.

[0068] In an embodiment of the present application, the projection device uses the initial mapping relationship to map each pixel in the first pixel to the captured image to obtain a third pixel set.

[0069] Step S502: for each pixel point included in the third pixel point set, use the third preset window to search for a corresponding matching pixel point from the second pixel point set to obtain a fourth pixel point set corresponding to the third pixel point set.

[0070] In an embodiment of the present application, the second pixel point set includes pixel points in the captured image. After the projection device obtains the third pixel point set, the pixel points included in the third pixel point set are the pixel points included in the first pixel point set mapped to the captured image. At this time, for each pixel point included in the third pixel point set, the third preset window can be used to search for corresponding matching pixel points from the second pixel point set to obtain a fourth pixel point set corresponding to the third pixel point set; the third preset window here can be set according to actual conditions and application requirements, and this application does not limit this. In the process of searching for pixel points, if multiple pixel points of different distances appear, the pixel point with the closest distance is selected as the matching pixel point; if multiple closest pixel points of the same distance appear, one is randomly selected as the matching pixel point; if only one pixel point appears in the third preset window, the pixel point that appears is directly used as the matching pixel point.

[0071] Step S503: Use the first pixel point set and the fourth pixel point set to update the initial mapping relationship to obtain a target mapping relationship.

[0072] In an embodiment of the present application, the projection device updates the initial mapping relationship using the first pixel point set and the fourth pixel point set to obtain a target mapping relationship.

[0073] In an embodiment of the present application, the projection device may perform the above step S503 by including the following steps S601 to S604:

[0074] Step S601: Use the first pixel point set and the fourth pixel point set to perform a first update on the initial mapping relationship to obtain a first mapping relationship.

[0075] In an embodiment of the present application, after acquiring the first pixel point set and the fourth pixel point set, the projection device updates the initial mapping relationship for the first time based on the first pixel point set and the fourth pixel point set to obtain a first mapping relationship.

[0076] Step S602: If the update number does not reach the preset update number, continue to use the first mapping relationship to determine that each pixel point included in the first pixel point set is mapped to a corresponding pixel point in the captured image, and obtain a fifth pixel point set corresponding to the first pixel point set.

[0077] In an embodiment of the present application, after updating the mapping relationship, the projection device will determine whether the number of updates has reached the preset number of updates. If the number of updates has not been reached, it will continue to use the first mapping relationship to determine the fifth pixel point set corresponding to the first pixel point set. Here, the step of determining the fifth pixel point set corresponding to the first pixel point set is different from the above step S501 only in the mapping relationship, and the implementation method is the same, which will not be repeated here.

[0078] Step S603: for each pixel point included in the fifth pixel point set, use the third preset window to search for a corresponding matching pixel point from the second pixel point set to obtain a sixth pixel point set corresponding to the fifth pixel point set.

[0079] In the embodiment of the present application, the step of determining the sixth pixel point set corresponding to the fifth pixel point set is implemented in the same manner as the above step S502, and will not be described in detail here.

[0080] Step S604: Use the first pixel point set and the sixth pixel point set to update the first mapping relationship for the second time until the updating number reaches the preset updating number and then stop updating, and determine the target mapping relationship based on the mapping relationship obtained in each updating within the preset updating number.

[0081] In an embodiment of the present application, the projection device uses the first pixel point set and the sixth pixel point set to perform a second update on the first mapping relationship. If the number of updates has not reached the preset number of updates, the third update is continued until the number of updates reaches the preset number of updates and the update is stopped, and the target mapping relationship is determined based on the mapping relationship obtained by each update in the preset number of updates. The preset number of updates can be 5, 10, 15 or any other value, and the specific preset number of updates can be set according to actual conditions and application requirements, which is not limited by the present application.

[0082] In an embodiment of the present application, after performing updates for a preset number of times, the projection device determines a target mapping relationship based on a mapping relationship obtained in each update for the preset number of times.

[0083] In some embodiments, the above step S604 determines the target mapping relationship based on the mapping relationship obtained in each update in the preset update times, which may include the following steps S701 to S702:

[0084] Step S701: for each update, using the mapping relationship obtained by the corresponding update, determine the image matching area where the central area of ​​the corresponding source image is mapped to the captured image, and determine the image difference value between the corresponding central area and the image matching area.

[0085] In an embodiment of the present application, after each update, the projection device uses the mapping relationship obtained from the corresponding update to calculate the central area of ​​the source image and the image matching area in the captured image, and then determines the image difference value between the corresponding central area and the image matching area.

[0086] Exemplarily, the mapping relationship obtained by the first update is the first mapping relationship, and the projection device can use the first mapping relationship to map the central area in the source image to the captured image to obtain the image matching area, and then determine the image difference value (ssim value) between the central area and the image matching area. Among them, the central area in the source image is the image area centered on the center point of the image, and the area size can be selected according to actual conditions and application requirements, which is not limited in this application.

[0087] In this way, if the preset number of updates is 10, the number of image difference values ​​obtained is also 10.

[0088] Step S702: determine the mapping relationship corresponding to the maximum image difference value as the target mapping relationship.

[0089] For example, if the preset update number is 10 times, then the corresponding mapping relationships include 10 and the image difference values ​​are also 10, that is, each update will obtain an updated mapping relationship and an image difference value. In this way, the mapping relationship corresponding to the maximum image difference value is determined as the target mapping relationship, and then the projection device continues to update the content of the current projection screen.

[0090] In some embodiments, the projection device may perform the above step S202 by including the following steps S801 to S803:

[0091] Step S801: using the target mapping relationship, determine a plurality of mapping pixel points in the captured image that correspond one-to-one to a plurality of pixel points included in a preset area in the source image.

[0092] In an embodiment of the present application, the preset area may be an image area including m*n pixels, and of course may be other area sizes. The specific size of the preset area may be set according to actual conditions and application requirements, and the present application does not limit this.

[0093] In an embodiment of the present application, the projection device searches for a corresponding mapping pixel point in the captured image for each pixel point among the multiple pixel points included in the preset area in the source image through a target mapping relationship, thereby obtaining multiple mapping pixel points that correspond one-to-one to the multiple pixel points included in the preset area.

[0094] For an exemplary method of determining a plurality of mapping pixels using the target mapping relationship, see formula (2):

[0095] (x, y, z) = (X, Y, 1)H (2);

[0096] Among them, H is the target mapping relationship, (x, y, z) is the coordinate of the pixel point in the captured image, and (X, Y, 1) is the coordinate of the pixel point in the source image.

[0097] In the embodiment of the present application, since there may be multiple identical or similar RGB values ​​in the source image, the projection device needs to filter the [RGB] data once, retain the non-repeated RGB values, and then use formula (2) to obtain the corresponding [rgb].

[0098] Step S802: Acquire a plurality of first color values ​​corresponding one-to-one to a plurality of pixel points, and a plurality of second color values ​​corresponding one-to-one to a plurality of mapping pixel points.

[0099] In the embodiment of the present application, the projection device acquires the first color value (RGB value) corresponding to each pixel point in the preset area, and obtains multiple first color values ​​corresponding to the multiple pixels one by one. For the specific acquisition method, see formula (3):

[0100] RGB = f(X, Y) (3);

[0101] Among them, RGB is the first color value of the pixel in the source image, and f is the value function of the color value.

[0102] In the embodiment of the present application, the projection device also obtains the second color value (RGB value) corresponding to each of the multiple mapping pixel points, and obtains multiple second color values ​​corresponding to the multiple mapping pixel points. For the specific acquisition method, see formula (4):

[0103] rgb = f(x, y) (4);

[0104] Wherein, rgb is the second color value of the mapped pixel in the captured image.

[0105] Step S803: determining a preset color brightness conversion relationship based on the plurality of first color values ​​and the plurality of second color values.

[0106] In an embodiment of the present application, the projection device determines a color-brightness conversion relationship based on a plurality of first color values ​​and a plurality of color values. For example, the determination method of the color-brightness conversion relationship is shown in formula (5):

[0107] M=([RBG]*[RGB] T )'([RGB] T [rgb]) (5);

[0108] Where [RGB] represents the set of all first color values ​​RGB, and similarly [rgb] represents the set of second color values ​​rgb. M represents the color brightness conversion matrix of RGB->rgb, ([RBG]*[RGB] T )' is the matrix ([RBG]*[RGB] T ) to find the inverse.

[0109] In some embodiments, the projection device may include the following steps S901 to S904 when performing the above step S204:

[0110] Step S901: determine corresponding pixel points in the color conversion image and the captured image as a pair of pixel points to obtain multiple pairs of pixel points.

[0111] In an embodiment of the present application, the projection device determines the corresponding pixel points in the color conversion image and the captured image as a pair of pixel points, so that multiple pairs of pixel points can be obtained.

[0112] Step S902: Determine the color value difference between each pair of pixel points in the plurality of pairs of pixel points, and obtain a plurality of color difference values ​​corresponding to the plurality of pairs of pixel points.

[0113] In the embodiment of the present application, for each pair of pixel points in the plurality of pairs of pixel points, the projection device determines the corresponding color difference value. For an exemplary determination method, see formula (6):

[0114] F=I r -I, or F = II r (6);

[0115] Among them, I r is the color value of the pixel in the captured image, I is the color value of the pixel in the color conversion image, and F is the color difference value.

[0116] Step S903: determining a color difference Euclidean distance value of each color difference value in the plurality of color difference values, and obtaining a plurality of color difference Euclidean distance values ​​corresponding to the plurality of color difference values ​​one by one.

[0117] In an embodiment of the present application, the projection device calculates the Euclidean distance for each color difference value among the obtained multiple color difference values ​​to obtain a corresponding color difference Euclidean distance value.

[0118] Step S904: When there are pixel pairs among the multiple pairs of pixel points whose color difference Euclidean distance values ​​are greater than or equal to a preset Euclidean distance threshold, it is determined that there are projection obstacles in the current projection area.

[0119] In an embodiment of the present application, if there are pixel pairs among multiple pairs of pixels whose color difference Euclidean distance value is greater than or equal to a preset Euclidean distance threshold, it indicates that there are projection obstacles in the current projection area. The preset Euclidean distance threshold can be set according to actual conditions and application scenarios, and this application does not limit this.

[0120] In some embodiments, the projecting device may determine the target projection area that avoids the location of the projection obstacle in executing the above step S101, which may include the following steps S1001 to S1004:

[0121] Step S1001: Determine, among multiple pairs of pixel points, pixel point pairs whose color difference Euclidean distance values ​​are greater than or equal to a preset Euclidean distance threshold as target pixel point pairs.

[0122] In an embodiment of the present application, the projection device determines, among multiple pairs of pixel points, pixel point pairs whose color difference Euclidean distance values ​​are greater than or equal to a preset Euclidean distance threshold as target pixel point pairs.

[0123] Step S1002: Determine, among the pixels included in the target pixel pair, the pixels that overlap with the pixels included in the captured image as target projection obstacle pixels.

[0124] In an embodiment of the present application, each pair of pixels in the target pixel pair will include a pixel in the source image and a pixel in the captured image. Therefore, the pixel points included in the target pixel pair that coincide with the pixel points included in the captured image are determined as target projection obstacle pixels. In fact, it is to determine which pixel in the captured image corresponds to a color difference Euclidean distance value greater than or equal to a preset Euclidean distance threshold, which indicates that there is a projection obstacle at the pixel point, that is, there is a projection obstacle at each pixel included in the target projection obstacle pixel point.

[0125] Step S1003: convert the target projection obstacle pixel point to the projection wall coordinate system to obtain the corresponding target projection obstacle position.

[0126] In the embodiment of the present application, the projection device converts the target projection obstacle pixel points into projection wall coordinates, so as to know where the projection obstacle exists on the projection wall, that is, the target projection obstacle position.

[0127] Step S1004: determine the target projection area from the projection wall according to the position of the target projection obstacle.

[0128] In an embodiment of the present application, the projection device plans an optimal projection area, namely, the target projection area, on the projection wall according to the position of the target projection obstacle, and then continues to project based on the target projection area.

[0129] Fig.11 The following is a flow chart of an exemplary projection method provided in an embodiment of the present application. Fig.11 As shown, the following steps S1101 to S1106 are included:

[0130] Step S1101: Acquire a captured image of a current projection screen projected on a current projection area, and capture a source image of the current projection screen.

[0131] Here, the projection device will focus on the current projection area. After the obstacle avoidance function is triggered, the update of the projection screen content will be suspended, and the camera will be used to capture the current projection screen (capture image). At the same time, the source image of the current projection needs to be captured (source image).

[0132] Step S1102: Determine a target mapping relationship between the source image and the captured image.

[0133] Here, the projecting device may determine the target mapping relationship between the source image and the captured image in the following manner:

[0134] 1. The projection device uses the Sobel operator to calculate the gradient value of each pixel in the camera image (captured image), and uses a w*w window to filter the local maximum gradient value; similarly, the projection device uses the Sobel operator to calculate the gradient of each pixel in the source image, and uses a 2w*2w window to filter the local maximum gradient value. Through the above calculation, some candidate local trapezoidal maximum pixel point sets C1 (first pixel point set) and C2 (second pixel point set) can be obtained.

[0135] 2. Based on the extracted and calibrated mapping relationship H1 (initial mapping relationship) between the camera and the optical machine, the candidate local pixel point C1 in the original image (source image) is mapped to the image taken by the camera (captured image) through H1 to obtain C3 (the third pixel point set). The points in C2 are searched near C3 with a window size of w1*w1. If there are multiple points, the nearest point is taken. H1 is updated based on the matched points (the fourth pixel point set). After updating H1, the image matching area from the central area of ​​the original image to the camera is calculated, and then the ssim value (image difference value) between the two is calculated.

[0136] 3. The updating step of the mapping relationship in loop 2 stops when the number of loops reaches the preset number of updates. When the maximum ssim value is calculated, the final H1 value (target mapping relationship) is obtained, and the projection image continues.

[0137] Step S1103: establishing a color brightness conversion relationship between the source image and the captured image based on the target mapping relationship.

[0138] Here, the projection device will find the corresponding pixels in the camera image through the final H1 (target mapping relationship) for the m*n pixels in the original image, that is, the matching of RGB->rgb. RGB is the value of the pixel in the original image, and rgb is the value of the pixel in the image captured by the camera. Since there may be multiple identical or similar RGB values ​​in the original image, it is necessary to filter the [RGB] data once, retain the non-repeated RGB values, and use the above formula (2) to obtain the corresponding [rgb], and calculate the original image pixel coordinates corresponding to the camera pixel coordinates. Then, obtain the pixel color value corresponding to each pixel, and then determine the color brightness conversion relationship based on the obtained color value using the above formula (5).

[0139] Step S1104: convert the source image into a color-converted image having the same size as the captured image through a color-brightness conversion relationship.

[0140] Here, the projection device uses the color-brightness conversion relationship to convert the source image into a color-converted image of the same size as the captured image. For the specific conversion method, see the above formula (1).

[0141] Step S1105: Determine the color difference between the color converted image and the captured image, and determine whether there is a projection obstacle in the current projection area.

[0142] Here, the projection device uses the above formula (6) to determine the color difference between the color conversion image and the captured image. Here, the color difference value between the corresponding pixel points in the color conversion image and the captured image is calculated, and O(F) is calculated. O(F) represents the color difference Euclidean distance (color difference Euclidean distance value). When the difference (color difference Euclidean distance value) exceeds the threshold (preset Euclidean distance threshold), it is considered that the pixel has a candidate obstacle, otherwise it is a wall area, which is suitable for the projection area.

[0143] Step S1106: When there is a projection obstacle in the current projection area, determine a target projection area that avoids the location of the projection obstacle, and perform projection based on the target projection area.

[0144] Here, after obtaining the projection area, the projection device can use the conventional process to transform the projection area corresponding to the camera coordinates to the wall coordinates, calculate the four points of the optimal projection rectangle (target projection area), and transfer the four points of the projection rectangle on the wall to the optical-mechanical image coordinates, which is the target position for obstacle avoidance.

[0145] The embodiment of the present application provides a projection method, which obtains a captured image of a current projection screen projected on a current projection area, and intercepts a source image of the current projection screen; obtains a color-brightness conversion relationship between the source image and the captured image, and based on the color-brightness conversion relationship, converts the source image into a color-converted image of the same size as the captured image; based on the color difference between the color-converted image and the captured image, determines whether there is a projection obstacle in the current projection area; in the case where there is a projection obstacle in the current projection area, determines a target projection area that avoids the location of the projection obstacle, and projects based on the target projection area. The projection method provided by the present application does not require an additional projection function image, and directly determines the color-converted image of the source image based on a preset color-brightness conversion relationship, and then based on the color difference between the color-converted image and the captured image, determines whether there is a projection obstacle in the current projection area, and in the case where there is a projection obstacle area in the projection area, plans an optimal target projection area for projection, thereby improving the efficiency of projection.

[0146] The present application embodiment provides a projection device, such as Fig.12 As shown, including:

[0147] The acquisition module 1201 is used to acquire a captured image of a current projection screen projected on a current projection area, and to intercept a source image of the current projection screen;

[0148] The conversion module 1202 is used to obtain a preset color-brightness conversion relationship, and based on the preset color-brightness conversion relationship, convert the source image into a color-converted image having the same size as the captured image;

[0149] A determination module 1203, configured to determine whether there is a projection obstacle in the current projection area based on a color difference between the color conversion image and the captured image;

[0150] The projection module 1204 is used to determine a target projection area avoiding the location of the projection obstacle when there is a projection obstacle in the current projection area, and perform projection based on the target projection area.

[0151] In one embodiment of the present application, the conversion module 1202 is further used to determine a target mapping relationship between the source image and the captured image; and determine a color brightness conversion relationship based on the target mapping relationship.

[0152] In one embodiment of the present application, the conversion module 1202 is also used to use the Sobel operator to respectively determine the gradient value of each pixel in the source image and the captured image; use the first preset window to filter out the pixel points corresponding to the local maximum gradient value from the source image to obtain the corresponding first pixel point set; use the second preset window to filter out the pixel points corresponding to the local maximum gradient value from the captured image to obtain the corresponding second pixel point set; based on the first pixel point set and the second pixel point set, the initial mapping relationship is updated to obtain the target mapping relationship; the initial mapping relationship is a pre-calibrated mapping relationship between the camera and the optical machine.

[0153] In one embodiment of the present application, the conversion module 1202 is also used to use a third preset window to search for corresponding matching pixel points from the second pixel point set for each pixel point included in the third pixel point set, so as to obtain a fourth pixel point set corresponding to the third pixel point set; and use the first pixel point set and the fourth pixel point set to update the initial mapping relationship to obtain a target mapping relationship.

[0154] In one embodiment of the present application, the conversion module 1202 is also used to use the first pixel point set and the fourth pixel point set to perform a first update on the initial mapping relationship to obtain a first mapping relationship; if the number of updates does not reach the preset number of updates, continue to use the first mapping relationship to determine that each pixel point included in the first pixel point set is mapped to the corresponding pixel point in the captured image, and obtain a fifth pixel point set corresponding to the first pixel point set; for each pixel point included in the fifth pixel point set, use the third preset window to find the corresponding matching pixel point from the second pixel point set to obtain a sixth pixel point set corresponding to the fifth pixel point set; use the first pixel point set and the sixth pixel point set to perform a second update on the first mapping relationship, until the number of updates reaches the preset number of updates and then stops updating, and determines the target mapping relationship based on the mapping relationship obtained in each update within the preset number of updates.

[0155] In one embodiment of the present application, the conversion module 1202 is also used to determine, for each update, the mapping relationship obtained by the corresponding update, to map the central area of ​​the corresponding source image to the image matching area of ​​the captured image, and determine the image difference value between the corresponding central area and the image matching area; and determine the mapping relationship corresponding to the maximum image difference value as the target mapping relationship.

[0156] In one embodiment of the present application, the conversion module 1202 is also used to determine, by using the target mapping relationship, a plurality of mapping pixel points in the captured image that correspond one-to-one to a plurality of pixel points included in a preset area in the source image; obtain a plurality of first color values ​​that correspond one-to-one to the plurality of pixel points, and a plurality of second color values ​​that correspond one-to-one to the plurality of mapping pixel points; and determine a preset color brightness conversion relationship based on the plurality of first color values ​​and the plurality of second color values.

[0157] In one embodiment of the present application, the determination module 1203 is also used to determine the corresponding pixel points in the color conversion image and the captured image as a pair of pixel points to obtain multiple pairs of pixel points; respectively determine the difference in color values ​​between each pair of pixel points in the multiple pairs of pixel points to obtain multiple color difference values ​​corresponding one-to-one to the multiple pairs of pixel points; determine the color difference Euclidean distance value of each color difference value in the multiple color difference values ​​to obtain multiple color difference Euclidean distance values ​​corresponding one-to-one to the multiple color difference values; when there are pixel pairs in the multiple pairs of pixel points whose color difference Euclidean distance values ​​are greater than or equal to a preset Euclidean distance threshold, it is determined that there is a projection obstacle in the current projection area.

[0158] In one embodiment of the present application, the projection module 1204 is also used to determine, among multiple pairs of pixels, pixel pairs whose color difference Euclidean distance values ​​are greater than or equal to a preset Euclidean distance threshold as target pixel pairs; determine, among the pixels included in the target pixel pairs, the pixels that coincide with the pixels included in the captured image as target projection obstacle pixels; convert the target projection obstacle pixels to the projection wall coordinate system to obtain the corresponding target projection obstacle position; and determine the target projection area from the projection wall according to the target projection obstacle position.

[0159] In one embodiment of the present application, the conversion module 1202 is also used to obtain the wall information of the projection wall where the current projection area is located; and select a preset color-brightness conversion relationship that matches the wall information from a preset color-brightness conversion relationship database.

[0160] The present application embodiment provides a projection device, such as Fig.13 As shown, the projection device includes: a processor 1301, a memory 1302 and a communication bus 1303;

[0161] A communication bus 1303, used to implement communication connection between the processor 1301 and the memory 1302;

[0162] The processor 1301 is used to execute the computer program stored in the memory 1302 to implement the above-mentioned projection method.

[0163] The embodiment of the present application provides a projection device, which obtains a captured image of a current projection screen projected on a current projection area, and intercepts a source image of the current projection screen; obtains a color-brightness conversion relationship between the source image and the captured image, and based on the color-brightness conversion relationship, converts the source image into a color-converted image of the same size as the captured image; determines whether there is a projection obstacle in the current projection area based on the color difference between the color-converted image and the captured image; in the case where there is a projection obstacle in the current projection area, determines a target projection area that avoids the location of the projection obstacle, and projects based on the target projection area. The projection device provided by the present application does not require an additional projection function image, and directly determines the color-converted image of the source image based on a preset color-brightness conversion relationship, and then based on the color difference between the color-converted image and the captured image, it can determine whether there is a projection obstacle in the current projection area, and in the case where there is a projection obstacle area in the projection area, plans an optimal target projection area for projection, thereby improving the efficiency of projection.

[0164] The embodiment of the present application provides a computer-readable storage medium, which stores one or more computer programs, and the one or more computer programs can be executed by one or more processors to implement the above-mentioned projection method. The computer-readable storage medium can be a volatile memory (volatile memory), such as a random access memory (Random-Access Memory, RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (Read-Only Memory, ROM), a flash memory (flash memory), a hard disk (Hard Disk Drive, HDD) or a solid-state drive (Solid-State Drive, SSD); or it can be a respective device including one or any combination of the above-mentioned memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc.

[0165] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0166] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0167] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0169] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A projection method, characterized in that: The method comprises: Acquire a captured image of a current projection picture projected on a current projection area, and intercept a source image of the current projection picture; Acquire a preset color-brightness conversion relationship, and based on the preset color-brightness conversion relationship, convert the source image into a color-converted image having the same size as the captured image; determining whether there is a projection obstacle in the current projection area based on a color difference between the color conversion image and the captured image; In the case that the projection obstacle exists in the current projection area, a target projection area avoiding the location of the projection obstacle is determined, and projection is performed based on the target projection area.

2. The method according to claim 1, characterized in that The obtaining of a preset color brightness conversion relationship includes: Determining a target mapping relationship between the source image and the captured image; The preset color brightness conversion relationship is determined based on the target mapping relationship.

3. The method according to claim 2, characterized in that The determining of a target mapping relationship between the source image and the captured image comprises: Using a Sobel operator, respectively determining the gradient value of each pixel in the source image and the captured image; Filtering pixel points corresponding to the local maximum gradient value from the source image using a first preset window to obtain a corresponding first pixel point set; Filtering out pixel points corresponding to the local maximum gradient value from the captured image using a second preset window to obtain a corresponding second pixel point set; The initial mapping relationship is updated based on the first pixel point set and the second pixel point set to obtain the target mapping relationship; the initial mapping relationship is a pre-calibrated mapping relationship between a camera and an optical machine.

4. The method according to claim 3, characterized in that The updating of the initial mapping relationship based on the first pixel point set and the second pixel point set to obtain the target mapping relationship includes: Using the initial mapping relationship, determine that each pixel point included in the first pixel point set is mapped to a corresponding pixel point in the captured image, and obtain a third pixel point set corresponding to the first pixel point set; For each pixel point included in the third pixel point set, using a third preset window to search for a corresponding matching pixel point from the second pixel point set, to obtain a fourth pixel point set corresponding to the third pixel point set; The initial mapping relationship is updated using the first pixel point set and the fourth pixel point set to obtain the target mapping relationship.

5. The method according to claim 4, characterized in that The updating the initial mapping relationship by using the first pixel point set and the fourth pixel point set to obtain the target mapping relationship includes: Using the first pixel point set and the fourth pixel point set to perform a first update on the initial mapping relationship to obtain a first mapping relationship; If the number of updates does not reach the preset number of updates, continue to use the first mapping relationship to determine that each pixel point included in the first pixel point set is mapped to a corresponding pixel point in the captured image, so as to obtain a fifth pixel point set corresponding to the first pixel point set; For each pixel point included in the fifth pixel point set, using the third preset window to search for a corresponding matching pixel point from the second pixel point set, to obtain a sixth pixel point set corresponding to the fifth pixel point set; The first mapping relationship is updated for a second time using the first pixel point set and the sixth pixel point set, and the updating is stopped after the update number reaches the preset update number, and the target mapping relationship is determined based on the mapping relationship obtained by each update within the preset update number.

6. The method according to claim 5, characterized in that The determining the target mapping relationship based on the mapping relationship obtained by each update in the preset number of updates includes: For each update, using the mapping relationship obtained by the corresponding update, determining the image matching area corresponding to the central area of ​​the source image mapped to the captured image, and determining the image difference value between the central area and the image matching area; The mapping relationship corresponding to the maximum image difference value is determined as the target mapping relationship.

7. The method according to claim 2, characterized in that The determining the preset color brightness conversion relationship based on the target mapping relationship includes: Determine, by using the target mapping relationship, a plurality of mapping pixel points in the captured image that correspond one-to-one to a plurality of pixel points included in a preset area in the source image; Acquire a plurality of first color values ​​corresponding one-to-one to the plurality of pixel points, and a plurality of second color values ​​corresponding one-to-one to the plurality of mapping pixel points; The preset color-brightness conversion relationship is determined based on the plurality of first color values ​​and the plurality of second color values.

8. The method according to claim 1, characterized in that: The determining whether the projection obstacle exists in the current projection area based on the color difference between the color conversion image and the captured image includes: Determine the color conversion image and corresponding pixel points in the captured image as a pair of pixel points to obtain a plurality of pairs of pixel points; Determine the difference in color value between each pair of pixel points in the plurality of pairs of pixel points respectively, and obtain a plurality of color difference values ​​corresponding to the plurality of pairs of pixel points one by one; Determine a color difference Euclidean distance value of each color difference value in the plurality of color difference values, and obtain a plurality of color difference Euclidean distance values ​​corresponding one-to-one to the plurality of color difference values; In the case that there are pixel pairs among the multiple pairs of pixel points whose color difference Euclidean distance values ​​are greater than or equal to a preset Euclidean distance threshold, it is determined that the projection obstacle exists in the current projection area.

9. The method according to claim 8, characterized in that The determining of the target projection area avoiding the location of the projection obstacle includes: Determine, among the multiple pairs of pixel points, pixel point pairs whose color difference Euclidean distance values ​​are greater than or equal to the preset Euclidean distance threshold as target pixel point pairs; Determine, among the pixels included in the target pixel pair, the pixel points that coincide with the pixel points included in the captured image as target projection obstacle pixel points; Convert the target projection obstacle pixel point to the projection wall coordinate system to obtain the corresponding target projection obstacle position; According to the position of the target projection obstacle, the target projection area is determined from the projection wall.

10. The method according to claim 1, characterized in that The obtaining of a preset color brightness conversion relationship includes: Obtaining the wall information of the projection wall where the current projection area is located; The preset color-brightness conversion relationship matching the wall information is selected from a preset color-brightness conversion relationship database.

11. A projection device, characterized in that: include: An acquisition module, used to acquire a captured image of a current projection picture projected on a current projection area, and to intercept a source image of the current projection picture; A conversion module, used for obtaining a preset color-brightness conversion relationship, and based on the preset color-brightness conversion relationship, converting the source image into a color-converted image having the same size as the captured image; a determination module, configured to determine whether there is a projection obstacle in the current projection area based on a color difference between the color conversion image and the captured image; The projection module is used to determine a target projection area avoiding the location of the projection obstacle when the projection obstacle exists in the current projection area, and perform projection based on the target projection area.

12. A projection device, characterized in that: include: processor, memory, and communication bus; The communication bus is used to realize the communication connection between the processor and the memory; The processor is used to execute the computer program stored in the memory to implement the projection method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more computer programs, and the one or more computer programs can be executed by one or more processors to implement the projection method according to any one of claims 1 to 10.

Images