Projection prompting method and device, electronic equipment and storage medium

Abstract

The application discloses a projection prompting method and device, electronic equipment and storage medium, and relates to the technical field of projection. The method comprises the following steps: projecting a white field image to a target object through a plurality of first projectors, and capturing an image through a camera to obtain a first image; projecting a black field image to the target object through the plurality of first projectors, and capturing an image through the camera to obtain a second image; determining a projection display area based on a difference image between the first image and the second image; determining a plurality of shadow areas of the target object in the projection display area according to a gray value of each pixel point of the projection display area in the first image and at least one gray threshold; and performing color filling on the plurality of shadow areas in the first image according to a preset filling strategy to obtain and display a third image. In this way, the user can quickly reduce or eliminate the shadow of the projected object by adjusting the target object and / or the first projector, and a better projection display effect can be presented.

Description

Technical Field

[0001] This application relates to the field of projection technology, and in particular to a projection prompting method, device, electronic device, and storage medium. Background Technology

[0002] In multi-projector systems, in order to solve the problem of uneven brightness in overlapping areas (i.e., the so-called "shadow effect"), a brightness compensation scheme is usually adopted to achieve a smooth transition between projected images and eliminate projected shadows, thereby avoiding visual discomfort for the audience and enhancing the continuity and consistency of the overall image.

[0003] However, in related technologies, users or technicians often have to visually judge and adjust the object to the appropriate position, which is cumbersome and time-consuming, affecting the user experience. Summary of the Invention

[0004] This application proposes a projection prompting method, device, electronic device, and storage medium to help users quickly eliminate the shadow of the projected object and present a better projection effect.

[0005] In a first aspect, embodiments of this application provide a projection prompting method applied to a projection system, the projection system including a camera and multiple first projectors, the camera's shooting range covering the total projection range of the multiple first projectors, the method including: projecting a white field image onto a target object through the multiple first projectors and capturing an image through the camera to obtain a first image; projecting a black field image onto the target object through the multiple first projectors and capturing an image through the camera to obtain a second image; acquiring a difference image between the first image and the second image, and based on the difference image, determining a projection display area formed by the projection of the multiple first projectors in the first image; determining a target shadow area of ​​the target object in the projection display area according to the grayscale value of each pixel in the projection display area of ​​the first image and at least one grayscale threshold, the target shadow area being a place that cannot be projected by any of the first projectors; marking the target shadow area in the first image according to a preset marking strategy to obtain a third image; displaying the third image to prompt the user to adjust the position of the target object and / or the projector position of at least one first projector to reduce or eliminate the target shadow area.

[0006] Secondly, embodiments of this application provide a projection prompting device applied to a projection system. The projection system includes a camera and multiple first projectors, the camera's shooting range covering the total projection range of the multiple first projectors. The device includes: an image acquisition module, used to project a white field image onto a target object through the multiple first projectors and capture an image using the camera to obtain a first image; and to project a black field image onto the target object through the multiple first projectors and capture an image using the camera to obtain a second image; and a display area determination module, used to acquire a difference image between the first image and the second image, and based on the difference image, determine the display area in the first image. The image is composed of a projection display area formed by projections from the plurality of first projectors; a shadow area determination module, used to determine the target shadow area of ​​the target object in the projection display area based on the grayscale value of each pixel in the projection display area in the first image and at least one grayscale threshold, wherein the target shadow area is a place that cannot be projected by any of the first projectors; a marking module, used to mark the target shadow area in the first image according to a preset marking strategy to obtain a third image; and a display module, used to display the third image to prompt the user to adjust the position of the target object and / or the projector position of at least one first projector to reduce or eliminate the target shadow area.

[0007] Thirdly, embodiments of this application provide an electronic device, including: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to perform the methods described above.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing program code that can be invoked by a processor to execute the methods described above.

[0009] The solution provided in this application involves projecting white field images onto a target object using multiple first projectors and capturing the images with a camera to obtain a first image; and projecting black field images onto the target object using multiple first projectors and capturing the images with a camera to obtain a second image. A difference image is obtained between the first and second images, and based on this difference image, the projection display area formed by the projections from the multiple first projectors in the first image is determined. Based on the grayscale value of each pixel in the projection display area of ​​the first image and at least one grayscale threshold, a target shadow area of ​​the target object in the projection display area is determined. The target shadow area is the area that cannot be projected by any of the first projectors. The target shadow area in the first image is marked according to a preset marking strategy to obtain a third image. The third image is displayed to prompt the user to adjust the position of the target object and / or the projector position of at least one first projector to reduce or eliminate the target shadow area. In this way, the user can quickly identify the shadow area that cannot be projected by any of the first projectors by viewing the displayed third image, thereby assisting the user in quickly reducing or eliminating the shadow area of ​​the projected object by adjusting the target object and / or the first projectors, resulting in a better projection display effect. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A schematic flowchart of a projection prompting method provided in an embodiment of this application is shown.

[0012] Figure 2 A system architecture diagram of a projection system provided in an embodiment of this application is shown.

[0013] Figure 3 It shows Figure 1 A flowchart illustrating a sub-step of step S120 in one embodiment.

[0014] Figure 4 A schematic flowchart of a projection prompting method provided in another embodiment of this application is shown.

[0015] Figure 5 This illustration shows a schematic diagram of a third image provided in an embodiment of this application.

[0016] Figure 6 A schematic diagram of a first image provided in an embodiment of this application is shown.

[0017] Figure 7 This is a block diagram of a projection prompting device according to an embodiment of this application.

[0018] Figure 8 This is a block diagram of an electronic device for performing a projection prompting method according to an embodiment of this application.

[0019] Figure 9 This is a storage unit in this application embodiment for storing or carrying program code that implements the projection prompting method according to this application embodiment. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0021] It should be noted that some processes described in the specification, claims, and accompanying drawings of this application include multiple operations that appear in a specific order. These operations may not be performed in the order they appear herein, or they may be performed in parallel. Operation numbers such as S110, S120, etc., are merely used to distinguish different operations and do not represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be performed sequentially or in parallel. Also, the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, such that a process, method, system, product, or server that includes a series of steps or sub-modules is not necessarily limited to those steps or sub-modules that are explicitly listed, but may include other steps or sub-modules that are not explicitly listed or that are inherent to such process, method, product, or device.

[0022] The inventors have proposed a projection prompting method, device, electronic device, and storage medium. The projection prompting method provided in the embodiments of this application will be described in detail below.

[0023] Please refer to Figure 1 , Figure 1This is a flowchart illustrating a projection prompting method according to an embodiment of this application, applied to a projection system. The projection system includes a camera and multiple first projectors, and the camera's shooting range covers the total projection range of the multiple first projectors. The following will be combined with... Figure 1 The projection prompting method provided in the embodiments of this application will be described in detail. The projection prompting method may include the following steps:

[0024] Step S110: Project a white field image onto the target object using the plurality of first projectors and capture the image using the camera to obtain a first image; and project a black field image onto the target object using the plurality of first projectors and capture the image using the camera to obtain a second image.

[0025] In this embodiment, as Figure 2 As shown, the projection system 10 may include multiple first projectors (i.e., projector 111, projector 112, and projector 113) and a camera 12. The camera 121's field of view covers the total projection range of the multiple first projectors. The camera 121 can be a monochrome camera or a color camera; this embodiment does not limit this. Of course, the projection system 10 may also include other... Figure 2 This embodiment does not limit the number of projectors beyond the three projectors shown. The target object can be, for example, […]. Figure 2 The bear model shown can also be other objects in a real scene, and this embodiment does not limit this. Before using multiple first projectors to perform 3D projection rendering of the target object, in order to avoid the shadow of the target object affecting the final projection rendering effect due to the projection, the position of the shadow generated by the target object can be determined in advance by projecting white field images and black field images, thereby prompting the user to adjust the camera and / or first projectors to eliminate the shadow.

[0026] Specifically, firstly, multiple first projectors are controlled to project white field images onto the target object, and then a camera is controlled to capture images of the area behind the target object, resulting in a first image; the white field image can be understood as a pure white image. Further, multiple first projectors are controlled to project black field images onto the target object, and then a camera is controlled to capture images of the area behind the target object, resulting in a second image; the black field image can be understood as a pure black image.

[0027] Step S120: Obtain the difference image between the first image and the second image, and based on the difference image, determine the projection display area formed by the projection of the plurality of first projectors in the first image.

[0028] Optionally, if the camera is a color camera, the first image is first converted to a grayscale image to obtain a first grayscale image, and the second image is first converted to a grayscale image to obtain a second grayscale image; then, a difference image between the first grayscale image and the second grayscale image is obtained. The difference image can be used to reflect pixel regions in the first and second images that show significant differences.

[0029] Optionally, if the camera is a monochrome camera, the first and second images obtained are both grayscale images, so the difference image between the first and second images can be directly obtained. It should be noted that the executing entity of this application is an electronic device with data processing capabilities within the projection system 10 (not included in...). Figure 2 As shown in the figure, the electronic device establishes a communication connection with the camera and each first projector, enabling the transmission of data and control signals.

[0030] Furthermore, in some implementations, please refer to Figure 3 The specific steps for determining the projection display area in step S120 may include the content from steps S121 to S123:

[0031] Step S121: Perform binarization processing on the difference image to obtain the binarized image corresponding to the difference image.

[0032] Binarization can be understood as setting a grayscale threshold, mapping the grayscale values ​​of pixels greater than or equal to the threshold to 255 (i.e., treating these pixels as white), and mapping the grayscale values ​​of pixels less than the threshold to 0 (i.e., treating these pixels as black). In other words, binarization converts the difference image from a grayscale image into a black-and-white image; the resulting binarized image is a grayscale image. This binarized image can also be understood as a mask image that excludes ambient light.

[0033] Step S122: Obtain the location of the white region in the binarized image.

[0034] Step S123: Determine the image region corresponding to the region position of the white area from the first image, as the projection display area formed by the projection of the plurality of first projectors.

[0035] In a binary image, the regions mapped as white are the pixel areas where there is a significant difference between the first and second images. Therefore, the location of the white region formed by the white pixels can be obtained based on the pixel positions of all white pixels in the binary image. This white region can be understood as the projected display area formed by multiple first projectors as captured by the camera. Therefore, the image region corresponding to the location of the white region can be determined from the first image as the projected display area formed by multiple first projectors.

[0036] Step S130: Based on the grayscale value of each pixel in the projection display area of ​​the first image and at least one grayscale threshold, determine the target shadow area of ​​the target object in the projection display area, wherein the target shadow area is a place that cannot be projected by any of the first projectors.

[0037] Understandably, since different projectors are placed in different positions, the positions of the shadows generated behind the target object when different projectors project white-field images onto the target object will also be different. Obviously, the grayscale value of the shadow area formed by a single shadow is different from that of the shadow area formed by multiple overlapping shadows. For example, the grayscale value of the shadow area formed by two overlapping shadows is darker than that of the shadow area formed by a single shadow; that is, the grayscale value of the shadow area formed by two overlapping shadows is greater than that of the shadow area formed by a single shadow. Similarly, the grayscale value of the shadow area formed by three overlapping shadows is greater than that of the shadow area formed by two overlapping shadows, and so on. This embodiment will not elaborate further. The number of shadows is generally the same as the number of first projectors; that is, each first projector generates a shadow behind the target object when projecting a white-field image onto it.

[0038] The target shadow region is essentially an area that cannot be projected onto by any of the first projectors; it can also be called a hard shadow region. In other words, no matter how the parameters of the multiple first projectors are adjusted, the target shadow region cannot be eliminated. Based on this, at least one grayscale threshold can be preset, thereby dividing the target shadow region from the projection display area based on the grayscale value of each pixel and at least one grayscale threshold.

[0039] Step S140: Mark the target shadow region in the first image according to the preset marking strategy to obtain the third image.

[0040] Optionally, the preset marking strategy can be a color filling strategy, that is, filling the target shadow area in the first image with a preset color to obtain the third image; wherein, the preset color is a pre-set color, such as red, and this application is not limited thereto. Optionally, the preset marking strategy can also be a character marking strategy, that is, marking at least one preset character in the target shadow area in the first image. Optionally, the preset marking strategy can also be a line filling strategy, that is, filling the target shadow area in the first image with multiple parallel lines. In other words, any strategy that can highlight the target shadow area in the first image can be used as the preset marking strategy of this application, and will not be described in detail here.

[0041] Step S150: Display the third image to prompt the user to adjust the position of the target object and / or the projector position of at least one first projector to reduce or eliminate the target shadow area.

[0042] Furthermore, a third image is displayed, allowing the user to quickly determine the location of the target shadow area behind the target object by viewing the third image. Furthermore, the position of the target object can be adjusted based on the location of the target shadow area in the third image, and / or the projector position of at least one of multiple projectors can be adjusted to reduce or eliminate the target shadow area.

[0043] In this embodiment, by capturing images of the projection area with a camera and determining the shadow areas that none of the first projectors can project onto based on the brightness of the shadow areas, the target shadow areas are identified. These target shadow areas are then marked in the first image to obtain a third image. This allows the user to quickly identify the shadow areas that none of the first projectors can project onto by viewing the displayed third image. This enables the user to quickly reduce or eliminate the shadow areas of the projected object by adjusting the target object and / or the first projectors, resulting in a better projection display effect.

[0044] Please refer to Figure 4 , Figure 4 This is a flowchart illustrating a projection prompting method according to another embodiment of this application, applied to a projection system. The projection system includes a camera and multiple first projectors, and the camera's shooting range covers the total projection range of the multiple first projectors. The following will be combined with... Figure 4 The projection prompting method provided in the embodiments of this application will be described in detail. The projection prompting method may include the following steps:

[0045] Step S201: Project a white field image onto the target object using the plurality of first projectors and capture the image using the camera to obtain a first image; and project a black field image onto the target object using the plurality of first projectors and capture the image using the camera to obtain a second image.

[0046] Step S202: Obtain the difference image between the first image and the second image, and based on the difference image, determine the projection display area formed by the projection of the plurality of first projectors in the first image.

[0047] In this embodiment, the specific implementation of steps S210 to S220 can be found in the content of the foregoing embodiments, and will not be repeated here.

[0048] Step S203: Determine the number of grayscale thresholds based on the number of projectors in the plurality of first projectors.

[0049] Understandably, the number of shadow regions is the same as the number of projectors. For example, if there are 3 projectors, there are only 3 types of shadow regions: the shadow region formed by the shadow of a single projector, the shadow region formed by the superposition of the shadows of two projectors, and the shadow region formed by the superposition of the shadows of three projectors. Clearly, the brightness of these three types of shadow regions differs. This means that different types of shadow regions can be distinguished by a single grayscale threshold. Therefore, for the 3 types of shadow regions, two grayscale thresholds can be set. Thus, the difference between the number of projectors and 1 can be used as the number of grayscale thresholds; that is, if the number of projectors (multiple first projectors) is N, the corresponding number of grayscale thresholds is N-1.

[0050] Step S204: Determine the value of each grayscale threshold based on the number of thresholds and the average grayscale value of the first image.

[0051] Furthermore, the grayscale threshold value can be determined based on the overall brightness of the first image; that is, the overall brightness of the first image varies depending on the ambient light surrounding the target object, resulting in different levels of brightness for each shadow area.

[0052] Specifically, the average grayscale value of the first image can be obtained based on the grayscale value of each pixel in the first image. It is understandable that the areas in the first image other than the shadow area are illuminated by white field images projected by multiple first projectors. Therefore, the grayscale values ​​of these other areas are relatively large, possibly even close to the grayscale value of 255 corresponding to white. The shadow area, on the other hand, has lower brightness, and its corresponding grayscale value is much smaller than that of other areas, possibly even close to the grayscale value of 0 corresponding to black. Since the area of ​​the shadow area is often much smaller than that of other areas, the average grayscale value of the first image obtained above will also be relatively small.

[0053] Furthermore, a preset scaling factor can be determined based on the number of grayscale thresholds; and based on the preset scaling factor, the grayscale value can be divided between 0 and the average grayscale value to determine the magnitude of the grayscale threshold. For example, if there are 3 types of shadow regions, the corresponding number of grayscale thresholds is 2, and preset scaling factors of 1 / 3 and 2 / 3 can be determined. If the average grayscale value is 168, then two grayscale thresholds can be determined: 168 × 1 / 3 = 56 and 168 × 2 / 3 = 112.

[0054] Step S205: If there are multiple grayscale thresholds, then the smallest grayscale threshold is determined from the multiple grayscale thresholds as the minimum grayscale threshold.

[0055] Step S206: If the gray values ​​of multiple pixels in the projection display area are all less than the minimum gray value threshold, then the area formed by the multiple pixels is determined as the target shadow area.

[0056] In this embodiment, if multiple pixels in the projection display area of ​​the first image have gray values ​​all less than the minimum gray value threshold, then the area formed by these multiple pixels is the target shadow area in the projection display area, which is the darkest part of the projection display area. The target shadow area is the area that cannot be projected by any one of the multiple first projectors.

[0057] Optionally, if the number of grayscale thresholds is 1, and there are multiple pixels in the projection display area whose grayscale values ​​are all less than this 1 grayscale threshold, then the area formed by the multiple pixels can be determined as the hard shadow area.

[0058] Step S207: Determine at least one non-hard shadow region based on the grayscale thresholds other than the minimum grayscale threshold among the plurality of grayscale thresholds.

[0059] For example, taking multiple grayscale thresholds, including a first grayscale threshold and a second grayscale threshold, as an example, the first grayscale threshold is the minimum grayscale threshold, and the second grayscale threshold is any other grayscale threshold. Based on this, if there are multiple first pixels in the projected display area of ​​the first image whose grayscale values ​​are greater than or equal to the first grayscale threshold and less than the second grayscale threshold, then the area composed of the multiple first pixels is determined to be a non-hard shadow area A. If there are multiple second pixels in the projected display area of ​​the first image whose grayscale values ​​are greater than or equal to the second grayscale threshold and less than the average grayscale value, then the area composed of the multiple second pixels is determined to be a non-hard shadow area B. Here, a non-hard shadow area is a shadow area that at least one of the multiple first projectors can project onto. It can be seen that three types of shadow areas will ultimately be determined: the target shadow area, non-hard shadow area A, and non-hard shadow area B. In a specific embodiment, non-hard shadow area A can be determined as the target non-hard shadow area.

[0060] Step S208: According to the preset marking strategy, mark the target shadow region and the at least one non-hard shadow region to obtain the third image.

[0061] The preset marking strategy can be understood as a marking strategy used to highlight and distinguish the target shadow region and all non-hard shadow regions in the first image. This allows the user to quickly determine the location of the target shadow region and the location of each type of non-hard shadow region by observing the third image.

[0062] Taking color filling as the default marking strategy as an example, the number of types of shadow areas is the same as the number of types of fill colors. Figure 2 Taking a projection system as an example, if it is determined that the first image contains three types of shadow regions, then according to the color filling strategy, a different color is filled for each type of shadow region. The color filling strategy can be pre-set with preset fill colors corresponding to different types of shadow regions; of course, users can also adjust the color filling strategy before color filling the first image according to their actual needs, so that the filled third image meets their actual requirements.

[0063] For example, please refer to Figure 5 , Figure 5 The image shown is a third image obtained by color filling during projection using the aforementioned projection system 10 as an example. Figure 5 The area filled with red shadows is the shadow area formed by three shadows, that is, the shadow area formed by the superposition of three shadows generated by the three first projectors. Figure 5 The area filled with yellow shadows is the shadow area formed by the superposition of the two shadows generated by the two first projectors. Figure 5The green-filled shadow area represents the shadow area formed by the projection of the first projector. In this way, by filling the shadow area with different colors, users can quickly distinguish between different types of shadow areas.

[0064] Step S209: Display the third image and the first prompt information, the first prompt information being used to prompt the user to adjust the position of the target object and / or the projector position of at least one first projector in order to reduce or eliminate the target shadow area.

[0065] In this embodiment, the first prompt information can be specifically displayed at the location corresponding to the hard shadow area in the third image. This provides a more intuitive reminder to the user to use different adjustment methods for different types of shadow areas, improving adjustment efficiency and faster improvement of the projection display effect. Based on this, after displaying the third image, the user can intuitively understand the location of the target shadow area and each type of non-hard shadow area by viewing the third image. At this time, regardless of whether the user adjusts the projection brightness parameter of any of the first projectors, since none of the first projectors can project onto the target shadow area, the user can move the position of the target object to reduce the hard shadow area; or, if multiple first projectors are not fixed, the user can move the position of at least one of the multiple first projectors to eliminate the hard shadow area.

[0066] Step S210: Adjust the projection brightness parameter of at least one of the plurality of first projectors to reduce or eliminate the at least one non-hard shadow area.

[0067] At the same time, brightness compensation for the non-hard shadow area can also be performed by increasing the projection brightness parameter of the first projector that can project onto the non-hard shadow area, thereby improving the brightness compensation efficiency for the non-hard shadow area to a certain extent.

[0068] In some implementations, after step S210, upon receiving an input shadow detection command, a white field image is again projected onto the repositioned target object through the plurality of first projectors, and an image is captured by the camera to obtain a fourth image. Similarly, a black field image is projected onto the repositioned target object through the plurality of first projectors, and an image is captured by the camera to obtain a fifth image. The difference image between the fourth and fifth images is obtained, and based on this difference image, a new projection display area formed by the projections from the repositioned first projectors in the fourth image is determined. The principle for obtaining the difference image between the fourth and fifth images is similar to that in the aforementioned embodiments, and can be found in the content of the aforementioned embodiments; it will not be repeated here.

[0069] Optionally, if only the target non-hard shadow area is determined from the new projection display area based on multiple new grayscale thresholds, then the target non-hard shadow area in the fourth image is marked according to a preset marking strategy to obtain a sixth image; the sixth image is displayed to prompt the user to adjust the position of the target object and / or the projector position of at least one first projector to reduce or eliminate the target non-hard shadow area. The specific implementation method for determining the target non-hard shadow area is the same as the aforementioned implementation method for determining at least one non-hard shadow area, and can be referred to the foregoing content, which will not be repeated here. Thus, in areas where the brightness compensation for the non-hard shadow area is poor after a certain degree of adjustment in step S210 (i.e., a certain degree of shadow still exists), marking the newly determined target non-hard shadow area in the fourth image and displaying the marked sixth image prompts the user to further fine-tune the position of the target object or the position of at least one first projector to minimize or eliminate the target non-hard shadow area to the greatest extent.

[0070] In other words, after the user moves the target object or the first projector and increases the projection brightness parameter of the first projector, the system will again identify whether a shadow area exists. If a shadow area exists, the system will continue to display its corresponding third image to assist the user in adjusting the shadow area until it disappears. Optionally, the system can also detect a trend in the size of the hard shadow area after the user moves the target object or the first projector. Based on this trend, an orientation adjustment indicator can be generated and displayed in the third image to prompt the user to continue moving the target object or the first projector so that the area of ​​the hard shadow area gradually decreases until it disappears.

[0071] Optionally, if the third image contains only non-hard shadow areas, the non-hard shadow areas can be eliminated by increasing the projection brightness parameter of the first projector capable of projecting onto those non-hard shadow areas. For example... Figure 6 As shown, behind the target object, there is only a single shadow area (i.e., a non-hard shadow area) formed by the projection of a single first projector. In this case, the brightness of the non-hard shadow area can be compensated by increasing the projection brightness parameter of the first projector that can project onto this non-hard shadow area, thereby eliminating the shadow. Figure 6 The three non-hard shadows behind the target object. Similarly, if brightness compensation cannot completely eliminate some areas of the non-hard shadow region, the target non-hard shadow region can still be identified from the reprojected display area as described above. According to the preset marking strategy, the target non-hard shadow region in the fourth image is marked to obtain the sixth image. The sixth image is displayed to prompt the user to adjust the position of the target object and / or the projector position of at least one first projector to minimize or eliminate the target non-hard shadow region to the greatest extent.

[0072] In other embodiments, after step S207, a preset fill color for each shadow region can be determined according to the preset fill strategy; the location of each shadow region in the real scene where the target object is located can be determined according to the relative position of the camera and the second projector; and the color-filled image corresponding to the preset fill color of each shadow region can be projected to its location in the real scene where the target object is located through the second projector. In other words, it is equivalent to projecting different colors to fill each shadow region, allowing the user to more intuitively understand the location of different shadow regions in the real scene, facilitating adjustments to the position of the target object and / or the first projector, and timely elimination of various shadow regions to ensure the effect of the projection display.

[0073] In some other implementations, different first projectors are controlled to display different numbers while projecting white field images, which can more intuitively show the user which shadow areas were produced by which first projector. This allows the user to more quickly know which first projector's projection brightness parameters need to be adjusted when the projection brightness parameters of the projectors need to be adjusted.

[0074] In this embodiment, users can quickly identify the shadow areas that the first projectors cannot project by viewing the displayed third image. This helps users quickly eliminate the hard shadow areas of the target object by adjusting the position of the target object and / or the position of at least one first projector. Furthermore, by adjusting the projection brightness parameters of at least one first projector, all non-hard shadow areas of the target object can be quickly eliminated to present a better projection display effect.

[0075] Please refer to Figure 7 The diagram illustrates a structural block diagram of a projection prompting device 300 according to an embodiment of this application, applied to a projection system. The projection system includes a camera and multiple first projectors, and the camera's shooting range covers the total projection range of the multiple first projectors. The device 300 may include: an image acquisition module 310, a display area determination module 320, a shadow area determination module 330, a marking module 340, and a display module 350.

[0076] The image acquisition module 310 is used to project a white field image onto the target object through the plurality of first projectors and capture the image with the camera to obtain a first image, and to project a black field image onto the target object through the plurality of first projectors and capture the image with the camera to obtain a second image.

[0077] The display area determination module 320 is used to obtain the difference image between the first image and the second image, and based on the difference image, determine the projection display area formed by the projection of the plurality of first projectors in the first image.

[0078] The shadow region determination module 330 is used to determine the target shadow region of the target object in the projection display area based on the gray value of each pixel in the projection display area of ​​the first image and at least one gray value threshold. The target shadow region is a place that cannot be projected by any of the first projectors.

[0079] The marking module 340 is used to mark the target shadow region in the first image according to a preset marking strategy to obtain a third image.

[0080] The display module 350 is used to display the third image to prompt the user to adjust the position of the target object and / or the projector position of at least one first projector to reduce or eliminate the target shadow area.

[0081] In some implementations, the display area determination module 320 may specifically be used to: perform binarization processing on the difference image to obtain a binarized image corresponding to the difference image; obtain the region position of the white area in the binarized image; and determine the image region corresponding to the region position of the white area from the first image as the projection display area formed by the projection of the plurality of first projectors.

[0082] In some embodiments, the projection prompting device 300 may further include a threshold quantity determination module and a threshold value determination module. The threshold quantity determination module may be used to determine the number of grayscale thresholds based on the number of projectors in the plurality of projectors before determining the shadow region of the target object in the projection display area based on the grayscale value of each pixel in the projection display area of ​​the first image and at least one grayscale threshold. The threshold value determination module may be used to determine the value of each grayscale threshold based on the threshold quantity and the average grayscale value of the first image.

[0083] In some implementations, the shadow region determination module 330 may be specifically used to: if there are multiple grayscale thresholds, determine the smallest grayscale threshold from the multiple grayscale thresholds as the minimum grayscale threshold; if there are multiple pixels in the projection display area whose grayscale values ​​are all less than the minimum grayscale threshold, determine the area formed by the multiple pixels as the target shadow region; and determine at least one non-hard shadow region based on the other grayscale thresholds among the multiple grayscale thresholds besides the minimum grayscale threshold.

[0084] In this mode, the marking module 340 can be specifically used to mark the target shadow region and the at least one non-hard shadow region according to the preset marking strategy to obtain the third image.

[0085] In this configuration, the display module 350 can specifically display the third image and a first prompt message. The first prompt message prompts the user to adjust the position of the target object or the position of at least one first projector to reduce or eliminate the target shadow area. The projection prompt device 300 may further include a parameter adjustment module, wherein the parameter adjustment module can be used to adjust the projection brightness parameter of at least one of the plurality of first projectors to reduce or eliminate the at least one non-hard shadow area.

[0086] In some embodiments, the image acquisition module 310 may be specifically used to, after adjusting the projection brightness parameters of at least one of the plurality of first projectors, project a white field image onto the target object after adjustment using the plurality of first projectors and capture an image using the camera to obtain a fourth image; and project a black field image onto the target object after adjustment using the plurality of first projectors and capture an image using the camera to obtain a fifth image. The display area determination module 320 may be specifically used to acquire the difference image between the fourth image and the fifth image, and based on the difference image between the fourth image and the fifth image, determine a new projection display area in the fourth image formed by the projection of the plurality of first projectors after adjustment. The marking module 340 may be specifically used to, if a target non-hard shadow area is determined from the new projection display area based on multiple new grayscale thresholds, mark the target non-hard shadow area in the fourth image according to a preset marking strategy to obtain a sixth image. The display module 350 can be specifically used to display the sixth image to prompt the user to adjust the position of the target object and / or the projector position of at least one first projector to reduce or eliminate the non-hard shadow area of ​​the target.

[0087] In some implementations, the display area determination module 320 may include a difference image acquisition unit. Specifically, the difference image acquisition unit may be used to: if the camera is a color camera, convert the first image to a grayscale image to obtain a first grayscale image, and convert the second image to a grayscale image to obtain a second grayscale image; acquire the difference image between the first grayscale image and the second grayscale image; if the camera is a monochrome camera, acquire the difference image between the first image and the second image.

[0088] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0089] In the several embodiments provided in this application, the coupling between modules can be electrical, mechanical, or other forms of coupling.

[0090] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0091] In summary, a first image is obtained by projecting white field images onto a target object using multiple first projectors and capturing the images with a camera; a second image is obtained by projecting black field images onto the target object using multiple first projectors and capturing the images with a camera; a difference image is obtained between the first and second images, and based on the difference image, the projection display area formed by the projections from multiple first projectors in the first image is determined; based on the grayscale value of each pixel in the projection display area of ​​the first image and at least one grayscale threshold, multiple shadow regions of the target object in the projection display area are determined, with different types of shadow regions containing different numbers of shadows of the target object; according to a preset filling strategy, the multiple shadow regions in the first image are filled with different colors to obtain a third image, wherein different types of shadow regions are filled with different colors; the third image is displayed to prompt adjustments to the position of the target object, the projection brightness parameters of at least one of the multiple projectors, and / or the projector position of at least one projector based on the third image, in order to eliminate multiple shadow regions. In this way, users can quickly identify the shadow areas that the first projectors cannot project by viewing the displayed third image. This helps users quickly eliminate the hard shadow areas of the target object by adjusting the position of the target object and / or the position of at least one first projector. Furthermore, by adjusting the projection brightness parameters of at least one first projector, all non-hard shadow areas of the target object can be quickly eliminated to present a better projection display effect.

[0092] The following will combine Figure 8 This application describes an electronic device.

[0093] Reference Figure 8 , Figure 8 This diagram illustrates a structural block diagram of an electronic device 400 according to an embodiment of this application. The method described above in this embodiment can be executed by this electronic device 400. The electronic device can be an electronic terminal with data processing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0094] The electronic device 400 in this application embodiment may include one or more of the following components: processor 401, memory 402, and one or more application programs, wherein the one or more application programs may be stored in memory 402 and configured to be executed by one or more processors 401, and the one or more programs are configured to perform the methods as described in the foregoing method embodiments.

[0095] Processor 401 may include one or more processing cores. Processor 401 connects to various parts within the electronic device 400 using various interfaces and lines, and performs various functions and processes data of the electronic device 400 by running or executing instructions, programs, code sets, or instruction sets stored in memory 402, and by calling data stored in memory 402. Optionally, processor 401 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 401 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the aforementioned modem can also be integrated into processor 401 and implemented as a separate communication chip.

[0096] The memory 402 may include random access memory (RAM) or read-only memory (ROM). The memory 402 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 402 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the electronic device 400 during use (such as the various correspondences described above).

[0097] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0098] In the several embodiments provided in this application, the coupling or direct coupling or communication connection between the modules shown or discussed may be an indirect coupling or communication connection through some interface, device or module, and may be electrical, mechanical or other forms.

[0099] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0100] Please refer to Figure 9 This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable medium 500 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0101] The computer-readable storage medium 500 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 500 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 500 has storage space for program code 510 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 510 may be compressed, for example, in a suitable form.

[0102] In some embodiments, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the steps in the above-described method embodiments.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A projection prompting method, characterized in that, Applied to a projection system, the projection system including a camera and a plurality of first projectors, the shooting range of the camera covering the total projection range of the plurality of first projectors, the method includes: A first image is obtained by projecting a white field image onto the target object using the plurality of first projectors and capturing the image using the camera; and a second image is obtained by projecting a black field image onto the target object using the plurality of first projectors and capturing the image using the camera. Obtain the difference image between the first image and the second image, and based on the difference image, determine the projection display area formed by the projection of the plurality of first projectors in the first image; Based on the grayscale value of each pixel in the projection display area of ​​the first image and at least one grayscale threshold, the target shadow area of ​​the target object in the projection display area is determined, and the target shadow area is a place that cannot be projected by any of the first projectors. According to a preset marking strategy, the target shadow region in the first image is marked to obtain the third image; The third image is displayed to prompt the user to adjust the position of the target object and / or the projector position of at least one first projector to reduce or eliminate the target shadow area.

2. The method according to claim 1, characterized in that, The step of determining the projection display area formed by the projections of the plurality of first projectors in the first image based on the difference image includes: The difference image is binarized to obtain the corresponding binarized image; Obtain the location of the white region in the binarized image; The image region corresponding to the location of the white area is determined from the first image and used as the projection display area formed by the projection of the plurality of first projectors.

3. The method according to claim 1, characterized in that, Before determining the shadow region of the target object in the projection display area based on the grayscale value of each pixel in the projection display area of ​​the first image and at least one grayscale threshold, the method further includes: The number of grayscale threshold values ​​is determined based on the number of projectors among the plurality of first projectors; The value of each grayscale threshold is determined based on the number of thresholds and the average grayscale value of the first image.

4. The method according to claim 1, characterized in that, Determining the target shadow region of the target object in the projection display area based on the grayscale value of each pixel in the projection display area of ​​the first image and at least one grayscale threshold includes: If there are multiple grayscale thresholds, then the smallest grayscale threshold is determined from the multiple grayscale thresholds as the minimum grayscale threshold. If the gray values ​​of multiple pixels in the projected display area are all less than the minimum gray value threshold, then the area formed by the multiple pixels is determined as the target shadow area. The method further includes: Based on the gray thresholds other than the minimum gray threshold among the plurality of gray thresholds, at least one non-hard shadow region is determined; The step of marking the target shadow region in the first image according to a preset marking strategy to obtain a third image includes: According to the preset marking strategy, the target shadow region and the at least one non-hard shadow region are marked to obtain the third image.

5. The method according to claim 4, characterized in that, The display of the third image includes: The third image and a first prompt message are displayed. The first prompt message is used to prompt the user to adjust the position of the target object or the position of at least one first projector in order to reduce or eliminate the target shadow area. The method further includes: The projection brightness parameters of at least one of the plurality of first projectors are adjusted to reduce or eliminate the at least one non-hard shadow area.

6. The method according to claim 5, characterized in that, After adjusting the projection brightness parameters of at least one of the plurality of first projectors, the method further includes: A white field image is projected onto the target object after its position has been adjusted by the plurality of first projectors, and the image is captured by the camera to obtain a fourth image; and a black field image is projected onto the target object after its position has been adjusted by the plurality of first projectors, and the image is captured by the camera to obtain a fifth image. Obtain the difference image between the fourth image and the fifth image, and based on the difference image between the fourth image and the fifth image, determine the new projection display area in the fourth image formed by the projection of the plurality of first projectors after the position is adjusted; If a target non-hard shadow region is determined from the new projection display area based on multiple new grayscale thresholds, then the target non-hard shadow region in the fourth image is marked according to the preset marking strategy to obtain the sixth image; The sixth image is displayed to prompt the user to adjust the position of the target object and / or the projector position of at least one first projector to reduce or eliminate the non-hard shadow area of ​​the target.

7. The method according to any one of claims 1-5, characterized in that, The step of obtaining the difference image between the first image and the second image includes: If the camera is a color camera, then the first image is converted into a grayscale image to obtain a first grayscale image, and the second image is converted into a grayscale image to obtain a second grayscale image; Obtain the difference image between the first grayscale image and the second grayscale image; If the camera is a monochrome camera, then the difference image between the first image and the second image is obtained.

8. A projection prompting device, characterized in that, An apparatus for use in a projection system, the projection system including a camera and a plurality of first projectors, wherein the camera's shooting range covers the total projection range of the plurality of first projectors, the apparatus comprising: The image acquisition module is used to project a white field image onto the target object through the plurality of first projectors and capture the image with the camera to obtain a first image, and to project a black field image onto the target object through the plurality of first projectors and capture the image with the camera to obtain a second image; The display area determination module is used to obtain the difference image between the first image and the second image, and based on the difference image, determine the projection display area formed by the projection of the plurality of first projectors in the first image; The shadow region determination module is used to determine the target shadow region of the target object in the projection display region based on the gray value of each pixel in the projection display region of the first image and at least one gray threshold. The target shadow region is a place that cannot be projected by any of the first projectors. The marking module is used to mark the target shadow region in the first image according to a preset marking strategy to obtain a third image; The display module is used to display the third image to prompt the user to adjust the position of the target object and / or the projector position of at least one first projector to reduce or eliminate the target shadow area.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that can be invoked by a processor to perform the method as described in any one of claims 1 to 7.

Images