Image processing method, image processing apparatus, and projection device

Abstract

Provided in the present application are an image processing method, an image processing apparatus, and a projection device. The image processing method comprises: acquiring a first image, wherein the first image is captured towards a projection medium after an infrared fill light is turned on, and a projected image is displayed on the projection medium; on the basis of the first image, determining whether the projected image is blocked by a target object; and when it is determined that the projected image is blocked by the target object, determining a first region of the projected image that is blocked, and blocking a second region of a source image, wherein the second region is a region of the source image corresponding to the first region of the projected image. A user usually uses a projection device for viewing in a dimly lit environment, and therefore turning on an infrared fill light can improve the imaging quality of a CMOS module for a target object, and an intelligent eye protection function can thus be achieved on the basis of a CMOS assembly. Compared with a projection device equipped only with a CMOS module, the present application involves slightly increased cost.

Description

Image processing method, image processing device and projection device TECHNICAL FIELD

[0001] The present application relates to the technical field of terminals, and in particular to an image processing method, an image processing device and a projection device. BACKGROUND

[0002] At present, most projectors on the market are equipped with a complementary metal-oxide semiconductor (CMOS) module. Based on the CMOS module, the projector can realize various intelligent sensing functions, such as automatic focusing, automatic trapezoidal correction, automatic obstacle avoidance, and automatic screen entering. In addition, some projectors are also equipped with an ambient light sensor (ALS), and the projector can also realize ambient light self-adaptive functions based on the ALS.

[0003] For projectors additionally equipped with a time of flight (TOF) module, they can further realize an intelligent eye protection function. This function detects whether a user is present in the projection area through the TOF module, and automatically closes the projection or blocks the screen in the area where the user is located when the user is present, thereby reducing the damage to the user's eyes.

[0004] However, the cost of the TOF module used to realize the intelligent eye protection function of the projector is relatively high. For example, the cost of a 120*90 TOF module is about 40 to 60 yuan, and even for a small array TOF (8*8), the cost needs to be 20 to 30 yuan. Therefore, TOF modules are usually only equipped in high-end projectors, and low-end projectors are often not equipped. Since the imaging quality of the visible light CMOS module is poor in a dark environment, it is difficult to identify a human body in real time through an algorithm, which leads to the fact that low-end projectors equipped with only a CMOS module cannot realize the intelligent eye protection function.

[0005] Therefore, how to realize the intelligent eye protection function in low-end projectors without affecting the cost or only increasing a small amount of cost has become a problem to be solved. SUMMARY

[0006] Therefore, the present application provides an image processing method, an image processing device and a projection device. The projection device can realize an intelligent eye protection function based on the equipped CMOS component (including a CMOS module and an infrared fill light), and only a small amount of cost is increased compared with the projection device equipped with only a CMOS module.

[0007] In a first aspect, the present application provides an image processing device, comprising a processing module and a complementary metal-oxide semiconductor (CMOS) component, the CMOS component comprising a CMOS module and an infrared light supplement lamp, and a light filtering structure in the CMOS module allowing infrared light to pass through.

[0008] The CMOS module is configured to capture an image, the image comprising a first image captured towards a projection medium displaying a projection image after the infrared light supplement lamp is turned on.

[0009] The processing module is configured to determine, based on the first image, whether the projection image is blocked by a target object, and determine a first area of the projection image that is blocked and a second area in a source image of the blocking source when it is determined that the projection image is blocked by the target object, the second area being a corresponding area of the first area in the projection image in the source image.

[0010] In some embodiments, the image processing device can be a part of a projection device, or the projection device.

[0011] Since the user usually watches a movie in a relatively dark environment using the projection device, the imaging quality of the CMOS module on the target object can be improved after the infrared light supplement lamp is turned on. Then, the target object can be accurately determined to enter the projection area according to the image (i.e., the first image) captured by the CMOS module, and the eye protection process (i.e., determining the first area of the projection image that is blocked and the second area in the source image of the blocking source) can be performed when it is determined that the target object enters the projection area. In this way, after the projection device projects the source image of the second area onto the projection medium, the second area in the source image will be projected onto the target object. Since the second area in the source image is blocked, the harm of the light emitted by the projection device to the user's eyes can be reduced. That is, the CMOS component with the infrared light supplement lamp can realize the intelligent eye protection function. Compared with the TOF-based intelligent eye protection function, the cost of the present application is lower.

[0012] In addition, the present application can realize the intelligent eye protection function by adding the infrared light supplement lamp (and the light filtering structure in some cases) to the existing projection device with only the CMOS module, which only increases a small amount of cost.

[0013] It should be understood that the projection device (such as a projector, a projector) is generally projected towards the projection medium, and the corresponding CMOS module is also captured towards the projection medium. Therefore, the CMOS module captures the projection medium and the projection image displayed by the projection medium after the infrared fill light is turned on. When the target object enters the projection area of the projection device, the CMOS module captures the target object after the infrared fill light is turned on.

[0014] Exemplarily, the target object can include a person, an animal, etc.

[0015] Exemplarily, the "source image" can be understood as an image to be projected by the projection device. The "projection image" can be understood as an image displayed on the projection medium after the source image is projected by the projection device. The projection image can also be referred to as a projection picture. The projection medium can refer to an entity for carrying and displaying the projection image, and the projection medium includes but is not limited to a screen, a projection curtain, a special coating material, a water surface, and a fog surface, etc.

[0016] Exemplarily, the projection area can refer to a spatial range in which the projection device can project the source image, including the physical space between the projection device and the projection medium.

[0017] Exemplarily, the first area in the projection image can be determined according to the first area and a second mapping relationship. The second mapping relationship can be determined according to the pose of the projection device, the distance between the projection device and the projection medium (i.e., the projection distance), and the projection parameters of the projection device, etc.

[0018] Exemplarily, the second area in the source image can be shielded by adding a dark mask such as a black mask.

[0019] According to the first aspect, the CMOS module includes a camera, and the camera includes a lens, a CMOS sensor, and one or more filter structures.

[0020] Exemplarily, at least one filter structure in the one or more filter structures allows infrared light to pass through.

[0021] Exemplarily, if the camera includes one filter structure, the camera can be referred to as a fixed filter structure camera; wherein the filter structure is an all-pass filter structure.

[0022] Exemplarily, if the camera includes a plurality of filter structures, the camera can be referred to as an adjustable (or switchable) filter structure camera; wherein the plurality of filter structures includes two or more of an all-pass filter structure, a narrow-band filter structure, or an infrared cut-off filter structure.

[0023] It should be understood that in some cases, one camera includes multiple lenses, or one camera includes multiple CMOS sensors.

[0024] According to the first aspect, or any one of the implementations of the above first aspect, the CMOS module includes multiple cameras, each of the multiple cameras including one lens, one CMOS sensor, and one filter structure, the filter structures of any two cameras being different.

[0025] That is, the CMOS module includes multiple fixed filter structure cameras.

[0026] According to the first aspect, or any one of the implementations of the above first aspect, the CMOS module includes a first camera and a second camera, the first camera including one lens, one CMOS sensor, and multiple filter structures, the second camera including one lens, one CMOS sensor, and one filter structure.

[0027] Exemplarily, the first camera can be one or more, and the second camera can be one or more.

[0028] For example, the first camera includes one lens, one CMOS sensor, and an infrared cut filter structure; the second camera includes one lens, one CMOS sensor, an all-pass filter structure, and a narrowband filter structure.

[0029] According to the first aspect, or any one of the implementations of the above first aspect, the filter structure includes an all-pass filter structure or a narrowband filter structure.

[0030] Exemplarily, the filter structure can be a filter coating or a filter sheet.

[0031] Exemplarily, the filter structure can further include a visible light filter (also referred to as an infrared cut filter (IR-CUT)).

[0032] According to the first aspect, or any one of the implementations of the above first aspect, the processing module is configured to:

[0033] determine, based on the first image, region description information of a region in which a target object is located in the first image; obtain region description information of a region in which a projection image is located in the first image; and determine, based on the region description information of the region in which the target object is located in the first image and the region description information of the region in which the projection image is located in the first image, whether the projection image is occluded by the target object.

[0034] In a possible implementation, the region where the target object is located can refer to a region covered by the pixel points included in the target object. In this case, the region description information of the region where the target object is located can refer to coordinates of edge pixel points of the region covered by the pixel points included in the target object.

[0035] In a possible implementation, the region where the target object is located can refer to a region framed by a bounding box of the region covered by the pixel points included in the target object. In this case, the region description information of the region where the target object is located can refer to description information of the bounding box (for example, coordinates of a center of the bounding box (or coordinates of a top-left corner of the bounding box), a height of the bounding box, and a width of the bounding box).

[0036] In a possible implementation, the region description information of the region where the projection image is located in the first image can be identified based on the first image. The projection image is usually rectangular, and the region description information of the region where the projection image is located in the first image can include coordinates of a center of the rectangle (or coordinates of a top-left corner of the rectangle), a height of the rectangle, and a width of the rectangle.

[0037] In a possible implementation, the region description information of the region where the projection image is located in the CMOS captured image generated in the process of implementing other perception functions (for example, an automatic focusing function, a trapezoidal correction function, an automatic obstacle avoidance function, and an automatic entering screen function) can be read from the storage module.

[0038] In a possible implementation, the region description information of the region where the projection image is located in the previous first image determined last time can be read from the storage module.

[0039] For example, whether the region where the target object is located in the first image overlaps the region where the projection image is located in the first image can be determined based on the region description information of the region where the target object is located in the first image and the region description information of the region where the projection image is located in the first image. If the regions do not overlap, it is determined that the projection image is not blocked by the target object. If the regions overlap, it is determined that the projection image is blocked by the target object.

[0040] According to the first aspect, or any one of the implementations of the first aspect, the processing module is further configured to:

[0041] determining difference information between the region description information of the region where the target object is located in the second image and the region description information of the region where the target object is located in the third image, the second image being the first image of the current shooting, and the third image being the first image of the last shooting; when the difference information satisfies a first preset condition, judging whether the projection image is occluded by the target object based on the region description information of the region where the target object is located in the second image and the region description information of the region where the projection image is located in the second image. In this way, it can be determined whether the target object detected from the first image of the current shooting is the same object as the target object detected from the first image of the last shooting, avoiding the object in the projection image from being detected as the target object, and the probability of misjudgment can be reduced.

[0042] According to the first aspect, or any one of the implementations of the first aspect, the processing module is further configured to:

[0043] When the difference information does not satisfy the first preset condition, obtaining motion speed information of the target object in the third image; determining predicted region description information of the region of the target object in the second image based on the motion speed information of the target object in the third image and the region description information of the region where the target object is located in the third image; and judging whether the projection image is occluded by the target object based on the predicted region description information of the region of the target object in the second image and the region description information of the region where the projection image is located in the second image. In this way, when it is determined that the target object detected from the first image of the current shooting is not the same object as the target object detected from the first image of the last shooting (i.e., when the object in the projection image is detected as the target object), the region description information of the region of the target object detected from the first image of the last shooting can be used to predict the region description information of the region of the target object in the first image of the current shooting.

[0044] According to the first aspect, or any one of the implementations of the first aspect, the processing module is configured to:

[0045] Any one of a human body detection algorithm, a human body segmentation algorithm, or a depth estimation algorithm is used to process the first image to obtain the region description information of the region where the target object is located in the first image; it is determined whether a second preset condition is satisfied based on the region description information of the region where the target object is located in the first image; and when the second preset condition is satisfied, it is determined whether the projection image is occluded by the target object based on the region description information of the region where the target object is located in the first image and the region description information of the region where the projection image is located in the first image. In this way, it can be ensured that the target object detected from the first image is not the object in the projection image.

[0046] According to a first aspect, or any possible implementation mode of the first aspect, the second preset condition comprises that a region where the target object is located in the first image in which the target object exists for the first time is not overlapped with a region where the projection image is located (or the target object in the first image in which the target object exists for the first time is located outside the region where the projection image is located).

[0047] It should be understood that the second preset condition can also be other conditions, for example, at least one of the following conditions: the target object exists in two consecutive first images, the target object moves towards the projection image, or the movement speed of the target object is within a preset range.

[0048] The first image in which the target object exists for the first time can be understood as the first image in which the target object exists for the first time acquired after the projection device is turned on and before the eye protection processing is performed; or can be understood as the first image in which the target object exists for the first time acquired after the eye protection processing is stopped and before the next eye protection processing is performed.

[0049] According to the first aspect, or any possible implementation mode of the first aspect, the processing module is configured to:

[0050] determine the similarity between each image block in the preset region in the first image and the image block at the corresponding position in the preset region in the reference image; and determine the region description information of the region where the target object is located in the first image based on the region description information of the connected region composed of the target image block in the first image, the similarity between the target image block in the first image and the image block at the corresponding position in the reference image being less than the similarity threshold.

[0051] Compared with the human body detection algorithm or the human body segmentation algorithm, the calculation amount of determining the region description information of the region where the target object is located in the first image based on the similarity is smaller.

[0052] According to a second aspect, the present application provides an image processing method, which comprises the following steps: first, acquiring a first image, the first image being obtained by shooting towards a projection medium after an infrared light supplement lamp is turned on, the projection medium displaying a projection image; then, judging whether the projection image is blocked by a target object based on the first image; and subsequently, when it is determined that the projection image is blocked by the target object, determining a first region where the projection image is blocked and a second region in a source image, the second region being a region corresponding to the first region in the projection image in the source image.

[0053] According to a second aspect, determining whether the projection image is occluded by the target object based on the first image comprises: determining region description information of a region where the target object is located in the first image based on the first image; obtaining region description information of a region where the projection image is located in the first image; and determining whether the projection image is occluded by the target object based on the region description information of the region where the target object is located in the first image and the region description information of the region where the projection image is located in the first image.

[0054] According to the second aspect or any one of the implementations of the second aspect, the method further includes: determining difference information between region description information of a region where the target object is located in a second image and region description information of a region where the target object is located in a third image, the second image being the first image obtained this time, and the third image being the first image obtained last time; and determining whether the projection image is occluded by the target object based on the region description information of the region where the target object is located in the first image and the region description information of the region where the projection image is located in the first image includes: when the difference information satisfies a first preset condition, determining whether the projection image is occluded by the target object based on the region description information of the region where the target object is located in the second image and the region description information of the region where the projection image is located in the second image.

[0055] According to the second aspect or any one of the implementations of the second aspect, determining whether the projection image is occluded by the target object based on the region description information of the region where the target object is located in the first image and the region description information of the region where the projection image is located in the first image further includes: when the difference information does not satisfy the first preset condition, obtaining motion speed information of the target object in the third image; determining predicted region description information of a region where the target object in the third image is located in the second image based on the motion speed information of the target object in the third image and the region description information of the region where the target object is located in the third image; and determining whether the projection image is occluded by the target object based on the predicted region description information of the region where the target object in the third image is located in the second image and the region description information of the region where the projection image is located in the second image.

[0056] According to a second aspect, or any possible implementation mode of the second aspect, the determining the region description information of the region where the target object is located in the first image based on the first image comprises: processing the first image by using any one of a human body detection algorithm, a human body segmentation algorithm or a depth estimation algorithm to obtain the region description information of the region where the target object is located in the first image; and the determining whether the projection image is occluded by the target object based on the region description information of the region where the target object is located in the first image and the region description information of the region where the projection image is located in the first image comprises: determining whether a second preset condition is met based on the region description information of the region where the target object is located in the first image; and when the second preset condition is met, determining whether the projection image is occluded by the target object based on the region description information of the region where the target object is located in the first image and the region description information of the region where the projection image is located in the first image.

[0057] According to the second aspect, or any possible implementation mode of the second aspect, the second preset condition comprises that the region where the target object is located in the first image in which the target object exists is not overlapped with the region where the projection image is located.

[0058] According to the second aspect, or any possible implementation mode of the second aspect, the determining the region description information of the region where the target object is located in the first image based on the first image comprises: determining the similarity between each image block in a preset region in the first image and a corresponding position image block in a preset region in a reference image; and determining the region description information of the region where the target object is located in the first image based on the region description information of a connected region composed of target image blocks in the first image, the similarity between the target image blocks in the first image and the corresponding position image blocks in the reference image being less than a similarity threshold.

[0059] According to the second aspect, or any possible implementation mode of the second aspect, the processing the first image by using the depth estimation algorithm to obtain the region description information of the region where the target object is located in the first image comprises: determining a depth map of the first image by using the depth estimation algorithm; binarizing the depth map of the image based on a binarization threshold to obtain a binarization image; and determining the region description information of the region where the target object is located based on the region description information of a connected region composed of target pixel points in the binarization image, the pixel value of the target pixel points in the binarization image being less than the binarization threshold.

[0060] The second aspect and any possible implementation mode of the second aspect correspond to the first aspect and any possible implementation mode of the first aspect respectively. The technical effects of the second aspect and any possible implementation mode of the second aspect can refer to the technical effects of the first aspect and any possible implementation mode of the first aspect, which will not be described here.

[0061] In a third aspect, the present application provides a projection device, comprising a processing module, a complementary metal-oxide-semiconductor (CMOS) component and a projection lens component, the CMOS component comprising: a CMOS module and an infrared light supplement lamp, the CMOS module having a light filter component allowing infrared light to pass through;

[0062] The projection lens component is configured to project a first source image to a projection medium to display a first projection image on the projection medium.

[0063] The CMOS module is configured to capture an image, the image comprising a first image captured towards the projection medium after the infrared light supplement lamp is turned on.

[0064] The processing module is configured to determine, based on the first image, whether the first projection image is blocked by a target object, and determine, when it is determined that the first projection image is blocked by the target object, a first area of the first projection image that is blocked and a second area in a second source image that is blocked, the second area being a corresponding area of the first area in the first projection image in the second source image.

[0065] The projection lens component is further configured to project the second source image of the blocked second area to the projection medium to display a second projection image on the projection medium, a third area in the second projection image being blocked, the third area being the same as the first area.

[0066] Exemplarily, the third area being the same as the first area can be understood as the third area being the same as the first area in terms of position and size.

[0067] In a fourth aspect, the present application provides a projection device, comprising: a memory, a processor, a complementary metal-oxide-semiconductor (CMOS) component and a projection lens component, the CMOS component comprising: a CMOS module and an infrared light supplement lamp, the CMOS module having a light filter component allowing infrared light to pass through, the memory being coupled to the processor; the memory storing program instructions, when the program instructions are executed by the processor, causing the projection device to perform the following steps:

[0068] Obtaining a first image, the first image being an image captured by the CMOS component towards a projection medium after the infrared light supplement lamp is turned on, the projection medium displaying a projection image;

[0069] Determining, based on the first image, whether the projection image is blocked by a target object.

[0070] When it is determined that the projection image is blocked by the target object, determining a first area of the projection image that is blocked and a second area in a source image that is blocked, the second area being a corresponding area of the first area in the projection image in the source image.

[0071] In a fifth aspect, the present application provides a projection device, the projection device comprising:

[0072] an information obtaining module configured to obtain a first image, the first image being obtained by photographing the projection medium after the infrared light supplement lamp is turned on, the projection medium displaying a projection image;

[0073] an occlusion determining module configured to determine, based on the first image, whether the projection image is occluded by the target object;

[0074] an eye protection processing module configured to, when it is determined that the projection image is occluded by the target object, determine a first area of the projection image that is occluded and a second area in a source image, the second area being a corresponding area of the first area in the source image.

[0075] Exemplarily, the projection device provided by the present application can be part of a projection device, or can be the projection device.

[0076] In a sixth aspect, the present application provides a chip, comprising one or more interface circuits and one or more processors; the one or more processors receive or send data through the one or more interface circuits; when the one or more processors execute computer instructions, the processors execute the method in the second aspect or any possible implementation manner of the second aspect.

[0077] In a seventh aspect, the present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, when the computer program runs on a computer or a processor, the computer or the processor executes the method in the second aspect or any possible implementation manner of the second aspect.

[0078] In an eighth aspect, the present application provides a computer program product, the computer program product comprising computer instructions, when the computer instructions are executed by a computer or a processor, the computer or the processor executes the method in the second aspect or any possible implementation manner of the second aspect.

[0079] The projection device, the computer readable storage medium, the computer program product or the chip provided by the present embodiment are all used to execute the corresponding method provided above, and thus the beneficial effects achieved thereby can refer to the beneficial effects in the corresponding method provided above. BRIEF DESCRIPTION OF DRAWINGS

[0080] FIG. 1A is a schematic diagram of an application scenario 100 according to an embodiment of the present application;

[0081] FIG. 1B is a schematic diagram of projection according to an embodiment of the present application;

[0082] FIG. 1C is a schematic diagram of projection according to an embodiment of the present application;

[0083] FIG. 2A is a schematic diagram of an appearance of a projection device 110 according to an embodiment of the present application;

[0084] FIG. 2B is a schematic diagram of an appearance of a CMOS component 210 according to an embodiment of the present application;

[0085] FIG. 2C is a schematic diagram of a filter switching mode according to an embodiment of the present application;

[0086] FIG. 2D is a system block diagram of a projection device 110 according to an embodiment of the present application;

[0087] FIG. 2E is a schematic diagram of a sensing function that can be implemented by a projection device 110 according to an embodiment of the present application;

[0088] FIG. 3A is a schematic diagram of an image processing process 300 according to an embodiment of the present application;

[0089] FIG. 3B is a schematic diagram of an eye protection processing process according to an embodiment of the present application;

[0090] FIG. 3C is a schematic diagram of another eye protection processing process according to an embodiment of the present application;

[0091] FIG. 4A is a schematic diagram of a part of another image processing process 400 according to an embodiment of the present application;

[0092] FIG. 4B is a schematic diagram of another part of another image processing process 400 according to an embodiment of the present application;

[0093] FIG. 4C is a schematic diagram of region description information of a target object in a fourth image according to an embodiment of the present application;

[0094] FIG. 5A is a schematic diagram of a part of another image processing process 500 according to an embodiment of the present application;

[0095] FIG. 5B is a schematic diagram of another part of another image processing process 500 according to an embodiment of the present application;

[0096] FIG. 5C is a schematic diagram of a preset region according to an embodiment of the present application;

[0097] FIG. 5D is a schematic diagram of division of an image block according to an embodiment of the present application;

[0098] FIG. 5E is a schematic diagram of a target connected region according to an embodiment of the present application;

[0099] FIG. 5F is a schematic diagram of another target connected region according to an embodiment of the present application;

[0100] FIG. 5G is a schematic diagram of another preset region according to an embodiment of the present application;

[0101] FIG. 6A is a schematic diagram of a part of another image processing procedure 600 according to an embodiment of the present application;

[0102] FIG. 6B is a schematic diagram of another part of the image processing procedure 600 according to an embodiment of the present application;

[0103] FIG. 7A is a schematic diagram of a part of another image processing procedure 700 according to an embodiment of the present application;

[0104] FIG. 7B is a schematic diagram of another part of the image processing procedure 700 according to an embodiment of the present application;

[0105] FIG. 8 is a schematic diagram of an image processing device 800 according to an embodiment of the present application;

[0106] FIG. 9A is a schematic diagram of an auto-focusing procedure 900 according to an embodiment of the present application;

[0107] FIG. 9B is a schematic diagram of a preset focusing map according to an embodiment of the present application;

[0108] FIG. 10A is a schematic diagram of an auto-keystone correction procedure 1000 according to an embodiment of the present application;

[0109] FIG. 10B is a schematic diagram of a preset keystone correction map according to an embodiment of the present application;

[0110] FIG. 10C is a schematic diagram of a keystone correction procedure according to an embodiment of the present application;

[0111] FIG. 11A is a schematic diagram of an ambient light adaptation procedure 1100 according to an embodiment of the present application;

[0112] FIG. 11B is a schematic diagram of a preset ambient light adaptation map according to an embodiment of the present application;

[0113] FIG. 12 is a schematic diagram of an auto-obstacle avoidance procedure 1200 according to an embodiment of the present application;

[0114] FIG. 13A is a schematic diagram of an auto-curtain-in procedure 1300 according to an embodiment of the present application;

[0115] FIG. 13B is a schematic diagram of a curtain according to an embodiment of the present application;

[0116] FIG. 14 is a schematic diagram of a device according to an embodiment of the present application. DETAILED DESCRIPTION

[0117] With reference to the drawings and brief description of the drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of the present application.

[0118] The term "and / or" used herein is only used to describe an association relationship of associated objects, and means that three relationships can exist, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone.

[0119] The terms "first" and "second" and the like in the description and claims of the embodiments of the present application are used to distinguish different objects, and are not used to describe a specific order of the objects. For example, the first target object and the second target object are used to distinguish different target objects, and are not used to describe a specific order of the target objects.

[0120] In the embodiments of the present application, the words "exemplarily" or "for example" and the like are used to mean as an example, illustration or description. Any embodiment or design scheme described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the words "exemplarily" or "for example" and the like are intended to present the relevant concept in a specific manner.

[0121] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more. For example, a plurality of processing units means two or more processing units; a plurality of systems means two or more systems.

[0122] In the description of the embodiments of the present application, the modules / components shown in the frame diagram (or structure diagram or system diagram) are only one example of the present application, and the actual frame (or structure or system) can include more or less modules / components than those shown in the diagram, or can have a different configuration of components. In addition, the various components / modules shown in the diagram can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0123] FIG. 1A is a schematic diagram of an application scenario 100 according to an embodiment of the present application.

[0124] In FIG. 1A, the user 130 can open the projection device 110 and select a video to be played; then, the projection device 110 can project a source image sequence (including multiple source images) corresponding to the video selected by the user 130 onto the projection screen 120 to display a projection image sequence (including multiple projection images 140) on the projection screen 120 (this process is also referred to as a projection process); in this way, the user 130 can watch the video.

[0125] Exemplarily, a "source image" can be understood as an image to be projected by a projection device. A "projection image" can be understood as an image displayed on a projection medium after a source image is projected by a projection device. The projection image can also be referred to as a projection picture. The projection medium can refer to an entity for carrying and displaying a projection image, and the projection medium includes but is not limited to a screen, the projection screen 120, a special coating material, a water surface, a fog surface, and the like.

[0126] In the projection process, the user 130 can enter a projection area of the projection device 110 (the projection area can refer to a spatial range in which the projection device can project a source image, including a physical space between the projection device and the projection medium), for example, the user 130 moves from position 1 to position 2; at this time, the light emitted by the projection device 110 is blocked by the user 130, and the corresponding projection image 140 displayed on the projection screen 120 is also blocked by the user 130. When the user faces or side faces the projection device 110, the light emitted by the projection device 110 can enter the eyes of the user 130, causing harm to the eyes of the user 130.

[0127] Based on this, the projection device 110 of the present application can perform eye protection processing when detecting that the user 130 enters the projection area; in this way, when the user 130 enters the projection area of the projection device 110, the harm of the light emitted by the projection device 110 to the eyes of the user 130 can be reduced.

[0128] FIG. 1B is a projection schematic diagram of an embodiment of the present application.

[0129] Referring to FIG. 1B, 160 is a source image, and 140 is a projection image. The projection device 110 can determine, based on the image captured by the projection device 110, a region 150 in which the projection image 140 is occluded by the user 130 (the region 150 is determined according to an overlapping region of a region in which the projection image 140 is located in the image captured by the projection device 110 and a region in which the user 130 is actually located (i.e., a region covered by pixels included in the user 130), and a first mapping relationship; the first mapping relationship can be determined according to a distance between the projection device 110 and a projection medium (i.e., a shooting distance), a shooting parameter, a pose of the projection device 110, and the like). Then, a region 170 in the source image 160 corresponding to the region 150 is occluded (the region 170 can be determined according to the region 150 and a second mapping relationship; the second mapping relationship can be determined according to the pose of the projection device 110, the distance between the projection device 110 and the projection medium (i.e., a projection distance), and a projection parameter of the projection device 110, and the like); for example, a black mask is added to the region 170 in the source image 160. Wherein, the region 150 in which the projection image 140 is occluded by the user 130 is determined, and the region 170 in the source image 160 corresponding to the region 150 is occluded, which is an implementation manner of the eye protection processing.

[0130] FIG. 1C is another projection schematic diagram of an embodiment of the present application.

[0131] Referring to FIG. 1C, 160 is a source image, and 140 is a projection image. The projection device 110 can determine, based on the image captured by the projection device 110, a region 180 in which the projection image 140 is occluded by the user 130 (the region 180 is determined according to an overlapping region of a region in which the projection image 140 is located in the image captured by the projection device 110 and a circumscribed frame of a region in which the user 130 is actually located, and a first mapping relationship). Then, a region 190 in the source image 160 corresponding to the region 180 is occluded (the region 180 can be determined according to the region 190 and a second mapping relationship); for example, a black mask is added to the region 190 in the source image 160. Wherein, the region 180 in which the projection image 140 is occluded by the user 130 is determined, and the region 190 in the source image 160 corresponding to the region 180 is occluded, which is an implementation manner of the eye protection processing.

[0132] FIG. 2A is a schematic diagram of an appearance of a projection device 110 according to an embodiment of the present application.

[0133] In FIG. 2A, the projection device 110 can include a complementary metal-oxide semiconductor (CMOS) component 210 and a projection lens component 220. It should be understood that the projection device 110 can further include a processing module and a storage module.

[0134] FIG. 2B is a schematic diagram of an appearance of a CMOS assembly 210 according to an embodiment of the present application.

[0135] Exemplarily, the CMOS assembly 210 can include a CMOS module 211 and an infrared light supplement lamp 212. The CMOS module 211 can include a lens, a CMOS sensor, and a filter structure.

[0136] In FIG. 2B(1), the CMOS assembly 210 can include a plurality of infrared light supplement lamps 212, the plurality of infrared light supplement lamps 212 surround the lens of the CMOS module 211, and the plurality of infrared light supplement lamps 212 form a ring shape.

[0137] In FIG. 2B(2), the CMOS assembly 210 can include a plurality of infrared light supplement lamps 212, the plurality of infrared light supplement lamps 212 surround the lens of the CMOS module 211, and the plurality of infrared light supplement lamps 212 form a rectangular shape.

[0138] In FIG. 2B(3), the CMOS assembly 210 can include one infrared light supplement lamp 212, and the one infrared light supplement lamp 212 is arranged side by side with the lens of the CMOS module 211.

[0139] It should be understood that the present application does not limit the number and distribution of the infrared light supplement lamps 212 included in the CMOS assembly 210; and when the power of a single infrared light supplement lamp is smaller, the CMOS assembly 210 includes more infrared light supplement lamps.

[0140] In a possible manner, the CMOS module 211 can include a camera, and the camera can include a lens, a CMOS sensor, and one or more filter structures. At least one of the one or more filter structures can allow infrared light to pass through. When the CMOS module 211 includes one filter structure, the filter structure can be an all-pass filter structure. When the CMOS module 211 includes two filter structures, the two filter structures can be an all-pass filter structure and an infrared cut-off filter structure, respectively; or the two filter structures can be a narrow-band filter structure and an infrared cut-off filter structure, respectively. When the CMOS module 211 includes three filter structures, the three filter structures can be an all-pass filter structure, a narrow-band filter structure, and an infrared cut-off filter structure, respectively.

[0141] Exemplarily, the filter structure can be a filter coating or a filter. The present application takes the filter as a filter for example. When the CMOS module is a camera and the camera includes one filter, the CMOS module 211 can be regarded as a fixed filter camera, and the filter of the camera is an all-pass filter. When the CMOS module is a camera and the camera includes a plurality of filters, the camera can be referred to as an adjustable (or switchable) filter camera.

[0142] FIG. 2C is a schematic diagram of a filter switching mode according to an embodiment of the present application.

[0143] As shown in FIG. 2C, the adjustable (or switchable) filter camera can switch the filter corresponding to the position of the lens in the CMOS module 211 by moving left and right, moving up and down, and rotating. In this way, light can be incident on the CMOS sensor in the CMOS module 211 through the lens in the CMOS module 211 and the filter corresponding to the position of the lens. It should be understood that the present application does not limit the manner of adjusting (or switching) the filter.

[0144] In one possible manner, the CMOS module 211 can include a plurality of cameras, each of the plurality of cameras including a lens, a CMOS sensor, and a filter structure. The filter structures of any two cameras are different. In this case, each of the plurality of cameras is a fixed filter camera. For example, the CMOS module 211 can include a camera including only a clear filter and a camera including only an infrared cut filter (IR-CUT). For another example, the CMOS module 211 can include a camera including only a narrowband filter and a camera including only an infrared cut filter (IR-CUT). For yet another example, the CMOS module 211 can include a camera including only a clear filter, a camera including only a narrowband filter, and a camera including only an infrared cut filter.

[0145] It should be understood that the CMOS module 211 of the present application can also include a first camera including a lens, a CMOS sensor, and a plurality of filter structures, and a second camera including a lens, a CMOS sensor, and a filter structure. That is, the CMOS module 211 can include both a fixed filter structure camera and an adjustable filter structure camera.

[0146] The following embodiments take the CMOS module 211 as an adjustable filter camera for example.

[0147] FIG. 2D is a system block diagram of a projection device 110 according to an embodiment of the present application.

[0148] In FIG. 2D, the projection device 110 can include an interaction module 21, a control module 22, a signal acquisition module 23, a storage module 24, a central processing module 25, and a projection module 26. The signal acquisition module 23 can include a CMOS component 210 and other sensing components. The projection module 26 can include a projection lens component 220. The above-mentioned processing modules can include the interaction module 21, the control module 22, the central processing module 25, and the projection module 26 (except the projection lens component 220).

[0149] The interaction module 21 can be configured to read and parse the interaction information input by the user, and then pass the parsed instructions to the control module. The user can input the interaction information through a remote controller, gestures, voice, a mobile phone application, etc. The parsed instructions can include, but are not limited to, turning on the ambient light self-adaptation function, turning on the automatic focusing function, turning on the automatic correction (or keystone correction) function, turning on the automatic entering function, turning on the automatic obstacle avoidance function, and turning on the intelligent eye protection function.

[0150] The control module 22 can control the signal acquisition module 23 according to the instructions parsed by the interaction module 21 or system instructions. For example, when the instruction parsed by the interaction module 21 is to turn on the automatic focusing function, the control module 22 is responsible for controlling the infrared fill light in the CMOS component 210 to be turned off, switching the position of the IR-CUT filter and the lens of the CMOS module, and instructing the CMOS module to acquire a visible light image. For another example, when the instruction parsed by the interaction module 21 is to turn on the intelligent eye protection function, the control module 22 is responsible for controlling the infrared fill light in the CMOS component 210 to be turned on, switching the position of the full-pass filter / narrow-band filter and the lens of the CMOS module, and instructing the CMOS module to acquire a full-pass / narrow-band image.

[0151] The signal acquisition module 23 is configured to acquire sensing data. For example, the CMOS module can be configured to acquire a visible light image, a full-pass image, or a narrow-band image.

[0152] The storage module 24 is configured to store the sensing data acquired by the signal acquisition module 23 and output the sensing data to the central processing module 25, and is further configured to store the cache information generated by the central processing module 25 (including cache data and processing results generated during algorithm running, etc.).

[0153] The central processing module 25 is configured to 1) process the sensing data (or pre-processed sensing data) output by the storage module 24 using algorithms corresponding to various sensing functions to obtain processing results; 2) pass data generated during the processing and required by subsequent processes to the storage module for storage; and 3) process the source image based on the processing results and output the processed source image to the projection module 26.

[0154] The projection module 26 is configured to invoke the light engine to process the processed source image, and then project the processed source image processed by the light engine onto the projection medium 120 through the projection lens assembly 220.

[0155] FIG. 2E is a schematic diagram of the sensing functions that can be implemented by the projection device 110 according to an embodiment of the present application. In FIG. 2E, the projection device 110 according to an embodiment of the present application is equipped with the CMOS assembly 210, and can implement the following sensing functions based on the CMOS assembly, including but not limited to: 1. ambient light adaptive function, 2. auto-focusing function, 3. keystone correction function, 4. automatic obstacle avoidance function, 5. automatic curtain entering function, and 6. intelligent eye protection function.

[0156] For example, after one or more of the auto-focusing function, the keystone correction function, the automatic obstacle avoidance function, and the automatic curtain entering function are turned on, the projection device 110 can perform one or more of the following processes during the booting process: auto-focusing process, keystone correction process, automatic obstacle avoidance process, and automatic curtain entering process. After that, during the projection process, if the projection device 110 is detected to be moved or receives user input interaction information, the projection device 110 can perform one or more of the following processes again: auto-focusing process, keystone correction process, automatic obstacle avoidance process, and automatic curtain entering process; otherwise, none of the auto-focusing process, the keystone correction process, the automatic obstacle avoidance process, and the automatic curtain entering process is performed.

[0157] For example, after the ambient light adaptive function is turned on, the projection device 110 can perform the ambient light adaptive process in real time during the projection process. After the intelligent eye protection function is turned on, the projection device 110 can perform the eye protection process in real time during the projection process.

[0158] The following describes the process of implementing the sensing functions by the projection device 110.

[0159] Intelligent eye protection function

[0160] FIG. 3A is a schematic diagram of an image processing process 300 according to an embodiment of the present application. The image processing process 300 can be performed by the central processing module 25.

[0161] S301, a first image is obtained, the first image is obtained by shooting towards the projection medium after the infrared fill light is turned on, and the projection medium displays a projection image.

[0162] Exemplarily, when implementing any of the sensing functions of the automatic focusing function, the trapezoidal correction function, the automatic obstacle avoidance function, the automatic entering screen function or the environment self-adaption function, the control module 22 can turn off the infrared fill-in light in the CMOS component and switch the position of the full-pass filter or the IR-CUT filter and the lens in the CMOS module. Then, the control module 22 instructs the CMOS module to capture images in real time or periodically. Subsequently, the CMOS module stores the captured images into the storage module 24. In the process of implementing the intelligent eye protection function, the control module 22 can turn on the infrared fill-in light in the CMOS component and switch the position of the full-pass filter or the narrow-band filter and the lens in the CMOS module. Then, the control module 22 instructs the CMOS module to capture images in real time or periodically. Subsequently, the CMOS module stores the captured images into the storage module 24. For the convenience of description, the application will refer to the image captured by the CMOS module after the infrared fill-in light is turned on and the CMOS module is directed towards the projection medium (the projection device 110 is usually directed towards the projection medium to project the source image, and correspondingly, the CMOS module is also directed towards the projection medium to capture the image) as the first image. It should be understood that the CMOS module captures the image after the infrared fill-in light is turned on and the CMOS module is directed towards the projection medium, which can not only capture the projection medium itself, but also capture the projection image displayed by the projection medium.

[0163] Exemplarily, when the target object enters the projection area of the projection device 110, the CMOS module captures the image after the infrared fill-in light is turned on and the CMOS module is directed towards the projection medium, which can also capture the target object. Further, the central processing module 25 can read the first image from the storage module 24. Then, based on the first image, it is judged whether the eye protection process needs to be performed, i.e., S302 is executed.

[0164] Exemplarily, the target object can include but is not limited to: human, animal, etc.

[0165] S302, based on the first image, it is judged whether the projection image is blocked by the target object.

[0166] Exemplarily, the central processing module 25 can read one first image from the storage module 24 each time according to the shooting time sequence of the first image. After reading one first image from the storage module 24 each time, the central processing module 25 can judge whether the projection image is blocked by the target object based on the first image. Since the target object enters the projection area, the projection image will be blocked by the target object; therefore, it is judged whether the projection image is blocked by the target object.

[0167] When it is determined that the projection image is blocked by the target object, it can be determined that the target object enters the projection area, at this time, the central processing module 25 can perform S303, i.e., the eye protection processing is performed. When it is determined that the projection image is not blocked by the target object, the central processing module 25 can return to perform S301, i.e., the next first image is read from the storage module 24.

[0168] Exemplarily, the central processing module 25 can determine the region description information of the region where the target object is located in the first image based on the first image, and obtain the region description information of the region where the projection image is located in the first image, and then determine whether the projection image is blocked by the target object based on the region description information of the region where the target object is located in the first image and the region description information of the region where the projection image is located in the first image.

[0169] Exemplarily, the central processing module 25 can first detect whether the target object exists in the first image, and when it is determined that the target object exists in the first image, the region description information of the region where the target object is located in the first image is determined based on the detection result of the above detection and / or other processing on the first image.

[0170] In a possible manner, the region where the target object is located can refer to the region covered by the pixel points contained by the target object, in which case, the region description information of the region where the target object is located can refer to the coordinates of the edge pixel points of the region covered by the pixel points contained by the target object.

[0171] In a possible manner, the region where the target object is located can refer to the region framed by the circumscribed frame of the region covered by the pixel points contained by the target object, in which case, the region description information of the region where the target object is located can refer to the description information of the circumscribed frame (such as the center coordinates of the circumscribed frame (or the upper left corner coordinates of the circumscribed frame), the height of the circumscribed frame and the width of the circumscribed frame).

[0172] When it is determined that the target object does not exist in the first image, the central processing module 25 can return to perform S301, i.e., the next first image is read from the storage module 24.

[0173] In a possible manner, the central processing module 25 can identify the region description information of the region where the projection image is located in the first image based on the first image. The region description information of the region where the projection image is located in the first image can include the center coordinates of the rectangle (or the upper left corner coordinates of the rectangle), the height of the rectangle and the width of the rectangle.

[0174] In one possible implementation, the central processing module 25 can read the region description information of the region where the projected image is located in the CMOS captured image generated in the process of implementing other perception functions (such as the auto-focusing function, the trapezoidal correction function, the automatic obstacle avoidance function, and the automatic entering screen function) from the storage module 24.

[0175] In one possible implementation, the central processing module 25 can read the region description information of the region where the projected image is located in the previous first image determined last time from the storage module 24.

[0176] Illustratively, the central processing module 25 can determine whether the region where the target object is located in the first image and the region where the projected image is located in the first image overlap based on the region description information of the region where the target object is located in the first image and the region description information of the region where the projected image is located in the first image.

[0177] When the central processing module 25 determines that the region where the target object is located in the first image and the region where the projected image is located in the first image overlap, it is determined that the projected image is occluded by the target object; at this time, S303 can be executed, i.e., the eye protection processing is performed.

[0178] When the central processing module 25 determines that the region where the target object is located in the first image and the region where the projected image is located in the first image do not overlap, it is determined that the projected image is not occluded by the target object; at this time, the eye protection processing is not needed, and S301 can be returned to be executed, i.e., the next first image is read from the storage module 24.

[0179] S303, when it is determined that the projected image is occluded by the target object, the first region where the projected image is occluded and the second region in the source image are determined, and the second region is the region corresponding to the first region of the projected image in the source image.

[0180] Illustratively, when it is determined that the projected image is occluded by the target object, the central processing module 25 can calculate the region (hereinafter referred to as the first region) occluded by the target object.

[0181] Illustratively, the central processing module 25 can calculate the overlapping region between the region where the target object is located in the first image and the region where the projected image is located in the first image; then, the central processing module 25 can determine the first region according to the first mapping relationship and the overlapping region. Subsequently, the central processing module 25 can determine the region (hereinafter referred to as the second region) corresponding to the first region of the projected image in the source image according to the second mapping relationship; then, the central processing module 25 can occlude the second region in the source image.

[0182] Exemplarily, one implementation of shielding the second region in the source image can be adding a black mask to the second region in the source image. It should be understood that other dark masks (dark colors are less harmful to eyes than light colors) can also be added to the second region in the source image, which is not limited in the present application.

[0183] In this way, after the projection lens assembly of the subsequent projection device 110 projects the source image of the second region onto the projection medium, the second region in the source image is projected onto the target object; since the second region in the source image is shielded, the harm of the light emitted by the projection device 110 to the eyes of the user 130 can be reduced.

[0184] FIG. 3B is a schematic diagram of an eye protection processing process according to an embodiment of the present application.

[0185] The region in the first image where the projection image is located and the circumscribed frame of the region covered by the pixel points included in the target object in the first image are shown in FIG. 3B(1). According to the overlapping region of the region in the first image where the projection image is located and the circumscribed frame of the region covered by the pixel points included in the target object in the first image, and the first mapping relationship, the first region can be determined, as shown in FIG. 3B(2). Then, according to the first region and the second mapping relationship, the second region in the source image can be determined; as shown in FIG. 3B(3), a black mask is added to the second region of the source image. After projecting the source image with the black mask added to the second region in FIG. 3B(3) to the projection medium, the projection image displayed on the projection medium and the target object is as shown in FIG. 3B(4).

[0186] FIG. 3C is a schematic diagram of another eye protection processing process according to an embodiment of the present application.

[0187] The region in the first image where the projection image is located and the region covered by the pixel points included in the target object in the first image are shown in FIG. 3C(1). According to the overlapping region of the region in the first image where the projection image is located and the region covered by the pixel points included in the target object in the first image, and the first mapping relationship, the first region can be determined, as shown in FIG. 3C(2). Then, according to the first region and the second mapping relationship, the second region in the source image can be determined; as shown in FIG. 3C(3), a black mask is added to the second region of the source image. After projecting the source image with the black mask added to the second region in FIG. 3C(3) to the projection medium, the projection image displayed on the projection medium and the target object is as shown in FIG. 3C(4).

[0188] Since the user usually watches the movie in a relatively dark environment using the projection device 110, the infrared fill light is turned on, and the imaging quality of the CMOS module on the target object can be improved. Then, according to the image (i.e., the first image) captured by the CMOS module, it can be accurately determined whether the target object enters the projection area. After determining that the target object enters the projection area, the eye protection process (i.e., determining the first area of the projection image that is blocked and the second area in the source image that is blocked) is performed. That is, based on the CMOS component equipped with the infrared fill light, the intelligent eye protection function can be realized.

[0189] It should be noted that the source image corresponding to the projection image displayed by the projection medium in S301 and the source image in S303 can be the same image (corresponding to the scene of the image (such as the main interface) projected by the projection device 110), or different images (corresponding to the scene of the video projected by the projection device 110). The present application does not limit this.

[0190] FIGS. 4A and 4B are schematic diagrams of another image processing process 400 according to an embodiment of the present application. The image processing process 400 is based on the image processing process 300, and describes an implementation manner of determining whether the projection image is blocked by the target object based on the first image captured by the CMOS component.

[0191] In S401, the control module switches the filter corresponding to the lens position in the CMOS module to a full-pass filter, turns on the infrared fill light, and instructs the CMOS module to capture an image.

[0192] Exemplarily, after the projection device 110 is powered on, the control module 22 can detect whether the intelligent eye protection function of the projection device 110 is in an open state. When it is detected that the intelligent eye protection function of the projection device 110 is in the open state, the filter corresponding to the lens position in the CMOS module is switched to a full-pass filter, and the infrared fill light is turned on. Then, the control module 22 can send a control instruction to the CMOS module to instruct the CMOS module to capture an image.

[0193] In one possible case, the default state of the intelligent eye protection function of the projection device 110 is the open state when the projection device 110 is shipped.

[0194] In one possible case, the default state of the intelligent eye protection function of the projection device 110 is the closed state when the projection device 110 is shipped. After the projection device 110 is turned on, the user can perform an operation of opening the intelligent eye protection function, and the projection device 110 can turn on the intelligent eye protection function in response to the operation of the user, i.e., switch the state of the intelligent eye protection function to the open state.

[0195] Exemplarily, the full-pass filter can allow visible light and infrared light to pass through.

[0196] S402, the CMOS module captures a first image and stores the first image in the storage module.

[0197] For example, the CMOS module can capture the first image according to the control instruction sent by the control module 22.

[0198] In one possible implementation, the control instruction sent by the control module 22 can be used to instruct the CMOS module to capture the first image in real time. In this way, the CMOS module can capture the first image in real time.

[0199] In one possible implementation, the control instruction sent by the control module 22 can be used to instruct the CMOS module to capture the first image at a preset period. In this way, the CMOS module can capture the first image at the preset period. For example, the preset period can be set according to the time length that the central processing module 25 determines whether the projection image is blocked by the target object based on the CMOS module capturing the first image; and the preset period can be less than or equal to the time length.

[0200] For example, the CMOS module can store the captured first image in the storage module 24.

[0201] S403, the central processing module reads an i-th first image from the storage module, and obtains an i-th fourth image, where i is an initial value of 1.

[0202] Wherein, i is a positive integer.

[0203] For example, the central processing module 25 can read a first image from the storage module 24 each time in the order of the capturing time of the first image, and then perform S404.

[0204] For example, the fourth image can be a first image read from the storage module 24 by the central processing module 25 after the projection device 110 is started and before the projection device 110 performs eye protection for the first time, or after the projection device 110 stops eye protection and before the next eye protection is performed, when the intelligent eye protection state of the projection device 110 is in the on state.

[0205] S404, the central processing module determines whether the target object exists in the i-th fourth image.

[0206] Exemplarily, the central processing module 25 can perform human body detection on the ith fourth image by using a human body detection algorithm to obtain a detection result; then, whether the target object exists in the ith fourth image can be determined according to the detection result. When the detection result is a detection frame (that is, the circumscribed frame of the region covered by the pixel points of the target object), it can be determined that the target object exists in the ith fourth image, and S406 can be executed. When the detection result is not the detection frame information, it can be determined that the target object does not exist in the ith fourth image; and S405 can be executed.

[0207] S405, the central processing module adds 1 to i.

[0208] Exemplarily, after S405 is executed, S403 can be executed, that is, the central processing module 25 can read the next first image from the storage module.

[0209] S406, the central processing module determines the motion state information of the target object in the ith fourth image based on the detection result obtained by performing human body detection on the ith fourth image by using the human body detection algorithm.

[0210] Exemplarily, the motion state information of the target object in the ith fourth image can include region description information of the region where the target object is located in the ith fourth image and motion speed information of the target object in the ith fourth image. In one possible manner, the motion speed information of the target object in the ith fourth image can include the motion speed of the target object in the ith fourth image. In one possible manner, the motion speed information of the target object in the ith fourth image can include the initial speed of the target object in the ith fourth image and the acceleration of the target object in the ith fourth image. The present application takes the motion speed information of the target object in the ith fourth image as the motion speed of the target object in the ith fourth image as an example for description.

[0211] In one possible case, the detection result obtained by human body detection can include the left upper corner coordinate of the detection frame and the right lower corner coordinate of the detection frame. The detection frame obtained by human body detection is shown in FIG. 3B.

[0212] In one possible case, the detection result obtained by human body detection can include the left upper corner coordinate of the detection frame and the width and height of the detection frame.

[0213] In one possible case, the detection result obtained by human body detection can include the center coordinate of the detection frame and the width and height of the detection frame.

[0214] Exemplarily, the result of the human body detection on the i-th fourth image can be taken as the region description information of the region where the target object is located in the i-th fourth image. In this application, the region description information of the region where the target object is located in the i-th fourth image is taken as an example, which includes the center coordinates of the detection frame and the width and height of the detection frame, as shown in FIG. 4C.

[0215] FIG. 4C is a schematic diagram of the region description information of the region where the target object is located in a fourth image according to an embodiment of the present application. In FIG. 4C, the center coordinates of the detection frame 0 of the (i-1)-th fourth image are (x0, y0), the height of the detection frame 0 is h0, and the width of the detection frame 0 is w0. The center coordinates of the detection frame 1 of the i-th fourth image are (x1, y1), the height of the detection frame 1 is h1, and the width of the detection frame 1 is w1.

[0216] Exemplarily, when i = 1, that is, the i-th fourth image is the first fourth image obtained by the central processing module 25 from the storage module 24, the motion speed of the target object in the i-th fourth image is 0. When i is greater than 1, that is, the i-th fourth image is not the first fourth image obtained by the central processing module 25 from the storage module 24, the motion speed of the target object in the i-th fourth image can be determined according to the position of the target object in the i-th fourth image and the position of the target object in the (i-1)-th fourth image. The position of the target object in the fourth image can be represented by the center coordinates of the detection frame. As shown in FIG. 4B, the position of the target object in the (i-1)-th fourth image is (x0, y0), and the position of the target object in the i-th fourth image is (x1, y1). The motion speed v of the target object in the i-th fourth image is x1-x0.

[0217] Exemplarily, the central processing module 25 can also store the motion state information of the target object in the i-th fourth image into the storage module 24.

[0218] S407, the central processing module obtains the region description information of the region where the projection image is located in the i-th fourth image.

[0219] In one possible way, the central processing module 25 can detect the region description information of the region where the projection screen or display screen is located in the i-th fourth image, and determine the region description information of the region where the projection screen or display screen is located in the i-th fourth image as the region description information of the region where the projection image is located in the i-th fourth image.

[0220] In one possible way, the central processing module 25 can take the region description information of the region where the projection image is located in the (i-1)-th fourth image as the region description information of the region where the projection image is located in the i-th fourth image.

[0221] It should be understood that the present application does not limit the execution order of S406 and S407.

[0222] S408, the central processing module determines whether i is equal to 1.

[0223] If i is equal to 1, the execution of S405 can be returned. If i is not equal to 1, S409 can be executed.

[0224] S409, the central processing module determines whether the second preset condition is met based on the region description information of the region where the target object is located in one or more fourth images and the region description information of the region where the projection image is located in the i-th fourth image.

[0225] Exemplarily, determining whether the second preset condition is met based on the region description information of the region where the target object is located in one or more fourth images can be understood as determining whether the second preset condition is met based on the region description information of the region where the target object is located in one or more of the first to i-th fourth images.

[0226] Exemplarily, in order to ensure that the target object detected from the i-th fourth image is not the object in the projection image, the second preset condition can be set as: the target object in the first fourth image where the target object exists is located outside the region where the projection image is located. Exemplarily, whether the target object in the first fourth image where the target object exists is located outside the region where the projection image is located can be determined according to the region description information of the region where the target object is located in the first fourth image where the target object exists and the region description information of the region where the projection image is located in the first fourth image where the target object exists.

[0227] When the target object in the first fourth image where the target object exists is located outside the region where the projection image is located, it can be determined that the second preset condition is met, and S410 can be executed. When the target object in the first fourth image where the target object exists is located inside the projection image, it can be determined that the second preset condition is not met, and the execution of S405 can be returned.

[0228] Exemplarily, the second preset condition can further include at least one of the following: the target object exists in two consecutive fourth images, the target object moves towards the projection image, or the movement speed of the target object is within a preset range.

[0229] Exemplarily, when the central processing module 25 determines that the movement state information of the target object in the i-1th fourth image and the movement state information of the target object in the i-th fourth image are stored in the storage module 24, it can be determined that the target object exists in two consecutive fourth images; otherwise, it is determined that the target object does not exist in two consecutive fourth images.

[0230] Exemplarily, the central processing module 25 can determine whether the target object moves towards the projection image according to the motion state information of the target object in the (i-1)th fourth image and the motion state information of the target object in the ith fourth image.

[0231] Exemplarily, the central processing module 25 can determine whether the motion speed of the target object is within a preset range according to the motion state information of the target object in the (i-1)th fourth image and the motion state information of the target object in the ith fourth image.

[0232] If at least one of the following conditions is met: the target object exists in two continuous fourth images, the target object moves towards the projection image, or the motion speed of the target object is within a preset range, it is determined that the second preset condition is met, and S410 can be performed; otherwise, it is determined that the second preset condition is not met, and S405 can be returned to perform.

[0233] S410, the central processing module determines whether the projection image is occluded by the target object based on the region description information of the region where the target object is located in the ith fourth image and the region description information of the region where the projection image is located in the ith fourth image.

[0234] Exemplarily, the central processing module 25 can determine whether the region where the target object is located in the ith fourth image overlaps with the region where the projection image is located in the ith fourth image based on the region description information of the region where the target object is located in the ith fourth image and the region description information of the region where the projection image is located in the ith fourth image; when it is determined that the region where the target object is located in the ith fourth image overlaps with the region where the projection image is located in the ith fourth image, it is determined that the projection image is occluded by the target object, and S411 can be performed. When it is determined that the region where the target object is located in the ith fourth image does not overlap with the region where the projection image is located in the ith fourth image, it is determined that the projection image is not occluded by the target object, and S405 can be returned to perform.

[0235] S411, the central processing module determines a first region where the projection image is occluded and a second region in the occlusion source image based on the region description information of the region where the target object is located in the ith fourth image and the region description information of the region where the projection image is located in the ith fourth image.

[0236] Exemplarily, S411 is essentially an eye protection process, and S411 can refer to the description of S303 described above, which will not be repeated here.

[0237] S412, the central processing module reads a first image from the storage module to obtain a kth fifth image, and the initial value of k is 1.

[0238] wherein k is a positive integer.

[0239] Exemplarily, after performing S411, the central processing module 25 can perform S412, i.e., the central processing module 25 reads one first image each time as a fifth image from the storage module 24 in the shooting time sequence of the first images, starting from the i-th fourth image.

[0240] Exemplarily, the fifth image can refer to the first image obtained by the central processing module 25 from the storage module 24 after the projection device 110 ends one eye protection process and before stopping the eye protection process.

[0241] S413, the central processing module determines whether the target object exists in the k-th fifth image.

[0242] Exemplarily, S413 can refer to the description of S404 described above, and will not be repeated here.

[0243] When it is determined that the target object exists in the k-th fifth image, the central processing module 25 can perform S414. When it is determined that the target object does not exist in the k-th fifth image, the central processing module 25 can perform S423.

[0244] S414, the central processing module obtains the region description information of the region where the projection image is located in the k-th fifth image.

[0245] In one possible manner, the central processing module 25 can detect the region description information of the region where the projection screen or display screen is located in the k-th fifth image; and can determine the region description information of the region where the projection image is located in the k-th fifth image as the region description information of the region where the projection screen or display screen is located in the k-th fifth image.

[0246] In one possible manner, the central processing module 25 can determine the region description information of the region where the projection image is located in the k-th fifth image as the region description information of the region where the projection image is located in the k-th-1 fifth image.

[0247] S415, the central processing module determines the motion state information of the target object in the k-th fifth image based on the detection result obtained by performing human body detection on the k-th fifth image using the human body detection algorithm.

[0248] Exemplarily, the process of determining the motion state information of the target object in the k-th fifth image in S414 can refer to the process of determining the motion state information of the target object in the i-th fourth image in S406 described above, and will not be repeated here.

[0249] It should be noted that when k is equal to 1, the motion speed of the target object in the kth fifth image can be determined according to the position of the target object in the kth fifth image and the position of the target object in the ith fourth image. When k is greater than 1, the motion speed of the target object in the kth fifth image can be determined according to the position of the target object in the k-1th fifth image and the position of the target object in the kth fifth image.

[0250] Exemplarily, the present application does not limit the execution order of S414 and S415.

[0251] S416, the central processing module determines the difference information between the region description information of the target object in the kth fifth image and the region description information of the target object in the k-1th fifth image.

[0252] Wherein, the kth fifth image can also be referred to as the second image, and the k-1th fifth image can also be referred to as the third image.

[0253] Exemplarily, when k is greater than 1, the central processing module 25 can determine the difference between the height in the region description information of the target object in the kth fifth image and the height in the region description information of the target object in the k-1th fifth image, to obtain the height difference. The central processing module 25 can determine the difference between the width in the region description information of the target object in the kth fifth image and the width in the region description information of the target object in the k-1th fifth image, to obtain the width difference. And the central processing module 25 can determine the distance between the center coordinates in the region description information of the target object in the kth fifth image and the center coordinates in the region description information of the target object in the k-1th fifth image, to obtain the position difference.

[0254] Exemplarily, when k is equal to 1, the central processing module 25 can determine the difference between the height in the region description information of the target object in the kth fifth image and the height in the region description information of the target object in the ith fourth image, to obtain the height difference. The central processing module 25 can determine the difference between the width in the region description information of the target object in the kth fifth image and the width in the region description information of the target object in the ith fourth image, to obtain the width difference. And the central processing module 25 can determine the distance between the center coordinates in the region description information of the target object in the kth fifth image and the center coordinates in the region description information of the target object in the ith fourth image, to obtain the position difference.

[0255] Wherein, the difference information can include the height difference, the width difference and the position difference.

[0256] S417, the central processing module determines whether the difference information satisfies a first preset condition.

[0257] Exemplarily, the first preset condition can include that the height difference is less than a height threshold, the width difference is less than a width threshold, and the position difference is less than a distance threshold. Thus, when the height difference in the difference information is less than the height threshold, the width difference is less than the width threshold, and the position difference is less than the distance threshold, it can be determined that the target object detected from the first image of this time of shooting is the same object as the target object detected from the first image of the last time of shooting, and the probability of misjudgment can be reduced.

[0258] When the difference information satisfies the first preset condition, S418 can be performed; when the difference information does not satisfy the first preset condition, S419 is performed.

[0259] S418, the central processing module determines whether the projection image is occluded by the target object based on the region description information of the region where the target object is located in the kth fifth image and the region description information of the region where the projection image is located in the kth fifth image.

[0260] Exemplarily, S418 can refer to the description of S410 described above, and will not be described here.

[0261] When it is determined that the projection image is occluded by the target object, S421 can be performed; when it is determined that the projection image is not occluded by the target object, S424 can be performed.

[0262] S419, the central processing module determines the predicted region description information of the region of the target object in the kth fifth image in the k-1th fifth image based on the motion state information of the target object in the k-1th fifth image.

[0263] Exemplarily, the central processing module 25 can determine the predicted motion speed of the target object in the kth fifth image according to the motion speed of the target object in the k-1th fifth image. For example, the predicted motion speed of the target object in the kth fifth image = the motion speed of the target object in the k-1th fifth image.

[0264] Exemplarily, the central processing module 25 can take the width and height of the detection box corresponding to the target object in the k-1th fifth image as the width and height of the corresponding predicted detection box of the target object in the k-1th fifth image in the kth fifth image; and take the vertical coordinate of the center coordinate of the detection box corresponding to the target object in the k-1th fifth image as the vertical coordinate of the center coordinate of the corresponding predicted detection box of the target object in the k-1th fifth image in the kth fifth image. The central processing module 25 can determine the horizontal coordinate of the center coordinate of the corresponding predicted detection box of the target object in the k-1th fifth image in the kth fifth image according to the motion speed of the target object in the k-1th fifth image and the position of the target object in the k-1th fifth image. For example, the horizontal coordinate of the center coordinate of the corresponding predicted detection box of the target object in the k-1th fifth image in the kth fifth image = the horizontal coordinate of the center coordinate of the detection box corresponding to the target object in the k-1th fifth image + the motion speed of the target object in the k-1th fifth image.

[0265] The predicted region description information of the region of the target object in the k-1th fifth image in the kth fifth image can include the height, width and center coordinate of the predicted detection box.

[0266] S420, the central processing module determines whether the projection image is occluded by the target object based on the predicted region description information of the region of the target object in the k-1th fifth image in the kth fifth image and the region description information of the region where the projection image is located in the kth fifth image.

[0267] Exemplarily, S420 can refer to the description of S410 described above, and will not be described here again.

[0268] When it is determined that the projection image is occluded by the target object, S421 can be performed; when it is determined that the projection image is not occluded by the target object, S424 can be performed.

[0269] S421, the central processing module determines a first region where the projection image is occluded and a second region in the occlusion source image.

[0270] Exemplarily, S421 can refer to the description of S303 described above, and will not be described here again. After S421 is performed, S422 can be performed.

[0271] S422, the central processing module adds 1 to k.

[0272] After S422 is performed, S412 can be returned to perform.

[0273] S423, the central processing module determines whether the target object in the k-1th fifth image is located at the edge of the region where the projection image is located in the k-1th fifth image.

[0274] Exemplarily, when k is not equal to 1, the central processing module 25 can determine, according to the region description information of the region where the target object is located in the (k-1)th fifth image, whether the target object in the (k-1)th fifth image is located at the edge of the region where the projection image is located in the (k-1)th fifth image. If it is determined that the target object in the (k-1)th fifth image is located at the edge of the region where the projection image is located in the (k-1)th fifth image, it is reasonable that the target object is not detected in the kth fifth image, and S424 can be performed at this time. If it is determined that the target object in the (k-1)th fifth image is not located at the edge of the region where the projection image is located in the (k-1)th fifth image, it is unreasonable that the target object is not detected in the kth fifth image, that is, the target object is missed in the kth fifth image, and S419 can be performed at this time.

[0275] Exemplarily, when k is equal to 1, the (k-1)th fifth image refers to the ith fourth image, and at this time, the central processing module 25 can determine, according to the region description information of the region where the target object is located in the ith fourth image, whether the target object in the ith fourth image is located at the edge of the region where the projection image is located in the ith fourth image. For details, reference can be made to the description above, which will not be repeated here.

[0276] S424, the central processing module sets i to 1 and k to 1.

[0277] It should be understood that S424 is performed (when it is determined that the target object does not exist in the kth fifth image, or the projection image is not occluded by the target object), that is, the central processing module 25 stops the eye protection processing, that is, the central processing module 25 does not occlude the second region in the source image.

[0278] After S424 is executed, S403 can be returned to monitor again whether the eye protection processing is needed.

[0279] It should be noted that when it is determined that the target object does not exist in the kth fifth image, or the projection image is not occluded by the target object, the central processing module 25 can also send the source image for occluding the second region to the projection module 26; and then the projection module 26 can project the source image onto the projection medium 120.

[0280] FIG. 5A is a schematic diagram of another image processing process 500 according to an embodiment of the present application. The image processing process 500 is based on the image processing process 300, and describes an implementation manner of determining whether the projection image is occluded by the target object based on the first image captured by the CMOS component.

[0281] S501, the control module switches the filter corresponding to the lens position in the CMOS module to a narrow-band filter, turns on the infrared fill light, and instructs the CMOS module to capture.

[0282] Illustratively, S501 can refer to the description of S401 described above, which will not be repeated here. It should be noted that the difference between S501 and S401 is that S501 switches the filter corresponding to the lens position in the CMOS module to a narrow-band filter. Among them, the narrow-band filter in the present application can refer to a filter that allows only infrared light to pass, for example, an 850nm narrow-band filter (i.e., allowing light with a wavelength of about 850 nanometers (nm) to pass, while blocking or absorbing light of other wavelengths).

[0283] S502, the CMOS module captures to obtain a first image and store the first image in the storage module.

[0284] Illustratively, S502 can refer to the description of S402 described above, which will not be repeated here.

[0285] S503, the central processing module reads a plurality of continuous first images from the storage module.

[0286] For example, the central processing module 25 can read a plurality of continuous first images captured by the CMOS module within a preset time period from the storage module 24; wherein the preset time period can be set according to the demand, such as 2s, which is not limited in the present application.

[0287] S504, the central processing module determines a reference image based on the plurality of continuous first images.

[0288] Illustratively, the central processing module 25 can calculate the similarity of any two first images in the plurality of continuous first images; if the similarity of any two first images in the plurality of continuous first images is greater than a preset value, it can be determined that the plurality of continuous first images are similar, and there is no target object in the plurality of continuous first images; and then any one of the plurality of continuous first images can be taken as a reference image.

[0289] The algorithm for calculating the similarity of two first images can include but is not limited to: histogram comparison method, Histogram of Oriented Gradients (HOG) comparison method, mean square error comparison method, deep learning-based feature comparison method, etc.

[0290] S505, the central processing module reads an image from the storage module to obtain the kth fifth image, and the initial value of k is 1.

[0291] Exemplarily, the fifth image can refer to the first image captured by the CMOS module and read by the central processing module 25 from the storage module 24 after the reference image is determined.

[0292] Exemplarily, the central processing module 25 can start from a plurality of first images in the storage module 24, read one first image captured by the CMOS module as the fifth image in the order of capturing the first images each time.

[0293] S506, the central processing module determines the similarity between each image block in the preset region in the kth fifth image and the corresponding position image block in the preset region in the reference image.

[0294] Exemplarily, at the initial time, i.e., k = 1, the preset region can refer to the initial region set in advance. Since the target object usually enters the projection region from both sides of the projection image, the initial region can be the region on both sides of the projection image in the fifth image, such as the region filled with dots in FIG. 5C.

[0295] Exemplarily, the central processing module 25 can divide the kth fifth image into a plurality of image blocks, and divide the reference image into a plurality of image blocks; wherein the kth fifth image and the reference image are divided in the same manner. As shown in FIG. 5D, the kth fifth image (as shown in FIG. 5D(2)) and the reference image (as shown in FIG. 5D(1)) are both divided into a plurality of image blocks of the same size. Then, for each image block in the preset region in the kth fifth image, the central processing module 25 can calculate the similarity between the image block and the image block at the corresponding position in the preset region in the reference image.

[0296] S507, the central processing module determines whether there is a target object in the kth fifth image based on the similarity between each image block in the preset region in the kth fifth image and the corresponding position image block in the preset region in the reference image.

[0297] Exemplarily, the image block in the preset region in the kth fifth image and the corresponding position image block in the reference image can be referred to as a target image block (or a dissimilar image block) if the similarity between the image block and the corresponding position image block is less than a similarity threshold. The similarity threshold can be set as required.

[0298] Exemplarily, the connected region composed of the target image blocks (hereinafter referred to as the first connected region) can be multiple, and it can be determined whether the area of the first connected region with the largest area is greater than an area threshold. The area threshold can be set as required.

[0299] When the area of the first connected region with the largest area is less than or equal to the area threshold, it can be determined that there is no target object in the kth fifth image, and at this time, S509 can be executed.

[0300] When the area of the largest dissimilar connected region is greater than the area threshold, it can be determined that the target object exists in the kth fifth image; at this time, S508 can be executed.

[0301] S508, the central processing module determines the region description information of the region where the target object is located in the kth fifth image based on the region description information of the connected region composed of the target image blocks in the kth fifth image; wherein the similarity between the target image blocks in the kth fifth image and the image blocks at the corresponding positions in the reference image is less than the similarity threshold.

[0302] Exemplarily, the central processing module 25 can obtain the target connected region by discarding one or more first connected regions, and / or merging multiple first connected regions.

[0303] Specifically, the central processing module 25 can select the largest first connected region as the reference connected region; then, the distance between the other first connected regions and the reference connected region is calculated. If the distance between a certain other first connected region and the reference connected region is less than a preset value, the central processing module 25 can merge the other first connected region and the reference connected region; if the distance between a certain other first connected region and the reference connected region is greater than or equal to the preset value, the central processing module 25 can discard the other first connected region.

[0304] FIG. 5E is a schematic diagram of a target connected region according to an embodiment of the present application.

[0305] In FIG. 5E, there are 7 target image blocks, of which 6 target image blocks form a connected region 2 (the gray translucent region including the 6 target image blocks), and 1 target image block (a noise image block) forms a connected region 1 (the gray translucent region including the 1 target image block). In this case, the connected region 1 can be discarded; thus, the target connected region obtained is the connected region 2.

[0306] FIG. 5F is another schematic diagram of a target connected region according to an embodiment of the present application.

[0307] In FIG. 5F(1), there are 6 target image blocks, of which 2 target image blocks form a connected region 1 (the gray translucent region including the 2 target image blocks), and 4 target image blocks form a connected region 2 (the gray translucent region including the 4 target image blocks). Therefore, in this case, the connected region 1 and the connected region 2 can be merged to obtain a connected region 3; thus, the target connected region obtained is the connected region 3, as shown in FIG. 5F(2).

[0308] Then, the region description information of the target connected region can be determined.

[0309] In a possible scenario, the region description information of the target connected region can include the top-left corner coordinate of the target connected region and the right-bottom corner coordinate of the target connected region.

[0310] In a possible scenario, the region description information of the target connected region can include the top-left corner coordinate of the target connected region and the width and height of the target connected region.

[0311] In a possible scenario, the region description information of the target connected region can include the center coordinate of the target connected region and the width and height of the target connected region.

[0312] Then, the region description information of the region where the target object is located in the kth fifth image can be determined based on the region description information of the target connected region. For details, refer to the description of determining the region description information of the region where the target object is located in the ith first image based on the detection result in S406, which will not be described herein again.

[0313] S509, determining whether k is equal to 1.

[0314] For example, if k is equal to 1, S519 is performed; if k is not equal to 1, S520 is performed.

[0315] S510, the central processing module updates the preset region based on the region description information of the region where the target object is located in the kth fifth image.

[0316] For example, after the target connected region is determined in S508, the central processing module 25 can update the preset region based on the region description information of the region where the target object is located in the kth fifth image. The preset region contains the target connected region, and the preset region is larger than the target connected region.

[0317] FIG. 5G is a schematic diagram of another preset region according to an embodiment of the present application.

[0318] For example, when the target connected region is the connected region 3 in FIG. 5F(2), the preset region filled with small black dots in FIG. 5C can be updated to the preset region filled with small black dots in FIG. 5G.

[0319] S511, the central processing module obtains the region description information of the region where the projection image is located in the kth fifth image.

[0320] For example, after S510 is performed, S511 can be performed.

[0321] S512, the central processing module determines whether k is equal to 1.

[0322] When k is equal to 1, S515 can be performed; when k is not equal to 1, S513 can be performed.

[0323] S513, the central processing module determines difference information between the region description information of the region where the target object is located in the kth fifth image and the region description information of the region where the target object is located in the k-1th fifth image.

[0324] S514, the central processing module determines whether the difference information meets a first preset condition.

[0325] When the difference information meets the first preset condition, S515 can be performed; when the difference information does not meet the first preset condition, S516 can be performed.

[0326] S515, the central processing module determines whether the projection image is occluded by the target object based on the region description information of the region where the target object is located in the kth fifth image and the region description information of the region where the projection image is located in the kth fifth image.

[0327] When it is determined that the projection image is occluded by the target object, S518 can be performed; when it is determined that the projection image is not occluded by the target object, S521 can be performed.

[0328] S516, the central processing module determines predicted region description information of the region of the target object in the kth fifth image based on the motion state information of the target object in the k-1th fifth image.

[0329] S517, the central processing module determines whether the projection image is occluded by the target object based on the predicted region description information of the region of the target object in the kth fifth image in the k-1th fifth image and the region description information of the region where the projection image is located in the kth fifth image.

[0330] When it is determined that the projection image is occluded by the target object, S518 can be performed; when it is determined that the projection image is not occluded by the target object, S521 can be performed.

[0331] S518, the central processing module determines a first region where the projection image is occluded and a second region in the occlusion source image.

[0332] After S518 is performed, S519 is performed.

[0333] S519, the central processing module adds 1 to k.

[0334] After S519 is performed, S505 can be returned to perform.

[0335] S520, the central processing module determines whether the target object in the k-1th fifth image is located at the edge of the region where the projection image is located in the k-1th fifth image.

[0336] If it is determined that the target object in the k-1th fifth image is not located at the edge of the region where the projection image is located in the k-1th fifth image, it is unreasonable that the kth fifth image does not detect the target object, that is, the kth fifth image misses the detection of the target object, and S516 can be performed at this time.

[0337] If it is determined that the target object in the k-1th fifth image is located at the edge of the region where the projection image is located in the k-1th fifth image, it is reasonable that the kth fifth image does not detect the target object, and S521 can be performed at this time.

[0338] S521, the central processing module sets k to 1 and initializes the preset region.

[0339] Exemplarily, S513-S521 can refer to the description of S416-S424 described above, and will not be described here. After S521 is executed, S505 can be executed.

[0340] Compared with the human shape detection algorithm or the human shape segmentation algorithm, the calculation amount of determining the region description information of the region where the target object is located in the first image based on the similarity is smaller.

[0341] FIGS. 6A and 6B are schematic diagrams of another image processing process 600 according to an embodiment of the present application. The image processing process 600 is based on the image processing process 300, and describes an implementation manner of determining whether the projection image is blocked by the target object based on the first image captured by the CMOS component.

[0342] S601, the control module switches the filter corresponding to the lens position in the CMOS module to a narrow-band filter, turns on the infrared fill light, and instructs the CMOS module to capture.

[0343] Exemplarily, S601 can refer to the description of S501 described above, and will not be described here.

[0344] S602, the CMOS module captures to obtain the first image and store the first image into the storage module.

[0345] S603, the central processing module reads one first image from the storage module to obtain the ith fourth image, and the initial value of i is 1.

[0346] Exemplarily, S602-S603 can refer to the description of S402-S403 described above, and will not be described here.

[0347] S604, the central processing module determines the depth map of the ith fourth image by using a depth estimation algorithm.

[0348] Exemplarily, the central processing module 25 can process the i-th fourth image by using a monocular depth estimation algorithm to determine a depth map of the i-th fourth image. The depth map of the i-th fourth image includes distance information from each pixel point in the i-th fourth image to the CMOS module.

[0349] S605, the central processing module binarizes the depth map of the i-th fourth image based on the binarization threshold to obtain an i-th binarization image.

[0350] It should be understood that the pixel points included in the projection image in the i-th fourth image have the same distance to the CMOS module, and the distance from the pixel points included in the projection image in the i-th fourth image to the CMOS module is greater than the distance from the pixel points included in the target object in the i-th fourth image to the CMOS module. Further, the pixel value of each pixel point in the i-th fourth image can be compared with the binarization threshold to binarize the depth map of the i-th fourth image; in this way, according to the obtained binarization image, the target object in the i-th fourth image and the object in the projection image can be distinguished.

[0351] In one possible manner, the pixel value of the pixel point greater than or equal to the binarization threshold in the i-th fourth image can be set to 1, and the pixel value of the pixel point less than the binarization threshold in the i-th fourth image can be set to 0; in this way, the pixel points included in the target object in the obtained binarization image are black, and other pixel points are white.

[0352] In one possible manner, the pixel value of the pixel point greater than or equal to the binarization threshold in the i-th fourth image can be set to 0, and the pixel value of the pixel point less than the binarization threshold in the i-th fourth image can be set to 1; in this way, the pixel points included in the target object in the obtained binarization image are white, and other pixel points are black.

[0353] S606, the central processing module determines whether the target object exists in the i-th fourth image based on the i-th binarization image.

[0354] Exemplarily, the central processing module can determine whether the area of the connected region composed of the target pixel points in the binarization image is greater than an area threshold.

[0355] In one possible manner, when the area of the connected region composed of the target pixel points in the binarization image is greater than the area threshold, it can be determined that the target object exists in the i-th fourth image, and S608 can be executed. When the area of the connected region composed of the target pixel points in the binarization image is less than or equal to the area threshold, it can be determined that the target object does not exist in the i-th fourth image, and S607 can be executed.

[0356] In a possible manner, when the area of the connected region composed of the target pixels in the binary image is greater than the area threshold, and the shape of the connected region composed of the target pixels in the binary image is similar to the shape of a human body (for example, the aspect ratio of the connected region composed of the target pixels in the binary image can be calculated; if the aspect ratio belongs to a preset range, it is determined that the shape of the connected region composed of the target pixels in the binary image is similar to the shape of a human body), it can be determined that the target object exists in the i th fourth image, and S608 can be performed. When the area of the connected region composed of the target pixels in the binary image is less than or equal to the area threshold, or the shape of the connected region composed of the target pixels in the binary image is not similar to the shape of a human body, it can be determined that the target object does not exist in the i th fourth image, and S607 can be performed.

[0357] It should be noted that the area threshold involved in S606 can be the same as or different from the area threshold involved in S507, and the present application does not limit this.

[0358] In the i th binary image, the pixel value of the target pixel is less than the binary threshold.

[0359] S607, the central processing module adds 1 to i.

[0360] Exemplarily, after S607 is performed, S603 can be returned to be performed.

[0361] S608, the central processing module determines the region description information of the region where the target object is located based on the region description information of the connected region composed of the target pixels in the i th binary image.

[0362] In a possible manner, the region description information of the connected region composed of the target pixels in the i th binary image can be taken as the region description information of the region where the target object is located in the i th fourth image. In this case, the region description information of the region where the target object is located in the i th fourth image is the contour pixel coordinates of the region covered by the pixels included in the target object in the i th fourth image.

[0363] In a possible manner, the region description information of the circumscribed frame of the connected region composed of the target pixel points in the i-th binary image can be taken as the region description information of the region where the target object is located in the i-th fourth image. In a possible case, the region description information of the connected region composed of the target pixel points in the i-th binary image can include the upper-left corner coordinate of the circumscribed frame and the lower-right corner coordinate of the circumscribed frame. In a possible case, the region description information of the connected region composed of the target pixel points in the i-th binary image can include the upper-left corner coordinate of the circumscribed frame and the width and height of the circumscribed frame. In a possible case, the region description information of the connected region composed of the target pixel points in the i-th binary image can include the center coordinate of the circumscribed frame and the width and height of the circumscribed frame.

[0364] When k is 1, the motion speed of the target object in the i-th fourth image is 0. When k is not equal to 1, the motion speed of the target object in the i-th fourth image can be determined according to the position of the target object in the i-th fourth image and the position of the target object in the (i-1)-th fourth image; for details, refer to the description in S406 above, which will not be repeated here.

[0365] S609, the central processing module obtains the region description information of the region where the projection image is located in the i-th fourth image.

[0366] It should be understood that the present application does not limit the execution order of S608 and S609.

[0367] S610, it is judged whether i is equal to 1.

[0368] If i is equal to 1, S607 can be returned. If i is not equal to 1, S611 can be executed.

[0369] S611, the central processing module judges whether the second preset condition is met based on the region description information of the region where the target object is located in one or more fourth images and the region description information of the region where the projection image is located in the i-th fourth image.

[0370] When the second preset condition is met, S612 can be executed. When the second preset condition is not met, S607 can be returned.

[0371] S612, the central processing module judges whether the projection image is occluded by the target object based on the region description information of the region where the target object is located in the i-th fourth image and the region description information of the region where the projection image is located in the i-th fourth image.

[0372] When it is determined that the region where the target object is located in the i-th fourth image overlaps with the region where the projection image is located, it is determined that the projection image is occluded by the target object, and S613 can be performed. When it is determined that the region where the target object is located in the i-th fourth image does not overlap with the region where the projection image is located, it is determined that the projection image is not occluded by the target object, and S607 can be returned to be performed.

[0373] S613, the central processing module determines a first region where the projection image is occluded and a second region in the occlusion source image based on the region description information of the region where the target object is located in the i-th fourth image and the region description information of the region where the projection image is located in the i-th fourth image.

[0374] S614, the central processing module reads a first image from the storage module to obtain a k-th fifth image, and the initial value of k is 1.

[0375] S615, the central processing module determines a depth map of the k-th fifth image by using a depth estimation algorithm.

[0376] S616, the central processing module binarizes the depth map of the k-th fifth image based on a binarization threshold to obtain a k-th binarization image.

[0377] S617, the central processing module determines whether the target object exists in the k-th fifth image based on the k-th binarization image.

[0378] Exemplarily, S615-S617 can refer to the description of S604-S606, and will not be described here.

[0379] When it is determined that the target object exists in the k-th fifth image, the central processing module 25 can perform S618. When it is determined that the target object does not exist in the k-th fifth image, the central processing module 25 can perform S627.

[0380] S618, the central processing module obtains the region description information of the region where the projection image is located in the k-th fifth image.

[0381] S619, the central processing module determines the region description information of the region where the target object is located based on the region description information of the connected region composed of the target pixel points in the k-th binarization image.

[0382] It should be understood that the present application does not limit the execution order of S618 and S619.

[0383] S620, the central processing module determines the difference information between the region description information of the region where the target object is located in the k-th fifth image and the region description information of the region where the target object is located in the k-1-th fifth image.

[0384] S621, the central processing module determines whether the difference information satisfies a first preset condition.

[0385] When the difference information satisfies the first preset condition, S622 can be executed; when the difference information does not satisfy the first preset condition, S623 can be executed.

[0386] S622, the central processing module determines whether the projection image is occluded by the target object based on the region description information of the region where the target object is located in the kth fifth image and the region description information of the region where the projection image is located in the kth fifth image.

[0387] When it is determined that the projection image is occluded by the target object, S625 can be executed; when it is determined that the projection image is not occluded by the target object, S628 can be executed.

[0388] S623, the central processing module determines the predicted region description information of the region of the target object in the kth fifth image based on the motion state information of the target object in the k-1th fifth image.

[0389] S624, the central processing module determines whether the projection image is occluded by the target object based on the predicted region description information of the region of the target object in the kth fifth image and the region description information of the region where the projection image is located in the kth fifth image.

[0390] When it is determined that the projection image is occluded by the target object, S625 can be executed; when it is determined that the projection image is not occluded by the target object, S628 can be executed.

[0391] S625, the central processing module determines a first region where the projection image is occluded and a second region in the occlusion source image.

[0392] After S625 is executed, S626 can be executed.

[0393] S626, the central processing module adds 1 to k.

[0394] After S626 is executed, S614 can be executed.

[0395] S627, the central processing module determines whether the target object in the k-1th fifth image is located at the edge of the region where the projection image is located in the k-1th fifth image.

[0396] If it is determined that the target object in the k-1th fifth image is not located at the edge of the region where the projection image is located in the k-1th fifth image, it means that it is unreasonable that the kth fifth image does not detect the target object, that is, the kth fifth image misses the detection of the target object, and at this time, S623 can be executed.

[0397] If it is determined that the target object in the k-1th fifth image is located at the edge of the region where the projection image is located in the k-1th fifth image, it is reasonable that the target object is not detected in the kth fifth image, and S628 can be executed at this time.

[0398] S628, the central processing module sets i to 1 and k to 1.

[0399] Exemplarily, S618-S628 can refer to the description of S414-S422 described above, and will not be repeated here.

[0400] After S628 is executed, S603 can be returned to monitor whether eye protection processing is needed again.

[0401] FIGS. 7A and 7B are schematic diagrams of another image processing process 700 according to an embodiment of the present application. The image processing process 700 is based on the image processing process 300, and describes an implementation manner of judging whether the projection image is blocked by the target object based on the first image captured by the CMOS component.

[0402] S701, the control module switches the filter corresponding to the lens position in the CMOS module to a narrow-band filter, turns on the infrared fill light, and instructs the CMOS module to capture.

[0403] Exemplarily, S701 can refer to the description of S501 described above, and will not be repeated here.

[0404] S702, the CMOS module captures to obtain a first image and store the first image into the storage module.

[0405] S703, the central processing module reads a first image from the storage module to obtain the ith fourth image, and the initial value of i is 1.

[0406] S704, the central processing module judges whether there is a target object in the ith fourth image.

[0407] Exemplarily, when it is determined that there is a target object in the ith fourth image, S706 can be executed; when it is determined that there is no target object in the ith fourth image, S705 can be executed.

[0408] Exemplarily, S702-S704 can refer to the description of S402-S404 described above, and will not be repeated here.

[0409] S705, the central processing module adds 1 to i.

[0410] Exemplarily, after S705 is executed, S703 can be returned to be executed, that is, the central processing module 25 can read the next first image from the storage module.

[0411] S706, the central processing module determines the motion state information of the target object in the i-th fourth image based on the detection result obtained by performing human segmentation on the i-th fourth image using the human segmentation algorithm.

[0412] Illustratively, the detection result obtained by human segmentation includes the contour pixel point coordinates of the region covered by the pixel points included in the target object.

[0413] Illustratively, the result obtained by performing human segmentation on the i-th first image can be used as the region description information of the region in which the target object is located in the i-th first image.

[0414] Illustratively, when i = 1, i.e., the i-th fourth image is the first fourth image obtained by the central processing module 25 from the storage module 24, the motion speed of the target object in the i-th fourth image is 0. When i is greater than 1, i.e., the i-th fourth image is not the first fourth image obtained by the central processing module 25 from the storage module 24, the motion speed of the target object in the i-th fourth image can be determined according to the position of the target object in the i-th fourth image and the position of the target object in the (i-1)-th fourth image. The position of the target object in the fourth image can be represented by the center coordinates determined by the contour pixel point coordinates of the region covered by the pixel points included in the target object, and the motion speed v of the target object in the i-th fourth image is equal to the horizontal coordinate of the center coordinates corresponding to the target object in the i-th fourth image minus the horizontal coordinate of the center coordinates corresponding to the target object in the (i-1)-th fourth image.

[0415] S707, the central processing module obtains the region description information of the region in which the projection image is located in the i-th fourth image.

[0416] Illustratively, the present application does not limit the execution order of S706 and S707.

[0417] S708, the central processing module determines whether i is equal to 1.

[0418] If i is equal to 1, the execution of S705 can be returned. If i is not equal to 1, S709 can be executed.

[0419] S709, the central processing module determines whether the second preset condition is met based on the region description information of the region in which the target object is located in one or more fourth images and the region description information of the region in which the projection image is located in the i-th fourth image.

[0420] When it is determined that the second preset condition is met, S710 can be executed. When it is determined that the second preset condition is not met, the execution of S705 can be returned.

[0421] S710, the central processing module determines whether the projection image is occluded by the target object based on the region description information of the region where the target object is located in the i-th fourth image and the region description information of the region where the projection image is located in the i-th fourth image.

[0422] When it is determined that the projection image is occluded by the target object, S711 can be performed. When it is determined that the projection image is not occluded by the target object, S705 can be returned to perform.

[0423] S711, the central processing module determines the first region where the projection image is occluded and the second region in the occlusion source image based on the region description information of the region where the target object is located in the i-th fourth image and the region description information of the region where the projection image is located in the i-th fourth image.

[0424] S712, the central processing module reads a first image from the storage module to obtain a k-th fifth image, and the initial value of k is 1.

[0425] S713, the central processing module determines whether the target object exists in the k-th fifth image.

[0426] When it is determined that the target object exists in the k-th fifth image, the central processing module 25 can perform S714. When it is determined that the target object does not exist in the k-th fifth image, the central processing module 25 can perform S723.

[0427] S714, the central processing module obtains the region description information of the region where the projection image is located in the k-th fifth image.

[0428] S715, the central processing module determines the motion state information of the target object in the k-th fifth image based on the detection result obtained by performing human body segmentation on the k-th fifth image using a human body segmentation algorithm.

[0429] It should be understood that the present application does not limit the execution order of S714 and S715.

[0430] S716, the central processing module determines the difference information between the region description information of the region where the target object is located in the k-th fifth image and the region description information of the region where the target object is located in the k-1-th fifth image.

[0431] S717, the central processing module determines whether the difference information satisfies a first preset condition.

[0432] When the difference information satisfies the first preset condition, S718 can be performed; when the difference information does not satisfy the first preset condition, S719 is performed.

[0433] S718, the central processing module determines whether the projection image is occluded by the target object based on the region description information of the region where the target object is located in the kth fifth image and the region description information of the region where the projection image is located in the kth fifth image.

[0434] When it is determined that the projection image is occluded by the target object, S721 can be performed; when it is determined that the projection image is not occluded by the target object, S724 can be performed.

[0435] S719, the central processing module determines the predicted region description information of the region of the target object in the kth fifth image based on the motion state information of the target object in the k-1th fifth image.

[0436] S720, the central processing module determines whether the projection image is occluded by the target object based on the predicted region description information of the region of the target object in the kth fifth image and the region description information of the region where the projection image is located in the kth fifth image.

[0437] When it is determined that the projection image is occluded by the target object, S721 can be performed; when it is determined that the projection image is not occluded by the target object, S724 can be performed.

[0438] S721, the central processing module determines the first region where the projection image is occluded and the second region in the occlusion source image.

[0439] After S721 is performed, S722 can be performed.

[0440] S722, the central processing module adds 1 to k.

[0441] After S722 is performed, S712 can be returned to be performed.

[0442] S723, the central processing module determines whether the target object in the k-1th fifth image is located at the edge of the region where the projection image is located in the k-1th fifth image.

[0443] If it is determined that the target object in the k-1th fifth image is not located at the edge of the region where the projection image is located in the k-1th fifth image, it means that it is unreasonable that the target object is not detected in the kth fifth image, that is, the target object is missed in the kth fifth image, and S719 can be performed at this time.

[0444] If it is determined that the target object in the k-1th fifth image is located at the edge of the region where the projection image is located in the k-1th fifth image, it means that it is reasonable that the target object is not detected in the kth fifth image, and S724 can be performed at this time.

[0445] S724, the central processing module sets i to 1 and sets k to 1.

[0446] After S724 is executed, S703 can be returned to monitor whether the eye protection process is needed again.

[0447] Exemplarily, S707-S724 can refer to the description of S407-S424, which will not be repeated here.

[0448] It should be noted that the human body segmentation algorithm in the graphic processing process 700 can also be replaced by the human body detection algorithm described above.

[0449] The embodiment of the application also provides an image processing device, which comprises a processing module and a complementary metal oxide semiconductor (CMOS) component, the CMOS component comprising a CMOS module and an infrared light supplement lamp, and the light filtering structure in the CMOS module allows infrared light to pass through.

[0450] The CMOS module is configured to capture an image, the image comprising a first image captured after the infrared light supplement lamp is turned on and towards a projection medium, the projection medium displaying a projection image.

[0451] The processing module is configured to determine, based on the first image, whether the projection image is blocked by a target object, and when it is determined that the projection image is blocked by the target object, determine a first area of the projection image that is blocked and a second area in a source image, the second area being a corresponding area of the first area in the source image.

[0452] The processing module in the image processing device can be the central processing module 25 described above.

[0453] FIG. 8 is a schematic diagram of an image processing device 800 according to an embodiment of the application. The image processing device 800 can be used to execute the method of the above-mentioned embodiments, and thus the beneficial effects achieved by the image processing device 800 can refer to the beneficial effects of the corresponding method provided above, which will not be repeated here.

[0454] Exemplarily, the image processing device 800 can comprise:

[0455] An information acquisition module 801 is configured to acquire a first image, the first image being obtained by capturing the projection medium after the infrared light supplement lamp is turned on, the projection medium displaying a projection image.

[0456] An occlusion determination module 802 is configured to determine, based on the first image, whether the projection image is blocked by a target object.

[0457] The eye protection processing module 803 is configured to, when it is determined that the projection image is blocked by the target object, determine a first area of the projection image that is blocked and a second area in the source image corresponding to the first area in the projection image.

[0458] It should be noted that the information acquisition module 801, the blocking judgment module 802, and the eye protection processing module 803 belong to the central processing module 25.

[0459] The projection device provided in the present application comprises a memory, a processor, a complementary metal oxide semiconductor (CMOS) component, and a projection lens component. The CMOS component comprises a CMOS module and an infrared light supplement lamp. The light filtering component of the CMOS module allows infrared light to pass through. The memory is coupled to the processor. The memory stores program instructions. When the program instructions are executed by the processor, the projection device performs the following steps:

[0460] acquiring a first image, the first image being captured by the CMOS component after the infrared light supplement lamp is turned on and facing the projection medium, the projection medium displaying a projection image;

[0461] judging whether the projection image is blocked by a target object based on the first image;

[0462] when it is determined that the projection image is blocked by the target object, determining a first area of the projection image that is blocked and a second area in the source image corresponding to the first area in the projection image.

[0463] The memory is an example of the storage module 24, and the processor is an example of the central processing module 25.

[0464] Auto-focusing function

[0465] FIG. 9A is a schematic diagram of an auto-focusing process 900 according to an embodiment of the present application.

[0466] S901, the control module switches the light filtering component corresponding to the lens position in the CMOS module to an IR-CUT / all-pass light filtering component, and turns on the infrared light supplement lamp.

[0467] Exemplarily, in the process of implementing the auto-focusing function, the reason for using the IR-CUT or all-pass light filtering component is that the content of the projection image needs to be clearly captured in the auto-focusing process.

[0468] The IR-CUT can block near-infrared light, prevent the CMOS sensor from being too sensitive to red light, and avoid color cast.

[0469] S902, the projection module uses different projection focal lengths to project a preset focusing image to the projection medium.

[0470] In order to realize the focusing function, the projection device 110 needs to accurately analyze the definition of the projection image content. To achieve this goal, a preset focusing picture with high contrast and definition and clear texture and detail information is usually projected by the projection module 26. An example of the preset focusing picture is shown in FIG. 9B.

[0471] Exemplarily, the control module 22 can drive the stepper motor to adjust the focal length of the lens in the projection lens assembly (i.e., the projection focal length) to project the projection image corresponding to different focal lengths to the projection medium.

[0472] S903, the CMOS module takes a picture to obtain a plurality of sixth images and store them in the storage module.

[0473] Exemplarily, the control module 22 drives the stepper motor to adjust the focal length of the lens in the projection lens assembly, and instructs the CMOS module to take pictures of the projection images corresponding to different projection focal lengths to obtain a plurality of sixth images and save the sixth images and their corresponding projection focal lengths in the storage module 24.

[0474] S904, the central processing module reads the plurality of sixth images from the storage module.

[0475] S905, the central processing module determines the definition of each sixth image in the plurality of sixth images.

[0476] Exemplarily, the definition of each sixth image can be represented by at least one of the following: contrast, image entropy, histogram, gradient intensity, etc. Then, the clearest sixth image and its corresponding projection focal length are saved in the storage module 24.

[0477] S906, the control module sets the focal length of the lens in the projection lens assembly to the projection focal length corresponding to the clearest sixth image.

[0478] Exemplarily, the control module 22 adjusts the focal length of the lens in the projection lens assembly to the projection focal length corresponding to the clearest sixth image. The subsequent projection module maps the image to be projected onto the projection medium through the projection lens assembly with the adjusted projection focal length to complete the automatic focusing.

[0479] Automatic trapezoidal correction function

[0480] FIG. 10A is a schematic diagram of an automatic trapezoidal correction process 1000 according to an embodiment of the present application.

[0481] S1001, the control module switches the filter corresponding to the lens position in the CMOS module to an IR-CUT / all-pass filter and turns on the infrared fill light.

[0482] S1002, the projection module projects a preset trapezoidal correction image to the projection medium.

[0483] In order to realize the trapezoidal correction function, the projection device 110 needs to detect the coordinates of the four vertices of the projection image. To achieve this goal, a preset trapezoidal correction image with clear edge information is usually projected by the projection module. An example of the preset trapezoidal correction image can be shown in FIG. 10B. The preset trapezoidal correction image in FIG. 10B has clear edge information, so it is easy to extract the coordinates of the four vertices of the projection image.

[0484] S1003, the CMOS module takes a picture to obtain a seventh image and store it in the storage module.

[0485] S1004, the central processing module reads the seventh image from the storage module.

[0486] S1005, the central processing module detects the vertex coordinates of the projection image in the seventh image.

[0487] For example, the central processing module can use corner detection algorithms (such as Harris corner detection), edge detection algorithms (such as Canny, Sobel operator), etc. to find the clear edges of the projection image. Then, through area, angle and other rules, a quadrilateral region is screened to obtain the four vertex coordinates of the projection image in the seventh image.

[0488] S1006, the central processing module performs perspective transformation on the vertex coordinates of the projection image in the seventh image to correct the positions of the four vertices of the projection image.

[0489] For example, after the central processing module 25 obtains the four vertex coordinates of the projection region in the seventh image, it can calculate the homography matrix. Then, based on the perspective transformation and the homography matrix, the trapezoidal region is transformed into a rectangular region, thereby realizing automatic trapezoidal correction.

[0490] FIG. 10C is a schematic diagram of a trapezoidal correction process according to an embodiment of the present application. In FIG. 10B, the dashed area is the projection image before trapezoidal correction, and the rectangular area is the projection image after trapezoidal correction. For example, the coordinates of the corrected projection image can be saved in the storage module for use in implementing other perception functions.

[0491] Ambient light adaptive function

[0492] FIG. 11A is a schematic diagram of an ambient light adaptive process 1100 according to an embodiment of the present application.

[0493] S1101, the control module switches the filter corresponding to the lens position in the CMOS module to an IR-CUT / all-pass filter and turns on the infrared fill light.

[0494] S1102, the projection module projects a preset ambient light adaptive image to the projection medium.

[0495] In order to realize the ambient light adaptive function, the projection device 110 needs to accurately estimate the brightness of the ambient light. To achieve this goal, a preset ambient light adaptive image with high contrast and significant light and dark areas can be projected by the projection module. An example of the preset ambient light adaptive image can be shown in FIG. 11B.

[0496] S1103, the CMOS module takes a picture to obtain an eighth image and store it in the storage module.

[0497] S1104, the central processing module reads the eighth image from the storage module.

[0498] S1105, the central processing module estimates the current ambient brightness according to the eighth image.

[0499] For example, the central processing module 25 can calculate image features such as contrast and grayscale value distribution of the eighth image; then, by comparing the image features of the eighth image with the image features of the source image corresponding to the eighth image, the brightness of the current environment can be estimated; and the current ambient brightness can be saved in the storage module 24.

[0500] S1106, the projection module adjusts the projection parameters according to the current ambient brightness.

[0501] For example, the projection module can adjust the projection parameters according to the current ambient brightness; for example, the projection parameters can include exposure, brightness, contrast, etc. For example, if the current ambient brightness is dark, the exposure or brightness of the projected image can be increased.

[0502] That is, the present application can realize the ambient light adaptive function based on the CMOS component without relying on the ALS; in this way, the ALS can not be carried in the projection device, thereby reducing the cost.

[0503] Automatic obstacle avoidance function

[0504] FIG. 12 is a schematic diagram of an automatic obstacle avoidance process 1200 according to an embodiment of the present application.

[0505] S1201, the control module switches the filter corresponding to the lens position in the CMOS module to an IR-CUT / through filter and turns on the infrared fill light.

[0506] S1202, the projection module projects a preset white background image to the projection medium.

[0507] S1203, the CMOS module takes a picture to obtain a ninth image and store it in the storage module.

[0508] S1204, the central processing module reads the ninth image from the storage module.

[0509] S1205, the central processing module detects the position and shape information of the obstacle based on the ninth image.

[0510] Exemplarily, the central processing module 25 can use edge detection, contour detection or deep learning-based method to detect the obstacle within the projection image, and save the position and shape information of the obstacle in the storage module 24.

[0511] S1206, the central processing module adjusts the position and shape of the source image according to the position and shape information of the obstacle.

[0512] For example, the source image can be scaled with the center point of the source image as the reference point until the projection image does not contain the obstacle.

[0513] Automatic curtain entering function

[0514] FIG. 13A is a schematic diagram of an automatic curtain entering process 1300 according to an embodiment of the present application.

[0515] S1301, the control module switches the filter corresponding to the lens position in the CMOS module to an IR-CUT / ALL-PASS filter, and turns on the infrared fill light.

[0516] S1302, the projection module projects a preset white background image to the projection medium.

[0517] S1303, the CMOS module takes a picture to obtain a tenth image and store it in the storage module.

[0518] S1304, the central processing module reads the tenth image from the storage module.

[0519] S1305, the central processing module detects the curtain position based on the tenth image.

[0520] Exemplarily, the central processing module 25 can determine the four vertex coordinates of the curtain by processing the tenth image. This step requires that the preset white background image encloses the curtain area. If the boundary of the curtain is black, better detection effect can be obtained. A typical curtain is shown in FIG. 13B, which has a clear boundary.

[0521] Exemplarily, the central processing module 25 can use corner detection algorithm (such as Harris corner detection), edge detection algorithm (such as Canny, Sobel operator) to detect the tenth image, find the clear edge; and then filter the quadrilateral area through area, angle and other rules, so as to obtain the four vertex coordinates of the curtain.

[0522] S1306, the central processing module adjusts the position and shape of the source image according to the curtain position.

[0523] Exemplarily, the central processing module 25 can perform perspective transformation on the source image to transform the projection image into the curtain range.

[0524] It should be understood that when the projection medium is not a curtain or a hard screen, the projection device 110 can not need to perform the automatic curtain entering processing.

[0525] In one example, FIG. 14 shows a schematic block diagram of an apparatus 1400 according to an embodiment of the present application. The apparatus 1400 can include a processor 1401 and a transceiver 1402, and optionally further include a memory 1403, a projection lens assembly and a CMOS assembly.

[0526] The various components of the apparatus 1400 are coupled together by a bus 1404, which can include a data bus, a power bus, a control bus, and a state signal bus. For the sake of clarity, the various buses are illustrated in FIG. 14 as the bus 1404, although the different buses can be implemented separately.

[0527] Optionally, the memory 1403 can be used to store instructions of the foregoing method embodiments. The processor 1401 can be used to execute the instructions in the memory 1403, and control the transceiver 1402 to receive signals and control the transceiver 1402 to send signals.

[0528] The apparatus 1400 can be a projection device or a chip of a projection device in the foregoing method embodiments.

[0529] All relevant content of the steps involved in the foregoing method embodiments can be cited from the function description of the corresponding functional modules, which will not be repeated here.

[0530] The embodiments of the present application also provide a chip, which includes one or more interface circuits and one or more processors; the one or more processors receive or send data through the one or more interface circuits; when the one or more processors execute computer instructions, the steps of the foregoing related methods are executed to implement the steps of the methods in the foregoing embodiments. The interface circuit is the transceiver 1402.

[0531] The embodiment further provides a computer readable storage medium, wherein computer instructions are stored in the computer readable storage medium, and when the computer instructions are executed on a projection device, the projection device executes the related method steps to implement the method in the above embodiment. Exemplarily, the computer readable storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage medium capable of storing program codes.

[0532] The embodiment further provides a computer program product, which contains computer instructions, and when the computer instructions are executed by a computer or a processor, the computer executes the related steps to implement the method in the above embodiment. Exemplarily, the computer program product can be stored in a random access memory (RAM), a flash memory, a ROM, an erasable programmable ROM (EPROM), an electrically EPROM (EEPROM), a register, a hard disk, a mobile hard disk, a compact disc read-only memory (CD-ROM), or any other form of storage medium well known in the art.

[0533] The projection device, the computer readable storage medium, the computer program product or the chip provided in the embodiment are used to execute the corresponding method provided above, and thus the beneficial effects achieved by the projection device, the computer readable storage medium, the computer program product or the chip can refer to the beneficial effects of the corresponding method provided above, which will not be described herein again.

[0534] Through the above description of the implementation mode, those skilled in the art can understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example for illustration, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0535] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, may be located in one place, or may be distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0536] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0537] Any content of each embodiment of the present application, and any content of the same embodiment, can be freely combined. Any combination of the above is within the scope of the present application.

[0538] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the specific embodiments described above, which are merely illustrative and not restrictive, and a person of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims.

Claims

An image processing apparatus characterized by comprising: The image processing device comprises a processing module and a complementary metal oxide semiconductor (CMOS) component, the CMOS component comprises a CMOS module and an infrared light supplement lamp, and a light filtering structure in the CMOS module allows infrared light to pass through; The CMOS module is configured to capture an image, and the image comprises a first image captured towards a projection medium after the infrared light supplement lamp is turned on, and the projection medium displays a projection image. The processing module is configured to determine, based on the first image, whether the projection image is blocked by a target object; when it is determined that the projection image is blocked by the target object, determine a first area of the projection image that is blocked and a second area in a source image, the second area being a corresponding area of the first area in the projection image in the source image. The apparatus of claim 1, wherein The CMOS module comprises one camera, and the camera comprises one lens, one CMOS sensor, and one or more light filtering structures. The apparatus of claim 1, wherein The CMOS module comprises a plurality of cameras, each camera comprising one lens, one CMOS sensor, and one light filtering structure, and the light filtering structures of any two cameras are different. The apparatus of claim 1, wherein The CMOS module comprises a first camera and a second camera, the first camera comprises one lens, one CMOS sensor, and a plurality of light filtering structures, and the second camera comprises one lens, one CMOS sensor, and one light filtering structure. The apparatus according to any one of claims 1 to 4, characterized in that The light filtering structure comprises an all-pass light filtering structure or a narrow-band light filtering structure. The apparatus according to any one of claims 1 to 5, characterized in that The processing module is configured to: determine, based on the first image, area description information of an area in which a target object is located in the first image; obtain area description information of an area in which a projection image is located in the first image; based on the area description information of the area in which the target object is located in the first image and the area description information of the area in which the projection image is located in the first image, determine whether the projection image is blocked by the target object. The apparatus of claim 6, wherein The processing module is further configured to: determine difference information between area description information of an area in which a target object is located in a second image and area description information of an area in which a target object is located in a third image, the second image being a first image captured this time, and the third image being a first image captured last time; when the difference information satisfies a first preset condition, determine, based on the area description information of the area in which the target object is located in the second image and the area description information of the area in which the projection image is located in the second image, whether the projection image is blocked by the target object. The apparatus of claim 7, wherein The processing module is further configured to: when the difference information does not satisfy the first preset condition, obtain motion speed information of the target object in the third image; based on the motion speed information of the target object in the third image and the area description information of the area in which the target object is located in the third image, determine predicted area description information of an area of the target object in the second image; based on the predicted area description information of the area of the target object in the second image and the area description information of the area in which the projection image is located in the second image, determine whether the projection image is blocked by the target object. The apparatus of claim 6, wherein The processing module is used for: adopting any one of a human body detection algorithm, a human body segmentation algorithm or a depth estimation algorithm to process the first image to obtain region description information of a region where a target object is located in the first image; based on the region description information of the region where the target object is located in the first image, determining whether a second preset condition is met; when the second preset condition is met, based on the region description information of the region where the target object is located in the first image and the region description information of the region where the projection image is located in the first image, determining whether the projection image is occluded by the target object. The apparatus of claim 9, wherein The second preset condition includes that the region where the target object is located in the first image in which the target object exists for the first time does not overlap with the region where the projection image is located. The apparatus of claim 6, wherein The processing module is used for: determining the similarity between each image block in a preset region in the first image and an image block at a corresponding position in a preset region in a reference image; based on the region description information of a connected region composed of target image blocks in the first image, determining the region description information of the region where the target object is located in the first image, the similarity between the target image blocks in the first image and the image blocks at the corresponding positions in the reference image being less than a similarity threshold. An image processing method, characterized by, The method comprises: obtaining a first image, the first image being obtained by shooting towards a projection medium after an infrared fill light is turned on, the projection medium displaying a projection image; based on the first image, determining whether the projection image is occluded by a target object; when it is determined that the projection image is occluded by the target object, determining a first region where the projection image is occluded and a second region in a source image, the second region being a region corresponding to the first region in the projection image in the source image. The method of claim 12, wherein The method further comprises: based on the first image, determining the region description information of the region where the target object is located in the first image; obtaining the region description information of the region where the projection image is located in the first image; based on the region description information of the region where the target object is located in the first image and the region description information of the region where the projection image is located in the first image, determining whether the projection image is occluded by the target object. The method of claim 13, wherein The method further comprises: determining difference information between the region description information of the region where the target object is located in the second image and the region description information of the region where the target object is located in the third image, the second image being the first image obtained this time, and the third image being the first image obtained last time; The method further comprises: when the difference information meets a first preset condition, based on the region description information of the region where the target object is located in the second image and the region description information of the region where the projection image is located in the second image, determining whether the projection image is occluded by the target object. The method of claim 14, wherein The method further includes: When the difference information does not satisfy the first preset condition, obtaining motion speed information of the target object in the third image; Based on the motion speed information of the target object in the third image and the region description information of the region where the target object is located in the third image, determining predicted region description information of the region of the target object in the second image in the third image; Based on the predicted region description information of the region of the target object in the second image in the third image and the region description information of the region where the projection image is located in the second image, determining whether the projection image is occluded by the target object. The method of claim 13, wherein The method further includes: processing the first image by using any one of a human body detection algorithm, a human body segmentation algorithm or a depth estimation algorithm to obtain the region description information of the region where the target object is located in the first image; The method further includes: Based on the region description information of the region where the target object is located in the first image, determining whether a second preset condition is satisfied; When the second preset condition is satisfied, determining whether the projection image is occluded by the target object based on the region description information of the region where the target object is located in the first image and the region description information of the region where the projection image is located in the first image. The method of claim 16, wherein The second preset condition includes that the region where the target object is located in the first image in which the target object exists for the first time does not overlap with the region where the projection image is located. The method of claim 13, wherein The method further includes: determining the similarity between each image block in a preset region in the first image and a corresponding position image block in a preset region in a reference image; Based on the region description information of a connected region composed of the target image block in the first image, determining the region description information of the region where the target object is located in the first image, the similarity between the target image block in the first image and the image block at the corresponding position in the reference image being less than a similarity threshold. The method according to claim 16 or 17, characterized in that The method further includes: processing the first image by using the depth estimation algorithm to obtain the region description information of the region where the target object is located in the first image, including: determining a depth map of the first image by using the depth estimation algorithm; binarizing the depth map of the image based on a binarization threshold to obtain a binarization image; based on the region description information of a connected region composed of a target pixel point in the binarization image, determining the region description information of the region where the target object is located, the pixel value of the target pixel point in the binarization image being less than the binarization threshold. A projection device, characterized by The projection device comprises a processing module, a complementary metal oxide semiconductor (CMOS) component and a projection lens component, the CMOS component comprises a CMOS module and an infrared light supplement lamp, and a light filtering assembly of the CMOS module allows infrared light to pass through; The projection lens component is configured to project a first source image to a projection medium to display a first projection image on the projection medium; The CMOS module is configured to capture an image, and the image comprises a first image captured towards the projection medium after the infrared light supplement lamp is turned on; The processing module is configured to determine whether the first projection image is blocked by a target object based on the first image; When it is determined that the first projection image is blocked by the target object, a first area of the first projection image that is blocked and a second area in a second source image that is blocked are determined, and the second area is a corresponding area of the first area in the second source image; The projection lens component is further configured to project the second source image of the blocked second area to the projection medium to display a second projection image on the projection medium, and a third area in the second projection image is blocked, and the third area is the same as the first area. A projection device, characterized by The projection device comprises a memory, a processor, a complementary metal oxide semiconductor (CMOS) component and a projection lens component, the CMOS component comprises a CMOS module and an infrared light supplement lamp, and a light filtering assembly of the CMOS module allows infrared light to pass through, and the memory is coupled with the processor; the memory stores program instructions, and when the program instructions are executed by the processor, the projection device performs the following steps: An image is acquired, and the image is captured by the CMOS component towards a projection medium after the infrared light supplement lamp is turned on, and the projection medium displays a projection image; It is determined whether the projection image is blocked by a target object based on the first image; When it is determined that the projection image is blocked by the target object, a first area of the projection image that is blocked and a second area in a source image that is blocked are determined, and the second area is a corresponding area of the first area in the source image. A chip characterized by The one or more interfaces receive or send data, and when the one or more processors execute computer instructions, the steps of the method in any one of claims 12 to 19 are performed. A computer-readable storage medium, characterized by The computer readable storage medium stores a computer program, and when the computer program runs on a computer or a processor, the computer or the processor executes the method in any one of claims 12 to 19. A computer program product, characterized in that The computer program product comprises computer instructions, and when the computer instructions are executed by a computer or a processor, the steps of the method in any one of claims 12 to 19 are performed.

Images