Projector non-inductive automatic focusing method based on camera

Abstract

The invention discloses a projector non-inductive automatic focusing method based on a camera, and the method comprises the following steps: S1, selecting a projection region in a video image shot by the camera; s2, dividing into n * n sub-regions; s3, recognizing a dynamic picture and a static picture in the n * n sub-region through a motion detection algorithm; s4, distinguishing the static picture and the changing picture of the video content in the n * n sub-regions in the S3, abandoning the first sub-region of the changing picture in the video content, and retaining the second sub-region of the static picture in the video content; s5, further extracting video picture texture information with rich texture image-text pictures according to the second sub-region reserved in the step S4, and finding out a third sub-region with rich edge textures; and S6, performing lens focusing. According to the invention, lens focusing is realized through a method of dividing the sub-regions.

Description

Technical Field

[0001] The invention relates to a projector, and in particular to a camera-based projector non-sensing automatic focusing method. Background Art

[0002] The current projector AF (Auto Focus) solutions include camera and tof (tof is the abbreviation of Time of Flight technology, which plays a key role in projectors); the camera needs to project a feature map when performing AF; tof does not need to project a feature map, but single tof cannot realize the screen entry and obstacle avoidance functions.

[0003] If the AF solution is based on a single camera, the projected feature map will interrupt the viewing experience; although a single tof can achieve sensorless AF, it cannot perform entry-screen and obstacle avoidance; if both sensorless AF and entry-screen and obstacle avoidance functions are required, a camera must be combined with tof, which will increase costs. Summary of the invention

[0004] In view of the deficiencies in the prior art, the technical problem to be solved by the present invention is to provide a camera-based projector non-sensing automatic focusing method. The purposes of developing the projector non-sensing automatic focusing method are: first, to ensure that the content of the projected picture remains unchanged; second, to provide sufficiently rich texture information, based on which the changes in clarity are compared to achieve lens focusing.

[0005] To solve the above technical problems, the present invention is implemented by the following scheme: A projector non-sensing automatic focusing method based on a camera of the present invention comprises the following steps: S1, select the projection area in the video image captured by the camera; S2, divide the projection area selected by S1 into n×n sub-areas; S3, during the auto-focus process and the video playback process, the dynamic image and the static image in the n×n sub-area are identified by a motion detection algorithm; S4, distinguishing still images and changing images of the video content in the n×n sub-regions in S3, discarding the first sub-region with changing images in the video content, and retaining the second sub-region with still images in the video content; S5, further extracting video image texture information having rich texture graphics and pictures based on the second sub-region retained in S4, and finding a third sub-region having rich edge texture; S6, comparing the change in clarity by retaining the second sub-area and the third sub-area, and focusing the lens.

[0006] Furthermore, in S3, the motion detection algorithm includes one of a mixed Gaussian model and a frame difference method.

[0007] Furthermore, the frame difference method includes formula 1: ; Among them, the is the absolute value of the difference between two frames of images; Among them, in formula ②, The value is filtered. When the D value is greater than the threshold T, the D value is Remain unchanged when When the value is less than the threshold T, D is reset to zero; The frame difference result is obtained through calculation: when the picture change is caused by video playback, the frame difference result is different; When the picture change is caused by zoom, after obtaining the frame difference result, the number of changed pixels is counted.

[0008] Furthermore, the statistical method includes Formula 2: ; In formula ③, when the frame difference result is not 0, the count value Increase; In formula ④, R represents the ratio of the calculated changed pixels to the total number of pixels in the n×n sub-region. When the ratio is greater than the set threshold, it is determined that the video content of the sub-region has changed.

[0009] Furthermore, in S5, the method for extracting video picture texture information is: after excluding the first sub-region where the video picture changes and retaining the second sub-region where the video content is still, a third sub-region where the edge gradient of the video picture changes greatly is identified from the second sub-region.

[0010] Furthermore, the edge information of the third sub-area becomes blurred or clear during the autofocus process, and the transition zone formed by the change in edge clarity is captured by the motion detection algorithm, while the sub-area lacking edge information has a zero motion detection result even if a focusing action occurs; By scoring the clarity of the picture with insufficient edge information to obtain a score, the score cannot be used to make a difference during the focusing process, and thus the focusing direction cannot be determined. The third sub-region is further screened out to find the third sub-region with rich edge texture.

[0011] Furthermore, the third sub-area at least includes text, icons or a combination thereof.

[0012] Furthermore, the method for judging the edge richness of the third sub-region is to convert the spatial domain into the frequency domain by the Fourier transform method, and judge the richness of the edge of the picture by the frequency, retain the sub-region with high frequency, and eliminate the sub-region with low frequency.

[0013] Compared with the prior art, the beneficial effects of the present invention are: 1. The present invention uses a camera to achieve non-sensitive automatic focusing without projecting a feature map, thereby improving the user's viewing experience.

[0014] 2. The present invention extracts sub-images that can be used for AF clarity scoring from the constantly changing overall projection image through a method of dividing the sub-areas, distinguishes between static and changing sub-area video contents, and selects static sub-images with rich textures. The extracted third sub-area is used to compare the changes in clarity, thereby achieving lens focusing. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a principle and step diagram of the camera-based projector sensorless autofocus method of the present invention.

[0016] Figure 2 This is an example diagram of the camera screen and the projection screen of the present invention.

[0017] Figure 3 It is a schematic diagram of the present invention after frame difference result.

[0018] Figure 4 This is an example diagram of the sub-region 1 and the sub-region 2 after the video content is still and retained according to the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more explicit definition of the protection scope of the present invention. Obviously, the embodiments described in the present invention are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0020] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0021] Embodiment 1: The specific structure of the present invention is as follows: Please refer to the attached Figure 1-4 , a camera-based projector non-sensing automatic focusing method of the present invention comprises the following steps: S1, in the video image captured by the camera, select the projection area, such as Figure 2 The camera screen in .

[0022] S2, divides the projection area selected by S1 into n×n sub-areas, where n is at least 2, that is, n≥2. When n=2, the number of sub-areas is 4, when n=3, the number of sub-areas is 9, and when n=4, the number of sub-areas is 16. Figure 2 The 3×3 area separated by the middle dotted line contains two trees, mountains, and two figures. Since the image changes during viewing, it is unreliable to score the clarity based on the entire projected image; however, even during viewing, the audio and video image still has a static background, such as Figure 2 As shown, the mountains and trees are static, while the people and rivers are changing; by finding the static part in the divided picture, that part can be used to judge the clarity.

[0023] S3, during the autofocus process and the video playback process, the dynamic and static images in the n×n sub-area are identified through a motion detection algorithm; the motion detection algorithm includes a mixed Gaussian model and a frame difference method. This embodiment preferably takes the frame difference method as an example for explanation.

[0024] S4, distinguishing the still images and the changing images of the video content in the n×n sub-areas in S3, discarding the first sub-area of ​​the changed images in the video content, and retaining the second sub-area of ​​the still images in the video content; the first sub-area represents the changed sub-area, which may include multiple sub-areas; the second sub-area represents the still images in the video content, which may include multiple sub-areas.

[0025] S5, further extracting the video image texture information with rich texture graphics and pictures according to the second sub-region retained in S4, and finding the third sub-region with rich edge texture; the method for extracting the video image texture information is: after excluding the first sub-region with video image changes and retaining the second sub-region with static images in the video content, further identifying the third sub-region with large edge gradient changes in the video image from the second sub-region. The edge information of the third sub-region becomes blurred or clear during the automatic focusing process, and the transition zone formed by the change of edge clarity is captured by the motion detection algorithm, while the sub-region with insufficient edge information has a zero motion detection result even if the focusing action occurs; by scoring the image clarity with poor edge information to obtain a score, the score cannot be opened during the focusing process, and thus the focusing direction cannot be determined, and the third sub-region is further screened out to find the third sub-region with rich edge texture. The third sub-region contains at least one or a combination of text and icons. The method for judging the rich edge of the third sub-region is to convert the spatial domain into the frequency domain by the Fourier transform method, and judge the richness of the edge of the picture by the frequency, and the sub-region with high frequency is retained, and the sub-region with low frequency is eliminated.

[0026] S6, comparing the change in clarity by retaining the second sub-area and the third sub-area, and focusing the lens.

[0027] Embodiment 2: In step S3, the frame difference method includes formula 1: ; Among them, the is the absolute value of the difference between two frames of images; Among them, in formula ②, The value is filtered. When the D value is greater than the threshold T, the D value is Remain unchanged when When the value is less than the threshold T, D is reset to zero; The frame difference result is obtained through calculation: when the picture change is caused by video playback, the frame difference result is different; When the picture change is caused by zoom, after obtaining the frame difference result, the number of changed pixels is counted.

[0028] like Figure 3 As shown, Figure 3 The first frame in the video is a scene of a person bouncing a ball with a socket next to him. Figure 3 The two pictures in the second frame are operated by Android on the same screen. Through the frame difference method, also called inter-frame difference, two sets of pictures are obtained. The upper right picture is the picture of picture change, and the lower right picture is the picture of picture stillness.

[0029] The statistical method includes formula 2: ; In formula ③, when the frame difference result is not 0, the count value Increase; In formula ④, R represents the ratio of the calculated number of changed pixels to the total number of pixels in the n×n sub-region. When the ratio is greater than the set threshold, it is determined that the sub-region video content has changed. After the above calculation, the sub-region video content is distinguished between static and changing; the sub-region with changed video content is discarded, and the sub-region with static video content is retained, which solves the key point mentioned at the beginning of "ensuring that the content of the projected image remains unchanged".

[0030] Here is a method to "provide sufficiently rich texture information": (1) Existing methods for calculating clarity all rely on the gradient change of the edge of the picture. The larger the gradient change, the clearer the picture. However, the calculation in step ③ of formula 2 not only excludes the sub-regions where the video content has changed, but also cleverly retains the sub-regions where the edge gradient changes greatly.

[0031] (2) If Figure 4As shown in the figure, both sub-area 1 and sub-area 2 are reserved areas where the video content is still. However, although the content remains unchanged, the performance of the two sub-areas is completely different: sub-area 2 contains elements with rich edge information such as text and icons. These edge information will become blurred / clear during the autofocus process, and the transition zone formed by the change in edge clarity will be captured by the motion detection algorithm; while sub-area 1 lacks edge information, and even if focusing occurs, the motion detection result is zero.

[0032] (3) The image with insufficient edge information is used for clarity scoring. This score cannot make a difference during the focusing process, and cannot be used to determine the focus direction. Therefore, it is necessary to further screen the sub-regions based on step ③ in formula 2 to find the sub-regions with rich edge textures. Since edge changes are usually long and thin strips, the method of using area ratio is no longer feasible. Fourier transform can be used to convert the spatial domain into the frequency domain, and the richness of the edge can be judged by the frequency. The sub-regions with high frequency are retained, and the sub-regions with low frequency are eliminated. At present, there are mature operation libraries for Fourier transform, such as the numpy.fft.fft2() interface provided in the numpy library of python to perform frequency transformation of images, which will not be described in detail in this invention.

[0033] After the calculations in step ③ and step ④ of formula 2, sub-regions with unchanged video content and rich edge textures are screened out. There may be multiple such sub-regions. Here, a guiding method for using sub-regions is proposed: (1) If a sub-region has a drastic change in its video content during the focusing process through motion detection, the sub-region is eliminated.

[0034] (2) Among the remaining sub-regions, the sub-regions with high Fourier transform frequencies are used preferentially.

[0035] (3) Among the sub-regions with basically consistent frequencies, the sub-region close to the center of the entire image is used preferentially.

[0036] The lens is focused by combining the above-mentioned methods of using the sub-areas.

[0037] In summary, the present invention uses a camera to achieve non-sensing autofocus without projecting a feature map, thereby improving the user's viewing experience. The present invention extracts sub-pictures that can be used for AF clarity scoring from the constantly changing overall projection picture by dividing the sub-areas, distinguishes the static and changing sub-area video content, and selects a static sub-picture with rich textures. The extracted third sub-area is used to compare the changes in clarity, thereby achieving lens focusing.

[0038] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A camera-based projector non-sensing automatic focusing method, characterized in that: The method comprises the following steps: S1, select the projection area in the video image captured by the camera; S2, divide the projection area selected by S1 into n×n sub-areas; S3, during the auto-focus process and the video playback process, the dynamic image and the static image in the n×n sub-area are identified by a motion detection algorithm; S4, distinguishing still images and changing images of the video content in the n×n sub-regions in S3, discarding the first sub-region with changing images in the video content, and retaining the second sub-region with still images in the video content; S5, further extracting video image texture information having rich texture graphics and pictures based on the second sub-region retained in S4, and finding a third sub-region having rich edge texture; S6, comparing the change in clarity by retaining the second sub-area and the third sub-area, and focusing the lens.

2. The method for projector automatic focusing based on camera according to claim 1, characterized in that: In S3, the motion detection algorithm includes one of a mixed Gaussian model and a frame difference method.

3. The method for projector automatic focusing based on camera according to claim 2, characterized in that: The frame difference method includes formula 1: ； Among them, the is the absolute value of the difference between two frames of images; Among them, in formula ②, The value is filtered. When the D value is greater than the threshold T, the D value is Remain unchanged when When the value is less than the threshold T, D is reset to zero; The frame difference result is obtained through calculation: when the picture change is caused by video playback, the frame difference result is different; When the picture change is caused by zoom, after obtaining the frame difference result, the number of changed pixels is counted.

4. The method for projector automatic focusing based on camera according to claim 3, characterized in that: The statistical method includes formula 2: ； In formula ③, when the frame difference result is not 0, the count value Increase; In formula ④, R represents the ratio of the calculated changed pixels to the total number of pixels in the n×n sub-region. When the ratio is greater than the set threshold, it is determined that the video content of the sub-region has changed.

5. The camera-based projector non-sensing automatic focusing method according to claim 1, characterized in that: In S5, the method for extracting the video picture texture information is: after excluding the first sub-region where the video picture changes and retaining the second sub-region where the video content is still, further identifying a third sub-region where the video picture edge gradient changes greatly from the second sub-region.

6. The method for projector automatic focusing based on camera according to claim 5, characterized in that: The edge information of the third sub-area becomes blurred or clear during the autofocus process, and the transition zone formed by the change in edge clarity is captured by the motion detection algorithm. For the sub-area with insufficient edge information, even if a focusing action occurs, the motion detection result is zero; By scoring the clarity of the picture with insufficient edge information to obtain a score, the score cannot be used to make a difference during the focusing process, and thus the focusing direction cannot be determined. The third sub-region is further screened out to find the third sub-region with rich edge texture.

7. The method for projector automatic focusing based on camera according to claim 6, characterized in that: The third sub-area at least includes text, icons or a combination thereof.

8. The camera-based projector non-sensing automatic focusing method according to claim 6, characterized in that: The method for judging the edge richness of the third sub-region is to convert the spatial domain into the frequency domain by the Fourier transform method, and judge the richness of the edge of the picture by the frequency, retain the sub-region with high frequency, and eliminate the sub-region with low frequency.

Images