A real-time focusing method and system for a projector

Abstract

The present application relates to the technical field of intelligent projectors, and in particular to a real-time focusing method and system for projectors, which includes: generating a test image and dividing it into blocks to obtain several test areas; generating a timed focus determination task, adding a frame of test image to the video source at the corresponding time point; triggering an image acquisition instruction in the timed focus determination task to obtain the test image at each time point and the projection image of the next frame; selecting part of the test area and selecting at least one valid pixel point to generate a feature block, performing clarity analysis on several feature blocks of the test image at adjacent time points to obtain a first result; fusing the projected image through the test image to obtain a verification image, performing clarity analysis on the verification image at adjacent time points to obtain a second result; generating a focus change rate based on the first result and the second result. The present application has the effect of realizing real-time, non-sensing focus clarity monitoring during the operation of the projector.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent projectors, and in particular to a real-time focusing method and system for a projector. Background Art

[0002] Projectors are currently very commonly used image display devices in homes, businesses, and commerce. They can conveniently display images from external devices or built-in software on a wall or screen while ensuring a larger screen size.

[0003] Currently, many projectors use autofocus technology, which uses Tof laser technology, visual technology, etc. to automatically detect the distance between the projector and the screen or wall to determine the optimal projection focal length when the projector is turned on. The position of the lens is adjusted by a stepper motor combined with the focal length to achieve zoom, without the need for users to manually zoom.

[0004] However, projectors in related technologies generally only perform autofocus when the projector is turned on, standby mode is ended, etc., and no real-time focus detection and focus action will be performed during the use of the projector. However, during actual use, the projector is often affected by external and internal factors. External influences include touch and movement of the projector, brightness changes caused by turning on and off the light, and internal influences include thermal defocus caused by long-term projection. These problems may cause the image of the projector to change in clarity due to changes in focal length during the actual projection process. The projected image cannot be detected in real time during the use of the projector for real-time focus correction, which will reduce the viewing experience. Summary of the invention

[0005] In order to achieve real-time and seamless focus clarity monitoring during the operation of a projector, the present application provides a real-time focusing method and system for a projector.

[0006] In a first aspect, the present application provides a real-time focusing method for a projector, which adopts the following technical solution:

[0007] A real-time focusing method for a projector comprises the following steps:

[0008] Generate a test image, divide the test image into blocks to obtain a number of test areas, each of the test areas contains a number of valid pixels;

[0009] Generate a timed focus determination task, and add a frame of the test image to the video source at a time point corresponding to the timed focus determination task;

[0010] Triggering an image acquisition instruction in a timing focal length determination task to acquire the test image at each time point and a projection image of one frame after the time point;

[0011] Selecting part of the test area and selecting at least one valid pixel point in each of the test area to generate a feature block, and performing clarity analysis on a plurality of the feature blocks of the test image at adjacent time points to obtain a first result;

[0012] Performing fusion processing on the projection image through the test image to obtain a verification image, and performing clarity analysis on the verification images at adjacent time points to obtain a second result;

[0013] A focus change rate is generated according to the first result and the second result, and a corresponding real-time focus action is performed according to the focus change rate.

[0014] In some embodiments, generating a test image, dividing the test image into blocks to obtain a plurality of test areas, each of which contains a plurality of valid pixels, comprises the following steps:

[0015] Dividing the test image into a blank portion and a pattern portion;

[0016] Selecting a block division method so that each of the test areas includes a pattern portion of not less than a preset percentage, wherein the block division method includes any one or a combination of horizontal block division and vertical block division;

[0017] The pixel points corresponding to the pattern part in each of the test areas are defined as the effective pixel points.

[0018] In some embodiments, generating a timed focus determination task, and adding a frame of the test image to the video source at a time point corresponding to the timed focus determination task, comprises the following steps:

[0019] Obtaining the number of video frames of the video source, and setting a timing duration based on the number of video frames, selecting the time point in the video source at intervals of the timing duration, wherein the timing duration is inversely proportional to the number of video frames;

[0020] Acquire a functional frame number, where the functional frame number is represented by a maximum frame number supported by the projector;

[0021] When the number of video frames is less than the number of functional frames, a frame slot is added in the adjacent video frames corresponding to each of the time points, and the test images are respectively inserted into each of the frame slots;

[0022] When the number of video frames is equal to the number of functional frames, the video frames corresponding to each of the time points are deleted to generate a frame slot, and the test images are respectively inserted into each of the frame slots.

[0023] In some of the embodiments, triggering an image acquisition instruction in a timing focus determination task to acquire the test image at each time point and a projection image of a frame after the time point includes the following steps:

[0024] Adjusting each of the time points forward by a preset time length to generate a plurality of trigger points;

[0025] When the trigger point is reached, the image acquisition instruction is triggered to acquire a projection picture of unit time length;

[0026] The video frame corresponding to the test image and the projection image corresponding to the next frame of the test image are respectively searched in the projection picture.

[0027] In some embodiments, selecting a portion of the test area and selecting at least one valid pixel point in each of the test area to generate a feature block, and performing clarity analysis on a plurality of the feature blocks of the test image at adjacent time points to obtain a first result, comprises the following steps:

[0028] Randomly select at least half of the test areas from all the test areas, and obtain at least one feature block consisting of x*y pixels and with the center point being the valid pixel in the selected test areas, wherein n% of the same feature blocks and (1-n)% of the different feature blocks exist in different test images;

[0029] grayscale processing is performed on a plurality of the feature blocks in each of the test images, and they are randomly spliced ​​to obtain a first spliced ​​image, and a gradient value in each of the first spliced ​​images is calculated;

[0030] The first spliced ​​image corresponding to the m-th test image is selected and the gradient value is compared with the first spliced ​​image corresponding to the m-1-th test image, and the comparison result is used as the first result.

[0031] In some of the embodiments, at least half of the test areas are randomly selected from all the test areas, and the steps further include:

[0032] Collecting the type of imaging object, and acquiring a preset basic flatness in a database based on the imaging object information;

[0033] Collecting flight time parameters through ToF, and setting a smoothing coefficient based on the flight time parameters;

[0034] A flattening score is calculated based on the basic flattening degree and the flattening coefficient, and the number of the selected test areas is adjusted according to the flattening score.

[0035] In some embodiments, the projected image is fused with the test image to obtain a verification image, and the verification image at adjacent time points is analyzed for clarity to obtain a second result, including the following steps:

[0036] Obtaining the point coordinates and pixel parameters of the valid pixel points in the test image;

[0037] Based on the coordinates of the plurality of points, the pixel points at the corresponding positions in the projected image are searched as the pixel points to be replaced, and each of the pixel points to be replaced is replaced with the pixel parameters of the corresponding valid pixel points;

[0038] Mapping each of the feature blocks on the test image corresponding to the previous frame to the verification image to obtain a number of verification blocks with the same position and size;

[0039] grayscale processing is performed on a plurality of the verification blocks in each of the test images, and the blocks are randomly spliced ​​to obtain a second spliced ​​image, and a gradient value in each of the second spliced ​​images is calculated;

[0040] The second spliced ​​image corresponding to the m-th verification image is selected and the gradient value is compared with the second spliced ​​image corresponding to the m-1-th verification image, and the comparison result is used as the second result.

[0041] In some embodiments, generating a focus change rate according to the first result and the second result, and performing a corresponding real-time focus action according to the focus change rate includes the following steps:

[0042] generating a basic focus change rate based on the first result and the second result, generating a clarity trust weighting based on a difference between the first result and the second result, and optimizing the basic focus change rate by using the clarity trust weighting to obtain a final focus change rate;

[0043] When the final focus change rate is greater than a preset value, a focal length adjustment amount is calculated by the final focus change rate and sent to the stepping motor.

[0044] In some of the embodiments, generating a focus change rate according to the first result and the second result, and performing a corresponding real-time focus action according to the focus change rate, further includes the following steps:

[0045] When the final focus change rate is greater than a preset value, a response instruction is generated to enable a focus component to operate and obtain current focus parameters, wherein the focus component includes a ToF and / or a gyroscope, and the current focus parameters include a flight distance and / or a projection angle;

[0046] It is determined whether there is a difference between the current focus parameter and the initial focus information, and based on the determination result, a defocus reason and a corresponding type of focus solution are generated, wherein the initial focus information is characterized as information when the focus is completed when the projector is powered on.

[0047] In a second aspect, the present application provides a projector real-time focusing system, which adopts the following technical solution:

[0048] A real-time focusing system for a projector, characterized by being used to implement the above method.

[0049] The technical solution provided by the embodiments of the present application has the following technical effects:

[0050] Part of the test image is inserted into the video source by interpolation, and part of the feature blocks are selected in the test image by block point selection. The change in clarity of the feature blocks at different time points is used to characterize the change in clarity in the projected image. At the same time, in order to improve the accuracy and credibility of image comparison, the projection image of the next frame after the test image is combined with the test image for another clarity comparison. The two results are integrated, analyzed and cross-validated to determine whether there is a change in focal length and whether refocusing is required due to the change in clarity. In this way, the clarity caused by the projection focal length can be detected in real time during the projection process without affecting the user's viewing experience. When clarity abnormality is detected, real-time refocusing can be performed in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of the steps of a projector real-time focusing method provided in this embodiment. DETAILED DESCRIPTION

[0052] To more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. However, it should be understood by those of ordinary skill in the art that the present application can be implemented without these details. In some cases, in order to avoid unnecessary descriptions that make various aspects of the present application obscure, well-known methods, processes, systems, components and / or circuits that have been described at a higher level will not be described in detail. For those of ordinary skill in the art, it is obvious that various changes can be made to the embodiments disclosed in the present application, and without departing from the principles and scope of the present application, the general principles defined in the present application can be applied to other embodiments and application scenarios. Therefore, the present application is not limited to the embodiments shown, but conforms to the broadest scope consistent with the scope claimed for protection of the present application.

[0053] It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention. In addition, the technical features involved in each embodiment of the present invention described below can be combined with each other as long as there is no conflict between them.

[0054] In the description of this application, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used to distinguish the technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0055] In the description of the present application, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples.

[0056] like Figure 1 As shown, the embodiment of the present application discloses a real-time focusing method for a projector, comprising the following steps:

[0057] S100, generating a test image, dividing the test image into blocks to obtain a number of test areas, each of which contains a number of valid pixels.

[0058] The test image is generally an image with a white background and several black graphic blocks added to the background.

[0059] First, the test image is divided into blocks, each block has the same size, and is a rectangular or square block. Each divided block corresponds to an independent test area.

[0060] Each test area divided in this way is composed of some white pixels and some black pixels.

[0061] Due to the different image structures of different projectors, the same color may have color cast in the projection images when emitted by different projectors. However, the color cast corresponding to white and black is less or non-existent, so the test image prefers black and white.

[0062] S200, generating a timed focus determination task, and adding a frame of test image to the video source at a time point corresponding to the timed focus determination task.

[0063] A focus determination task is generated periodically. The focus determination task is characterized by continuously determining the focus state of the projector during the continuous operation of the projector to pay attention to whether the projector is out of focus during use.

[0064] The timed focus judgment task corresponds to different trigger time points. When the time point is reached, the system will make a focus judgment. Then, at the judgment time point, the video frames originally corresponding to these time points in the video source are replaced with test images. In this way, each time a focus judgment is made at a time point, the acquired picture is a test image. In this way, the test image projected by the projector can be used as the object to be tested.

[0065] At the same time, since the minimum refresh rate of projectors is generally 60hz, 120hz, and 240hz, that is, 60, 120, and 240 frames of images can be refreshed in one second, then inserting a frame of test image under the premise of high refresh rate is difficult for the user's naked eye to detect, so reasonable frame insertion in the video source will not affect the user's viewing experience.

[0066] S300, triggering an image acquisition instruction in a timing focal length determination task to acquire a test image at each time point and a projection image one frame after the time point.

[0067] After inserting frames in the video source according to each time point, the projection activity is carried out normally, and when the current time reaches the time point, the camera on the projector is triggered to start working to shoot the projection picture, and the test image on the frame corresponding to the time point and the projection image in the next frame of the test image are screened out from the captured projection picture.

[0068] In this way, it can be considered that two frames of images need to be acquired at the time point corresponding to each timed focus determination task, one of which is a test image and the other is a projection image. Both frames of images are used for subsequent focus state determination.

[0069] S400, selecting a portion of the test area and selecting at least one valid pixel point in each test area to generate a feature block, and performing clarity analysis on a plurality of feature blocks of the test image at adjacent time points to obtain a first result.

[0070] In the embodiment of the present application, the effective pixel point is characterized as a black pixel point corresponding to the black pattern.

[0071] Among all the test areas, some test areas are selected as feature areas, and at least one valid pixel is selected in each feature area, and feature blocks are generated based on these valid pixels. The feature blocks are characterized as pixel blocks containing multiple pixels and one valid pixel at the same time.

[0072] The analysis result is obtained by performing clarity analysis on several feature blocks in a test image and several pixel blocks in other test images at adjacent time points. In this way, the clarity analysis of the entire image can be replaced by a partial pixel area in the entire test image, thereby reducing the processing process.

[0073] S500, performing fusion processing on the projected image through the test image to obtain a verification image, and performing clarity analysis on the verification images at adjacent time points to obtain a second result.

[0074] The test image acquired in the previous frame is fused with the projection image in the next frame to obtain a verification image that includes both pixel features of the test image and pixel features of the projection image, and a second result is also obtained based on the clarity analysis between the verification images at adjacent time points.

[0075] S600: Generate a focus change rate according to the first result and the second result, and perform a corresponding real-time focusing action according to the focus change rate.

[0076] The first result and the second result obtained in the two comparison processes are integrated and analyzed and cross-validated to calculate the focus change rate. The focus change rate is characterized by the degree of change in the focus effect of the projected image from one time point to the next time point after analyzing the images of adjacent time points. The focus change rate is used to determine whether there is a change in the focus state to determine whether a new focus action is needed.

[0077] Through the above steps, part of the test image is inserted into the video source by means of interpolation, and part of the feature blocks are selected in the test image by means of block point selection. The change in clarity of the feature blocks at different time points is used to characterize the change in clarity in the projected image. At the same time, in order to improve the accuracy and reliability of image comparison, the projection image of the next frame after the test image is combined with the test image and then a clarity comparison is performed. The two results are integrated, analyzed and cross-validated to determine whether there is a change in focal length and whether refocusing is required due to the change in clarity. In this way, the clarity caused by the projection focal length can be detected in real time during the projection process without affecting the user's viewing experience. When clarity abnormality is detected, real-time refocusing can be performed in a timely manner.

[0078] In some embodiments, generating a test image, dividing the test image into blocks to obtain a plurality of test areas, each of which contains a plurality of valid pixels, comprises the following steps:

[0079] S110, dividing the test image into a blank part and a pattern part.

[0080] The test image is divided into a blank part consisting of white pixels and a pattern part consisting of black pixels.

[0081] S120, selecting a block division method so that each test area includes a pattern portion of not less than a preset percentage, wherein the block division method includes any one or a combination of horizontal blocking and vertical blocking.

[0082] When dividing the test image into blocks, you can choose horizontal blocking, vertical blocking or a combination of horizontal and vertical blocking. The horizontal blocking means dividing the image into 1 column * n rows based on the horizontal direction of the test image. The vertical blocking means dividing the image into n columns * 1 row based on the vertical direction of the test image. The combination means dividing the image both horizontally and vertically to obtain n columns * m rows.

[0083] Different block division methods are used to ensure that each test area has a certain percentage of pattern parts, wherein the preset percentage is 40% in the embodiment of the present application.

[0084] S130, defining the pixel points corresponding to the pattern part in each test area as valid pixel points.

[0085] The black pixels in each divided test area are defined as valid pixels.

[0086] In some embodiments, generating a timed focus determination task and adding a frame of test image to a video source at a time point corresponding to the timed focus determination task includes the following steps:

[0087] S210, obtaining the number of video frames of the video source, and setting a timing duration based on the number of video frames, selecting a time point in the video source at intervals of the timing duration, wherein the timing duration is inversely proportional to the number of video frames.

[0088] The video frame number is represented by the number of playback frames corresponding to the currently played video source, and the video frame number is generally 60 frames or 120 frames.

[0089] For video sources with different video frame rates, the corresponding timing durations when performing the timed focus determination task are different. This is because the larger the video frame rate, the smaller the impact on the user's viewing experience when a test frame is inserted, because it is more difficult for the user to notice. Conversely, the smaller the video frame rate, the greater the impact on the user's viewing experience when a test frame is inserted.

[0090] Therefore, when the number of video frames is small, the longer the timing duration is set, so as to reduce the number of test images inserted within a certain period of time and reduce the number of times that may cause negative viewing perception to users. When the number of video frames is large, the timing duration can be set shorter.

[0091] After selecting the timing duration, take the first frame of the video source as the starting point and select the corresponding time point in the video source according to the timing duration.

[0092] S220, obtaining a functional frame number, where the functional frame number is represented by a maximum frame number supported by the projector.

[0093] The functional frame rate is characterized by the maximum projection frame rate supported by the projector, where the functional frame rate is generally greater than the video frame rate. When the functional frame rate is less than the video frame rate, the video source can only be projected onto the corresponding surface through the functional frame rate.

[0094] S230, when the number of video frames is less than the functional frame number, a frame slot is added in the adjacent video frames corresponding to each time point, and a test image is inserted into each frame slot respectively.

[0095] When the video frame number is less than the functional frame number, it means that the video source does not reach the maximum frame number supported by the projector during playback. In this case, a frame gap can be added directly between two adjacent video frames at the corresponding time point and a test image can be inserted into the frame gap.

[0096] This may change the output of 60 frames per second at some video source locations to 61 frames per second, that is, the video source location where the test image exists is allowed to be played out from the functional frame rate at a frame rate greater than the original video frame rate, and the functional frame rate also allows and supports this behavior.

[0097] S240, when the number of video frames is equal to the number of functional frames, the video frames corresponding to each time point are deleted to generate a frame slot, and the test images are respectively inserted into each frame slot.

[0098] When the video frame is equal to the functional frame number, adding another frame to the corresponding video frame number per second will cause it to be greater than the functional frame number and cannot be effectively displayed. Therefore, in order to ensure that the number of frames output by the video source per second remains unchanged, it is necessary to remove the video frame at the position corresponding to the time point. The removed position leaves a vacant frame, and then insert the test image into the vacant frame. In this way, frame insertion can be achieved while keeping the number of frames per second of the video source unchanged.

[0099] In some of the embodiments, triggering an image acquisition instruction in a timing focus determination task to acquire a test image at each time point and a projection image of a frame after the time point includes the following steps:

[0100] S310, adjusting each time point forward by a preset time length to generate a plurality of trigger points.

[0101] S320, triggering an image acquisition instruction to acquire a projection picture of unit time length when the trigger point is reached.

[0102] S330, searching for a video frame corresponding to the test image and a projection image corresponding to the next frame of the test image in the projection picture.

[0103] Get each time point, and get the trigger point after extending the preset time forward at each time point. The trigger point is also a time feature. Because the projection screen needs to be photographed at the time point, the camera on the projector needs to be triggered to work. The camera is often turned off or in standby mode when not working. If it is required to shoot at the specified time, a certain amount of time advance is required to put the camera into working state. The forward adjustment time is generally 1-2s, and after the camera starts working, it can shoot a video of about 1s. This is because a frame of the picture is often only 1 / 60s, and its display time is very short. In order to avoid missing the test image, it is necessary to shoot a 1s video and obtain the test image by screening each frame of the video.

[0104] The shot video of about 1 second is used as the projection picture of unit time length, and each frame in the picture of 1 second is obtained, and the video frame corresponding to the test image and the projection image adjacent to the test image and one frame after the test image are screened out from several frames.

[0105] Among them, in other embodiments, due to the performance differences of cameras on different projectors, the acquisition position of the projection image can be modified. Since the shutter speed and performance of some cameras are poor, the projection image of the next frame may have an afterimage of the test image of the previous frame. Therefore, in order to reduce the image difference caused by the difference in camera performance, the projection image after several frames of the test image can be selected.

[0106] In some embodiments, a portion of the test area is selected and at least one valid pixel point is selected in each test area to generate a feature block, and a plurality of feature blocks of the test image at adjacent time points are subjected to clarity analysis to obtain a first result, including the following steps:

[0107] S410, randomly select at least half of the test areas from all the test areas, and obtain at least one feature block in the selected test areas, which is composed of x*y pixels and whose center point is a valid pixel, wherein n% of the same feature blocks and (1-n)% of different feature blocks exist in different test images.

[0108] At least one feature block is selected in each selected test area, each feature block is composed of an array of x pixels in the horizontal direction and y pixels in the vertical direction, and the center point of each feature block is a valid pixel point. The divided feature blocks may contain both pattern parts and blank parts, or may only contain pattern parts.

[0109] At the same time, in order to ensure that there is a certain degree of difference when performing clarity comparison on different test images while ensuring high confidence in the comparison, a certain number of identical feature blocks and a certain number of different feature blocks need to exist in different test patterns.

[0110] The same characterization means that the color and position of the pixels between the feature blocks are exactly the same. In the embodiment of the present application, n is 0.3. In this way, different feature blocks ensure the sample difference when comparing multiple test images to achieve high accuracy, and the same feature blocks ensure the sample credibility when comparing multiple test images.

[0111] S420, grayscale processing is performed on a plurality of feature blocks in each test image, and the blocks are randomly spliced ​​to obtain a first spliced ​​image, and a gradient value in each first spliced ​​image is calculated.

[0112] A plurality of feature blocks selected from each test image are gray-scaled to obtain a gray-scale image, and a plurality of gray-scale images are spliced ​​to obtain a first spliced ​​image. The splicing method is to make the edges of a plurality of feature blocks adjacent to each other and to align the edge lines when splicing. In this way, the selected plurality of regions can be spliced ​​into a gray-scale overall image.

[0113] Get the gray value of each pixel in the first mosaic image, and take the sum of the squares of the gray value differences of adjacent pixels in the x and y directions as the gradient value of each pixel, and add the gradient values ​​of each pixel in the first mosaic image as the clarity evaluation function value. The expression is as follows:

[0114] .

[0115] The larger the gradient value, the clearer the image.

[0116] S430: Select a first spliced ​​image corresponding to the m-th test image and compare the gradient values ​​with the first spliced ​​image corresponding to the (m-1)-th test image, and use the comparison result as the first result.

[0117] The gradient value of the first spliced ​​image between each test image and the test image at the previous time point is compared, and the comparison result is used as the first result.

[0118] Among them, there are three main first results. The first is that the gradient value of the m-1th first mosaic is greater than the gradient value of the mth first mosaic, which indicates that the clarity of the projected image deteriorates with the use of the projector; the second is that the gradient value of the m-1th first mosaic is equal to the gradient value of the mth first mosaic, which indicates that the clarity of the projected image does not change with the use of the projector; the third is that the gradient value of the m-1th first mosaic is less than the gradient value of the mth first mosaic, which indicates that the clarity of the projected image improves with the use of the projector. The size of each different numerical result corresponds to the degree of clarity or blur.

[0119] It should be noted that, when m=1, the m-1th test picture is represented as a test picture when the projector is initially focused when it is first powered on.

[0120] In some of the embodiments, at least half of the test areas are randomly selected from all the test areas, and the steps further include:

[0121] S411, collecting the imaging object type, and acquiring a preset basic flatness in a database based on the imaging object information.

[0122] S412, collecting flight time parameters through ToF, and setting a smoothing coefficient based on the flight time parameters.

[0123] S413, calculating a flatness score based on the basic flatness and the flatness coefficient, and adjusting the number of selected test areas according to the flatness score.

[0124] When choosing the number of test areas, that is, the number of feature blocks, it is necessary to consider the presentation medium of the image projected by the projector.

[0125] The general imaging objects of the projector include walls, screens, building exterior walls, ceilings, desktops, etc. Different types of imaging objects represent different flatness of the imaging objects. For example, screens are generally the type with the highest flatness, walls may have low flatness due to partial wall falls or decorations, and ceilings, desktops, etc. are generally blocked by some objects.

[0126] Different degrees of flatness will cause the imaging effects of some areas to be different from those of other flat areas during projection. Therefore, when performing block selection detection on the test image, there will be abnormal differences in the grayscale values ​​of uneven positions and other positions. In order to avoid or reduce the comparison result errors caused by such abnormal differences, theoretically, the more uneven the imaging object is, the more test areas should be selected for projection imaging, so as to reduce the impact of the feature blocks being selected in uneven positions. At the same time, the errors caused by the grayscale results in uneven positions can also be diluted by a large number of grayscale results in flat positions.

[0127] Therefore, the basic flatness is first selected in the preset database according to the imaging object type. The basic flatness is a prediction result based on experience and a large amount of data. In fact, the flatness of the currently projected wall may be smoother or more uneven than the basic flatness. The basic flatness is only a theoretical value estimated for different imaging object types under ideal conditions.

[0128] At the same time, the ToF component installed on the projector emits laser light, and the distance between the object and the projector is determined by the flight time of the laser light from the time it is emitted to the time it is reflected back to the component position after passing through the wall or curtain. Due to the different depths of flatness, the flight time of the laser light at the same position is different at different positions, and the overall flatness of the imaging object can be determined based on the difference in flight time.

[0129] Current projectors generally use 2d ToF or 3d Tof. 2d ToF often has 10-30 sensing units, which can measure the distance to dozens of points on the imaging object. 3d Tof has thousands to tens of thousands of sensing units, which can perform overall three-dimensional distance measurement of the imaging object.

[0130] Then, the corresponding time-of-flight parameter acquisition scheme is selected according to the projector with different configurations. The flattening coefficient is calculated by acquiring the time-of-flight parameters, and the flattening coefficient is characterized as a weighting coefficient of the actual measured flatness after ranging by the ToF component.

[0131] The basic flatness is weightedly calculated using the flatness coefficient to obtain a flatness score. The higher the flatness score, the higher the flatness of the imaging object, so a smaller number of test areas can be selected; the lower the flatness score, the lower the flatness of the imaging object, so more test areas need to be selected to reduce the projection image presentation error caused by uneven positions.

[0132] In other embodiments, if the projector does not have a Tof component, the number of test areas may be selected based only on the basic flatness.

[0133] In some embodiments, the projected image is fused with the test image to obtain a verification image, and the verification images at adjacent time points are analyzed for clarity to obtain a second result, including the following steps:

[0134] S510, obtaining the point coordinates and pixel parameters of valid pixel points in the test image.

[0135] S520, searching for pixel points at corresponding positions in the projected image as pixel points to be replaced based on the coordinates of the plurality of points, and replacing each pixel point to be replaced with the pixel parameters of the corresponding valid pixel point.

[0136] S530, mapping each feature block on the test image corresponding to the previous frame to the verification image to obtain a number of verification blocks with the same position and size.

[0137] First, the point coordinate position and pixel parameters of each effective pixel point in the test image are obtained. The pixel parameters in the embodiment of the present application are RGB value, brightness and contrast.

[0138] According to the coordinates of each pixel point, the corresponding pixel point position is found in the projection image, and the pixel point at the corresponding position is replaced with the pixel parameters of the valid pixel point in the test image. The verification image obtained in this way is a fusion image of the test image and the projection image, wherein the test image contains the pattern part composed of all valid pixels, and the blank part in the original test image is the display content on the projection image.

[0139] The verification image retains both a portion of the pattern in the test image and the projected content displayed as the video source is played.

[0140] Each feature block on the test image corresponding to the same time point is mapped to the corresponding position in the verification image, so that the feature block that is exactly the same as that in the test image is obtained on the verification image and defined as the verification block.

[0141] S540, grayscale processing is performed on a plurality of verification blocks in each test image, and they are randomly spliced ​​to obtain a second spliced ​​image, and a gradient value of each pixel point in each second spliced ​​image is calculated.

[0142] S550: Select the second spliced ​​image corresponding to the m-th verification image and compare the gradient values ​​with the second spliced ​​image corresponding to the (m-1)-th verification image, and use the comparison result as the second result.

[0143] The calculation method is the same as the first result. Each verification block is grayscale processed and randomly spliced ​​to obtain a second spliced ​​image and the gradient value of each second spliced ​​image is calculated. The second result is generated by comparing it with the gradient value of the second spliced ​​block at the previous time point.

[0144] Among them, because in the second spliced ​​image, the white parts originally in the first spliced ​​image have all become different colors corresponding to the projected image, the grayscale value of each pixel will change after grayscale processing, but the gradient value calculated based on the overall calculation is generally the same as the tendency of the first result, but there will be certain differences in the numerical value of the comparison result during comparison.

[0145] In some embodiments, generating a focus change rate according to the first result and the second result, and performing a corresponding real-time focus action according to the focus change rate includes the following steps:

[0146] S610, generating a basic focus change rate based on the first result and the second result, generating a clarity trust weighting based on the difference between the first result and the second result, and optimizing the basic focus change rate by the clarity trust weighting to obtain a final focus change rate.

[0147] The basic focus change rate is generated by the average value of the first result and the second result. The basic focus change rate is characterized by the average change value of the gradient value between two time points. Among them, when both data are reliable, the numerical values ​​of the two results should be the same or close. When the two data are unreliable, the average change value of the gradient values ​​of the two results is also unreliable and needs to be adjusted.

[0148] The clarity trust weighting is generated based on the difference between the first result and the second result. If the difference is large, it is considered that the credibility between the two results is low. Conversely, if the difference is small, it is considered that the credibility between the two results is high.

[0149] When the credibility is high, the credibility reward or credibility probability of the data can be obtained based on the Bayesian algorithm or reinforcement learning algorithm to obtain the corresponding weight, which is generally between 1-1.2, and the basic focus change rate is multiplied by the weight to obtain the final focus change rate.

[0150] When the credibility is low, the credibility reward or credibility probability of the first result and the second result are obtained respectively based on the Bayesian algorithm or the reinforcement learning algorithm, and the result with a higher credibility reward or credibility probability is selected, the clarity trust weight is set to 0, and the previously calculated basic focus change rate is multiplied by 0 and then added to the value corresponding to the result with better credibility to obtain the final focus change rate.

[0151] S620, when the final focus change rate is greater than a preset value, a focal length adjustment amount is calculated by the focus change rate and sent to the stepping motor.

[0152] When the final focus change rate is greater than the preset value, it is considered that the projected image is already in a relatively blurred state. At this time, the calculated focus change rate is needed to control the stepper motor to adjust the position of the lens to achieve refocusing.

[0153] Among them, when the final focus change rate is less than the preset value, the projection picture may become clearer or blurred but not to the extent that the user can intuitively experience it. At this time, in order to reduce the impact of the focusing process on the user's viewing experience, the focusing process may not be performed.

[0154] In some embodiments, a focus change rate is generated according to the first result and the second result, and a corresponding real-time focus action is performed according to the focus change rate, further comprising the following steps:

[0155] S630, when the final focus change rate is greater than a preset value, a response instruction is generated to enable the focus component to run and obtain current focus parameters, the focus component includes a ToF and / or a gyroscope, and the current focus parameters include a flight distance and / or a projection angle.

[0156] When it is detected that the clarity of the projected image is greatly blurred, before performing the real-time focusing action, it is necessary to send a response command to some of the focusing auxiliary devices on the projector to enable each focusing auxiliary device to work.

[0157] The focus auxiliary device includes a ToF component and / or a gyroscope component. The ToF component is mainly used to detect the projection distance, and the gyroscope component is mainly used to detect changes in the projector's own posture (projection horizontal angle, projection vertical angle).

[0158] When the current focus parameters of the projector change or remain unchanged, it corresponds to different reasons for the clarity change, and different reasons for the clarity change require the selection of a more appropriate focus solution to improve the focus accuracy. At the same time, when the clarity changes due to different problems, other related operations may be required while refocusing.

[0159] S640, determining whether there is a difference between the current focus parameters and the initial focus information, and generating a defocus reason and a corresponding type of focus solution based on the determination result, wherein the initial focus information is information when the projector is powered on and the focus is completed.

[0160] When there is a difference between the current focus parameters and the initial focus information, it is considered that the position and / or posture of the projector has changed during the projection process, such as being touched or moved. At this time, the cause of the out-of-focus is considered to be a change in the projector state. In order to maintain a good projection effect, keystone correction, brightness adjustment and other steps are required after refocusing.

[0161] If there is no difference between the current focus parameters and the initial focus information, it is considered that the projector itself has not changed its position and / or posture. It may be due to thermal defocus caused by the heat of the projector or changes in ambient brightness caused by turning the lights on and off. In this case, the cause of the defocus is considered to be a projection environment problem. The corresponding focus solution can be to only adjust the focal length to refocus, and the brightness can also be adjusted.

[0162] The present application also discloses a real-time focusing system for a projector, which is characterized by being used to implement the above method.

[0163] The implementation principle is:

[0164] Part of the test image is inserted into the video source by interpolation, and part of the feature blocks are selected in the test image by block point selection. The change in clarity of the feature blocks at different time points is used to characterize the change in clarity in the projected image. At the same time, in order to improve the accuracy and credibility of image comparison, the projection image of the next frame after the test image is combined with the test image for another clarity comparison. The two results are integrated, analyzed and cross-validated to determine whether there is a change in focal length and whether refocusing is required due to the change in clarity. In this way, the clarity caused by the projection focal length can be detected in real time during the projection process without affecting the user's viewing experience. When clarity abnormality is detected, real-time refocusing can be performed in a timely manner.

[0165] It should be understood that although the steps in the flowchart of the accompanying drawings are shown in sequence according to the instructions of the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise clearly stated in this document, the execution of these steps is not strictly limited in order and can be performed in other orders.

[0166] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A real-time focusing method for a projector, characterized in that: The following steps are involved: Generate a test image, divide the test image into blocks to obtain a number of test areas, each of the test areas contains a number of valid pixels; Generate a timed focus determination task, and add a frame of the test image to the video source at a time point corresponding to the timed focus determination task; Triggering an image acquisition instruction in a timing focal length determination task to acquire the test image at each time point and a projection image of one frame after the time point; Selecting portions of the test area on each frame of the test image and selecting at least one valid pixel point in each of the test areas to generate a feature block, and performing clarity analysis on a plurality of the feature blocks of the test image at adjacent time points to obtain a first result; The projection image is fused with the test image to obtain a verification image, each of the feature blocks on the test image corresponding to the previous frame is mapped to the verification image to obtain a number of verification blocks of the same position and size, and a clarity analysis is performed on a number of the verification blocks of the verification image at adjacent time points to obtain a second result; A focus change rate is generated according to the first result and the second result, and a corresponding real-time focus action is performed according to the focus change rate.

2. The projector real-time focusing method according to claim 1, characterized in that: Generate a test image, divide the test image into blocks to obtain a plurality of test areas, each of the test areas contains a plurality of valid pixels, including the following steps: Dividing the test image into a blank portion and a pattern portion; Selecting a block division method so that each of the test areas includes a pattern portion of not less than a preset percentage, wherein the block division method includes any one or a combination of horizontal block division and vertical block division; The pixel points corresponding to the pattern part in each of the test areas are defined as the effective pixel points.

3. The projector real-time focusing method according to claim 1, characterized in that: Generating a timed focus determination task, and adding a frame of the test image to the video source at a time point corresponding to the timed focus determination task, comprises the following steps: Obtaining the number of video frames of the video source, and setting a timing duration based on the number of video frames, selecting the time point in the video source at intervals of the timing duration, wherein the timing duration is inversely proportional to the number of video frames; Acquire a functional frame number, where the functional frame number is represented by a maximum frame number supported by the projector; When the number of video frames is less than the number of functional frames, a frame slot is added in the adjacent video frames corresponding to each of the time points, and the test images are respectively inserted into each of the frame slots; When the number of video frames is equal to the number of functional frames, the video frames corresponding to each of the time points are deleted to generate a frame slot, and the test images are respectively inserted into each of the frame slots.

4. The projector real-time focusing method according to claim 1, characterized in that: Triggering an image acquisition instruction in a timing focal length determination task to acquire the test image at each time point and a projection image of one frame after the time point includes the following steps: Adjusting each of the time points forward by a preset time length to generate a plurality of trigger points; When the trigger point is reached, the image acquisition instruction is triggered to acquire a projection picture of unit time length; The video frame corresponding to the test image and the projection image corresponding to the next frame of the test image are respectively searched in the projection picture.

5. The projector real-time focusing method according to claim 2, characterized in that: Selecting a portion of the test area on each frame of the test image and selecting at least one valid pixel point in each test area to generate a feature block, and performing clarity analysis on a plurality of the feature blocks of the test image at adjacent time points to obtain a first result, including the following steps: Randomly select at least half of the test areas from all the test areas, and obtain at least one feature block consisting of x*y pixels and with the center point being the valid pixel in the selected test areas, wherein n% of the same feature blocks and (1-n)% of the different feature blocks exist in different test images; grayscale processing is performed on a plurality of the feature blocks in each of the test images, and the blocks are randomly spliced ​​to obtain a first spliced ​​image, and a gradient value in each of the first spliced ​​images is calculated; The first spliced ​​image corresponding to the m-th test image is selected and the gradient value is compared with the first spliced ​​image corresponding to the m-1-th test image, and the comparison result is used as the first result.

6. The projector real-time focusing method according to claim 5, characterized in that: Randomly selecting at least half of the test areas from all the test areas, specifically further comprising the following steps: Collecting the type of imaging object, and acquiring a preset basic flatness in a database based on the imaging object information; Collecting flight time parameters through ToF, and setting a flattening coefficient based on the flight time parameters; A flattening score is calculated based on the basic flattening degree and the flattening coefficient, and the number of the selected test areas is adjusted according to the flattening score.

7. The projector real-time focusing method according to claim 5, characterized in that: The projected image is fused with the test image to obtain a verification image, each of the feature blocks on the test image corresponding to the previous frame is mapped to the verification image to obtain a number of verification blocks of the same position and size, and a clarity analysis is performed on a number of the verification blocks of the verification image at adjacent time points to obtain a second result, including the following steps: Obtaining the point coordinates and pixel parameters of the valid pixel points in the test image; Based on the coordinates of the plurality of points, the pixel points at the corresponding positions in the projected image are searched as the pixel points to be replaced, and each of the pixel points to be replaced is replaced with the pixel parameters of the corresponding valid pixel points; grayscale processing is performed on a plurality of the verification blocks in each of the test images, and the blocks are randomly spliced ​​to obtain a second spliced ​​image, and a gradient value in each of the second spliced ​​images is calculated; The second spliced ​​image corresponding to the m-th verification image is selected and the gradient value is compared with the second spliced ​​image corresponding to the m-1-th verification image, and the comparison result is used as the second result.

8. The real-time focusing method for a projector according to claim 6 or 7, characterized in that: Generating a focus change rate according to the first result and the second result, and performing a corresponding real-time focus action according to the focus change rate, comprises the following steps: generating a basic focus change rate based on the first result and the second result, generating a clarity trust weighting based on a difference between the first result and the second result, and optimizing the basic focus change rate by using the clarity trust weighting to obtain a final focus change rate; When the final focus change rate is greater than a preset value, a focal length adjustment amount is calculated by the final focus change rate and sent to the stepping motor.

9. The projector real-time focusing method according to claim 8, characterized in that: Generating a focus change rate according to the first result and the second result, and performing a corresponding real-time focus action according to the focus change rate, further comprising the following steps: When the final focus change rate is greater than a preset value, a response instruction is generated to enable a focus component to operate and obtain current focus parameters, wherein the focus component includes a ToF and / or a gyroscope, and the current focus parameters include a flight distance and / or a projection angle; It is determined whether there is a difference between the current focus parameter and the initial focus information, and based on the determination result, a defocus reason and a corresponding type of focus solution are generated, wherein the initial focus information is characterized as information when the focus is completed when the projector is powered on.

Images