Projection picture adjustment method and apparatus, storage medium, and device

Abstract

The present application relates to the technical field of projection, and discloses a projection picture adjustment method and apparatus, a storage medium, and a device. The method comprises: extracting a projection picture area and an area outside a projection picture from a captured image; performing feature extraction on the projection picture area to obtain a first picture feature, and performing feature extraction on the area outside the projection picture to obtain a second picture feature; and adjusting a projection parameter on the basis of the first picture feature and the second picture feature. The present application can improve a projection effect.

Description

Projection image adjustment methods, devices, storage media and equipment

[0001] This application claims priority to Chinese Patent Application No. 202411605296.X, filed on November 11, 2024, entitled “Projection Screen Adjustment Method, Apparatus, Storage Medium and Device”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of projection technology, specifically to a projection image adjustment method, device, storage medium, and equipment. Background Technology

[0003] When using a projector, users usually project the image onto a flat surface such as a wall to watch movies. The viewing environment may be a dark environment or a bright environment. In addition, the flat surface may be a white wall or a wall covered with colored wallpaper.

[0004] Given the uncertainty of the viewing environment and screen color, current projection methods typically involve simply adjusting the projector based on the ambient brightness, often resulting in poor projection quality and a subpar viewing experience for users. Technical issues

[0005] Current projection methods typically involve simply adjusting the projector based on ambient brightness, often resulting in poor projection quality and a subpar viewing experience for users. Technical solutions

[0006] This application provides a solution that can effectively improve projection effects and enhance the user's projection viewing experience.

[0007] The embodiments of this application provide the following technical solutions:

[0008] According to one embodiment of this application, a projection screen adjustment method includes: acquiring a captured image of a test projection screen projected by a projector onto a plane; extracting a projection screen area from the captured image and extracting an area outside the projection screen from the captured image; performing feature extraction on the projection screen area to obtain a first screen feature characterizing the planar features of the plane located within the projection screen area, and performing feature extraction on the area outside the projection screen to obtain a second screen feature characterizing the environmental features of the projection environment; and adjusting the projection parameters of the projector according to the first screen feature and the second screen feature.

[0009] In some embodiments of this application, the test projection screen includes a pure white screen, and the captured image includes a first image obtained by capturing the pure white screen; the step of extracting the projection screen region from the captured image includes: obtaining a perspective transformation matrix for transforming the image from a first coordinate system of the captured image to a second coordinate system of the projector; and performing perspective transformation processing on the first image based on the perspective transformation matrix to obtain the projection screen region located in the second coordinate system.

[0010] In some embodiments of this application, the test projection screen further includes a feature point screen, and the captured image includes a second image obtained by capturing the feature point screen; obtaining the perspective transformation matrix for image transformation from the first coordinate system of the captured image to the second coordinate system of the projector includes: detecting the coordinates of feature points in the feature point screen to obtain first coordinates; detecting the coordinates of feature points in the second image to obtain second coordinates; and performing coordinate transformation calculation based on the second coordinates and the first coordinates to obtain the perspective transformation matrix.

[0011] In some embodiments of this application, the step of extracting the projection area outside the captured image includes: extracting the area outside the feature point image area corresponding to the feature point image from the second image to obtain the projection area outside the image; or, extracting the area outside the pure white image area corresponding to the pure white image from the first image to obtain the projection area outside the image.

[0012] In some embodiments of this application, the step of extracting the region outside the feature point image area corresponding to the feature point image from the second image to obtain the region outside the projected image includes: performing linear fitting analysis based on the second coordinates to obtain the region corner coordinates of the feature point image area corresponding to the feature point image; and extracting the region outside the feature point image area from the second image based on the region corner coordinates of the feature point image area to obtain the region outside the projected image.

[0013] In some embodiments of this application, the first image feature includes the color with the highest area proportion within the projected image area, and the second image feature includes the grayscale value of the area outside the projected image area; the step of extracting features from the projected image area to obtain the first image feature representing the planar features of the plane located within the projected image area, and extracting features from the area outside the projected image area to obtain the second image feature representing the environmental features of the projection environment, includes: performing pixel color clustering analysis on the projected image area to obtain the color with the highest area proportion within the projected image area; and calculating the grayscale value of the area outside the projected image area to obtain the grayscale value of the area outside the projected image area.

[0014] In some embodiments of this application, adjusting the projection parameters of the projector based on the first image feature and the second image feature includes: obtaining a first parameter table; querying the first target color temperature and first target brightness corresponding to the color with the highest area ratio in the projected image area and the grayscale value of the area outside the projected image from the first parameter table to obtain a first parameter; and adjusting the color temperature and brightness of the projector to the first target color temperature and first target brightness based on the first parameter.

[0015] In some embodiments of this application, the first image feature includes a first feature map, and the second image feature includes a second feature map; the step of extracting features from the projected image area to obtain a first image feature representing the planar features of the plane located within the projected image area, and extracting features from the area outside the projected image to obtain a second image feature representing the environmental features of the projection environment, includes: using a first convolutional neural network to extract features from the projected image area to obtain the first feature map; and using a second convolutional neural network to extract features from the area outside the projected image to obtain the second feature map.

[0016] In some embodiments of this application, adjusting the projection parameters of the projector based on the first image feature and the second image feature includes: fusing the first feature map and the second feature map to obtain a fused feature; analyzing the fused feature using a parameter analysis model to obtain a second parameter; and adjusting the projection parameters of the projector to the second parameter based on the second parameter.

[0017] According to one embodiment of this application, a projection screen adjustment device includes: an image acquisition module, configured to: acquire a captured image of a test projection screen projected onto a plane by a projector; a region extraction module, configured to: extract a projection screen region from the captured image, and extract a region outside the projection screen from the captured image; a feature extraction module, configured to: extract features from the projection screen region to obtain a first image feature characterizing the planar features of the plane located within the projection screen region, and extract features from the region outside the projection screen to obtain a second image feature characterizing the environmental features of the projection environment; and a projection control module, configured to adjust the projection parameters of the projector according to the first image feature and the second image feature.

[0018] In some embodiments of this application, the test projection screen includes a pure white screen, and the captured image includes a first image obtained by capturing the pure white screen; the region extraction module is used to: obtain a perspective transformation matrix for image transformation from a first coordinate system of the captured image to a second coordinate system of the projector; and perform perspective transformation processing on the first image based on the perspective transformation matrix to obtain the projection screen region located in the second coordinate system.

[0019] In some embodiments of this application, the test projection screen further includes a feature point screen, and the captured image includes a second image obtained by capturing the feature point screen; the region extraction module is used to: detect the coordinates of feature points in the feature point screen to obtain first coordinates; detect the coordinates of feature points in the second image to obtain second coordinates; and perform coordinate transformation calculation based on the second coordinates and the first coordinates to obtain the perspective transformation matrix.

[0020] In some embodiments of this application, the region extraction module is used to: extract the region outside the feature point image region corresponding to the feature point image from the second image to obtain the region outside the projected image, or extract the region outside the pure white image region corresponding to the pure white image from the first image to obtain the region outside the projected image.

[0021] In some embodiments of this application, the region extraction module is used to: perform linear fitting analysis based on the second coordinates to obtain the region corner coordinates of the feature point image area corresponding to the feature point image; and extract the region outside the feature point image area from the second image based on the region corner coordinates of the feature point image area to obtain the region outside the projected image.

[0022] In some embodiments of this application, the first image feature includes the color with the highest area proportion within the projected image area, and the second image feature includes the grayscale value of the area outside the projected image area; the feature extraction module is used to: perform pixel color clustering analysis on the projected image area to obtain the color with the highest area proportion within the projected image area; and calculate the grayscale value of the area outside the projected image area to obtain the grayscale value of the area outside the projected image area.

[0023] In some embodiments of this application, the projection control module is configured to: obtain a first parameter table; query the first target color temperature and first target brightness corresponding to the color with the highest area ratio in the projection screen area and the grayscale value of the area outside the projection screen from the first parameter table to obtain a first parameter; and adjust the color temperature and brightness of the projector to the first target color temperature and first target brightness according to the first parameter.

[0024] In some embodiments of this application, the first image feature includes a first feature map, and the second image feature includes a second feature map; the feature extraction module is configured to: use a first convolutional neural network to extract features from the projected image area to obtain the first feature map; and use a second convolutional neural network to extract features from the area outside the projected image to obtain the second feature map.

[0025] In some embodiments of this application, the projection control module is configured to: fuse the first feature map and the second feature map to obtain a fused feature; analyze the fused feature using a parameter analysis model to obtain a second parameter; and adjust the projection parameters of the projector to the second parameter according to the second parameter.

[0026] According to another embodiment of this application, a storage medium stores a computer program thereon, which, when executed by a device's processor, causes the device to perform the methods described in the embodiments of this application.

[0027] According to another embodiment of this application, an apparatus may include: a memory storing a computer program; and a processor reading the computer program stored in the memory to execute the methods described in the embodiments of this application.

[0028] According to another embodiment of this application, a computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of the device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the device to perform the methods provided in the various optional implementations described in the embodiments of this application. Beneficial effects

[0029] In this embodiment, a photographic image is obtained by capturing a test projection image projected onto a plane by a projector; the projected image area and the area outside the projected image are extracted from the photographic image; a first image feature is obtained by extracting features from the projected image area to represent the planar features of the plane located within the projected image area, and a second image feature is obtained by extracting features from the area outside the projected image to represent the environmental features of the projection environment; the projection parameters of the projector are adjusted according to the first image feature and the second image feature.

[0030] In this way, by capturing images of the test projection screen of the projector, the projected screen area and the area outside the projected screen are extracted from the captured images. First image features of the projected screen area and second image features of the area outside the projected screen are extracted respectively. The projection parameters of the projector are adjusted according to the first and second image features. By combining the image features of the projected screen area and the area outside the projected screen through capturing the projected image, the projection effect can be effectively improved, enhancing the user's viewing experience. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 shows a flowchart of a projection screen adjustment method according to an embodiment of this application.

[0033] Figure 2 shows a projected pure white image according to an embodiment of this application.

[0034] Figure 3 shows an image captured according to an embodiment of this application.

[0035] Figure 4 shows a projected image according to an embodiment of this application.

[0036] Figure 5 shows a projection of feature points according to an embodiment of this application.

[0037] Figure 6 shows an image captured according to another embodiment of this application.

[0038] Figure 7 shows a block diagram of a projection screen adjustment device according to an embodiment of this application.

[0039] Figure 8 shows a block diagram of a device according to an embodiment of this application.

[0040] Implementation methods of this application

[0041] The present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments provided herein are merely illustrative of the present disclosure and are not intended to limit the present disclosure. Furthermore, the embodiments provided below are some embodiments for implementing the present disclosure, and not all embodiments for implementing the present disclosure. Unless otherwise specified, the technical solutions described in the embodiments of the present disclosure can be implemented in any combination.

[0042] It should be noted that, in the embodiments of this disclosure, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a method or apparatus that includes a list of elements includes not only the elements expressly described, but also other elements not expressly listed, or elements inherent to implementing the method or apparatus. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other related elements (e.g., steps in the method or units in the apparatus, such as portions of circuitry, processors, programs, or software, etc.) in the method or apparatus that includes that element.

[0043] For example, the projection screen adjustment method provided in this embodiment includes a series of steps, but the projection screen adjustment method provided in this embodiment is not limited to the steps described. Similarly, the projection screen adjustment device provided in this embodiment includes a series of units, but the device provided in this embodiment is not limited to the units explicitly described, and may also include units that need to be set for obtaining relevant information or processing based on information.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure.

[0045] It is understood that in the specific implementation of this application, relevant data is involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0046] Figure 1 schematically illustrates a flowchart of a projection screen adjustment method according to an embodiment of this application. The execution entity of this projection screen adjustment method can be any device or server with processing capabilities used to control the projector. The device can be, for example, a computer, a control module within the projector, a mobile phone, a smartwatch, or a home appliance. The server can be, for example, a physical server or a cloud server. In one specific embodiment of this application, the execution entity of the projection screen adjustment method is specifically a control module within the projector used to control the projector.

[0047] As shown in Figure 1, the projection screen adjustment method may include steps S110 to S140.

[0048] Step S110: Acquire a captured image of the test projection screen projected onto the plane by the projector; Step S120: Extract the projection screen area and the area outside the projection screen from the captured image; Step S130: Extract features from the projection screen area to obtain a first image feature representing the planar features of the plane located within the projection screen area, and extract features from the area outside the projection screen to obtain a second image feature representing the environmental features of the projection environment; Step S140: Adjust the projection parameters of the projector according to the first image feature and the second image feature.

[0049] After the projector projects the test screen onto the plane, a camera can capture the test screen to obtain the captured image, which can then be obtained by the executing entity.

[0050] The captured image includes the area where the test projection screen is located (i.e., the projection screen area) and the area outside the projection screen area (i.e., the area outside the projection screen). The executing entity can then extract the projection screen area and the area outside the projection screen from the captured image. Feature extraction is performed on the projection screen area to obtain first image features, and feature extraction is performed on the area outside the projection screen to obtain second image features. The first image features accurately characterize the planar features of a plane (such as a wall) located within the projection screen area, while the second image features characterize environmental features such as ambient brightness of the projection environment.

[0051] In some methods, the first image feature refers to the color with the highest area proportion within the statistically obtained projection image area, and the second image feature is the calculated grayscale value of the area outside the projection image. Alternatively, in other methods, the first image feature is the feature map of the projection image area extracted by a first convolutional neural network, and the second image feature is the feature map of the area outside the projection image extracted by a second convolutional neural network.

[0052] Based on the first and second image features, the corresponding projection parameters can be determined, and these parameters can be used to adjust the projector's projection parameters, thereby improving the projection effect and viewing experience. These projection parameters may include, but are not limited to, brightness and color temperature.

[0053] In this way, by capturing images of the test projection screen of the projector, the projected screen area and the area outside the projected screen are extracted from the captured images. The first image feature of the projected screen area and the second image feature of the area outside the projected screen area are extracted respectively. The projection parameters of the projector are adjusted according to the first and second image features. By combining the image features of the projected screen area and the area outside the projected screen area (i.e., the features of the plane (such as the wall) in the projected screen area and the environmental features of the projection environment) through capturing the projected screen, the projection effect can be effectively improved, thus enhancing the user's viewing experience.

[0054] The following describes further optional embodiments of the steps performed during projection in the embodiment shown in Figure 1.

[0055] In one embodiment, the test projection screen includes a pure white screen, and the captured image includes a first image obtained by capturing the pure white screen; the step of extracting the projection screen region from the captured image includes: obtaining a perspective transformation matrix for transforming the image from a first coordinate system of the captured image to a second coordinate system of the projector; and performing perspective transformation processing on the first image based on the perspective transformation matrix to obtain the projection screen region located in the second coordinate system.

[0056] For example, as shown in Figure 2, a pure white image 210 (i.e., the rectangular image corresponding to A1, B1, C1 and D1) is projected onto a plane by a projector. A first image 310 as shown in Figure 3 is obtained by taking a picture of the pure white image with a camera. The first image 310 includes the area 311 corresponding to the pure white image in the first coordinate system (i.e., the area within the small frame corresponding to A1, B1, C1 and D1) and the area outside the image 312 (i.e., the area outside the small frame).

[0057] Specifically, the first image is captured by a camera. Therefore, the first coordinate system can specifically refer to the image coordinate system corresponding to the camera, and the second coordinate system can specifically refer to the optical-mechanical coordinate system corresponding to the projector. A perspective transformation matrix is ​​obtained to transform the image from the image coordinate system of the captured image to the optical-mechanical coordinate system of the projector. Based on this perspective transformation matrix, the first image is subjected to perspective transformation processing, thus obtaining the projected image area located in the optical-mechanical coordinate system. For example, as shown in Figure 4, the perspective transformation processing yields only the area 410 corresponding to the pure white image located in the optical-mechanical coordinate system (i.e., the projected image area located in the second coordinate system).

[0058] A first image feature is extracted from the projection image area located in the second coordinate system. This first image feature can reflect the projection effect from the corner point of the projector. This first image feature is used in the relevant embodiments of this application to further improve the projection control effect.

[0059] In one embodiment, the test projection screen further includes a feature point screen, and the captured image includes a second image obtained by capturing the feature point screen; obtaining the perspective transformation matrix for image transformation from the first coordinate system of the captured image to the second coordinate system of the projector includes: detecting the coordinates of feature points in the feature point screen to obtain first coordinates; detecting the coordinates of feature points in the second image to obtain second coordinates; and performing coordinate transformation calculation based on the second coordinates and the first coordinates to obtain the perspective transformation matrix.

[0060] For example, Figure 5 shows a feature point image 510 (i.e., the rectangular images corresponding to A2, B2, C2, and D2) projected onto a plane by a projector. A second image 610, as shown in Figure 6, is obtained by capturing this feature point image with a camera. This second image 610 includes the region 611 corresponding to the feature point image in the first coordinate system (i.e., the region within the small frames corresponding to A2, B2, C2, and D2) and the region 612 outside the feature point image (i.e., the region outside the small frames). In this example, the feature point image includes a checkerboard pattern, with every four squares forming a 2x2 square. The center point of this square is the feature point (also referred to as an internal corner point).

[0061] The second image was captured by a camera; therefore, it lies in the first coordinate system corresponding to the camera. By detecting feature points in the original image projected onto the projector to obtain the feature point image, the coordinates of these feature points in the feature point image can be obtained, which are also the first coordinates in the projector's second coordinate system. Detecting the coordinates of feature points in the second image yields the second coordinates in the first coordinate system. For example, the `findChessboardCorners` function from OpenCV (a computer vision image processing library) can be used to detect both the first and second coordinates.

[0062] By performing coordinate transformation calculations based on the second and first coordinates, the perspective transformation matrix used for image transformation from the first coordinate system of the captured image to the second coordinate system of the projector can be accurately obtained.

[0063] In one example, the perspective transformation matrix is ​​calculated by performing a coordinate transformation based on the second coordinate and the first coordinate. Specifically, the calculation can be performed using the following formula:

[0064] That is,

[0065] Where u and v can represent the second coordinates, and x and y can represent the first coordinates, solving this formula yields a 3x3 perspective transformation matrix, which can be accurately used to transform the image from the first coordinate system of the captured image to the second coordinate system of the projector.

[0066] In one embodiment, extracting the region outside the projected image from the captured image includes: extracting the region outside the feature point image region corresponding to the feature point image from the second image to obtain the region outside the projected image; or, extracting the region outside the pure white image region corresponding to the pure white image from the first image to obtain the region outside the projected image.

[0067] The area outside the feature point area corresponding to the feature point image in the second image, or the area outside the pure white image area corresponding to the pure white image in the first image, can be used as the area outside the projection image. Extracting the features of the second image based on the area outside the projection image can reflect the characteristics of the projection environment, such as the ambient brightness.

[0068] Furthermore, in one embodiment, the step of extracting the region outside the feature point image area corresponding to the feature point image from the second image to obtain the region outside the projected image may specifically include: performing linear fitting analysis based on the second coordinates to obtain the region corner coordinates of the feature point image area corresponding to the feature point image; and extracting the region outside the feature point image area from the second image based on the region corner coordinates of the feature point image area to obtain the region outside the projected image.

[0069] For example, as shown in Figure 6, by performing linear fitting analysis based on the second coordinates of the feature points, the coordinates of the corner points of the corresponding feature point image region (i.e., the coordinates of A2, B2, C2, and D2) can be obtained. Furthermore, based on these corner point coordinates, the area outside the feature point image region formed by A2, B2, C2, and D2 (i.e., the area outside the projected image) can be cropped out. Since the second image includes feature points, the area outside the projected image can be extracted more conveniently from the second image through fitting.

[0070] In one embodiment, the first image feature includes the color with the highest area proportion within the projected image area, and the second image feature includes the grayscale value of the area outside the projected image area. The step of extracting features from the projected image area to obtain the first image feature representing the planar features of the plane located within the projected image area, and extracting features from the area outside the projected image area to obtain the second image feature representing the environmental features of the projection environment, includes: performing pixel color clustering analysis on the projected image area to obtain the color with the highest area proportion within the projected image area; and calculating the grayscale value of the area outside the projected image area to obtain the grayscale value of the area outside the projected image area.

[0071] Accordingly, adjusting the projection parameters of the projector based on the first image feature and the second image feature may include: obtaining a first parameter table; querying the first target color temperature and first target brightness corresponding to the color with the highest area ratio in the projected image area and the grayscale value of the area outside the projected image from the first parameter table to obtain a first parameter; and adjusting the color temperature and brightness of the projector to the first target color temperature and first target brightness based on the first parameter.

[0072] In this embodiment, pixel color clustering analysis is performed on the projected image area to obtain the color with the highest area proportion within the projected image area, which is used as the first image feature. The color with the highest area proportion within the projected image area can effectively reflect the color characteristics of planes (such as walls) within the projected image area. Grayscale values ​​are calculated for the area outside the projected image area to obtain the grayscale values ​​of the area outside the projected image area, which are used as the second image feature. The grayscale values ​​of the area outside the projected image area can effectively reflect the ambient brightness of the projection environment.

[0073] The first parameter is obtained by querying the color with the highest area ratio in the projection screen area and the grayscale value of the area outside the projection screen area from the first parameter table, corresponding to the first target color temperature and first target brightness. The color temperature and brightness of the projector can be adjusted according to the first parameter. Adjusting the color temperature and brightness of the projector to the first target color temperature and first target brightness, and effectively taking into account the color temperature characteristics of the plane (such as the wall) in the projection screen area and the ambient brightness of the projection environment, can effectively improve the projection effect and enhance the user's viewing experience.

[0074] The projector system has a color temperature adjustment interface, allowing the projector's color temperature parameters to be adjusted based on the color temperature information retrieved from the first parameter table. For example, if green is the color with the largest area coverage in the projected image (e.g., green wallpaper on the user's wall) and its RGB values ​​are (40, 160, 40), the projector's R color temperature can be increased to X times its current R color temperature, and the projector's B color temperature can be increased to X times its current B color temperature. This will prevent the projected image from appearing as greenish as before. Furthermore, if the grayscale value of areas outside the projected image is very low, the projector's brightness can be reduced based on the brightness information retrieved from the first parameter table; conversely, the projector's brightness can be increased.

[0075] When performing pixel color clustering analysis on the projected image area to obtain the color with the highest area proportion in the projected image area, taking the K-means algorithm for clustering as an example, it may include steps (1) to (6).

[0076] (1) Determine the number K of clusters to be divided into for the "color data of pixels in the projection area". For example, K can be 5. (2) Initialize cluster centers: randomly select K pixel color values ​​as initial cluster centers. (3) Assign data points: assign the color value of each pixel in the "color data of pixels in the projection area" to the nearest cluster center. Euclidean distance can be used to calculate the distance between the color value of each pixel and the cluster center. The color value of each pixel is assigned to the cluster that is closest to the cluster center. (4) Update cluster centers: calculate the average value of all data in each cluster and update the cluster center of each cluster to the average value calculated for the corresponding cluster. (5) Repeat the above operations (3) and (4) until the cluster center of each cluster no longer changes. Then output the final cluster center and the number of pixel color values ​​included in each cluster. (6) Determine the cluster center of the cluster with the most color values ​​as the color with the highest area proportion in the projection area.

[0077] In one embodiment, the first image feature includes a first feature map, and the second image feature includes a second feature map; the step of extracting features from the projected image area to obtain a first image feature representing the planar features of the plane located within the projected image area, and extracting features from the area outside the projected image to obtain a second image feature representing the environmental features of the projection environment, includes: using a first convolutional neural network to extract features from the projected image area to obtain the first feature map; and using a second convolutional neural network to extract features from the area outside the projected image to obtain the second feature map.

[0078] Accordingly, adjusting the projection parameters of the projector based on the first image feature and the second image feature may include: fusing the first feature map and the second feature map to obtain a fused feature; analyzing the fused feature using a parameter analysis model to obtain a second parameter; and adjusting the projection parameters of the projector to the second parameter based on the second parameter.

[0079] In this embodiment, a first convolutional neural network is used to extract features from the projected image area, resulting in a first feature map as the first image feature. This first feature map can reflect the detailed features of planes (such as walls) within the projected image area in more detail. A second convolutional neural network is used to extract features from the area outside the projected image, resulting in a second feature map as the second image feature. This second feature map can reflect the environmental features of the projection environment in more detail.

[0080] The first feature map and the second feature map are fused to obtain the fused feature; the fused feature is analyzed using a parametric analysis model to obtain the second parameter; based on the second parameter, the projector can be adjusted in more detail to further improve the projection effect.

[0081] One method for fusing the first and second feature maps to obtain the fused feature is to concatenate the first and second feature maps. Alternatively, another method is to fuse the first and second feature maps using an attention mechanism. This fused feature, obtained through an attention mechanism, is then used in a parameter analysis model to more accurately determine the second parameter. The parameter analysis model can be a pre-trained convolutional neural network model used for projector parameter analysis based on the fused feature, or it can be a large model that has been fine-tuned and trained.

[0082] Specifically, in one embodiment, before adjusting the projection parameters of the projector based on the first and second image features, the method may further include: acquiring fused feature samples and parameter samples corresponding to the fused features; feeding the fused feature samples and parameter samples corresponding to the fused features into a selected large model for learning to obtain model parameters; and adding the model parameters to the selected large model to obtain a fine-tuned, trained large model. Here, the fused feature samples are the fused features collected as model training samples, and the parameter samples are the parameters collected as model training labels. The selected large model can be a Large Oracle Model (LLM) based on actual needs.

[0083] To facilitate better implementation of the projection screen adjustment method provided in the embodiments of this application, the following describes the content of the foregoing embodiments in conjunction with a process for adjusting the projection screen in a specific scenario. In this scenario, by applying the foregoing embodiments of this application to adjust the projection screen, the projection effect is effectively improved. The meanings of the terms are the same as in the above embodiments, and can be found in the descriptions in the foregoing embodiments.

[0084] In this scenario, the process of adjusting the projected image using the aforementioned embodiments of this application may include steps (1) to (12).

[0085] (1) Project test projection screen, wherein the test projection screen includes a pure white screen and a feature point screen;

[0086] (2) The captured image is obtained by taking a picture with a camera, wherein the captured image includes a first image obtained by capturing the pure white screen and a second image obtained by capturing the feature point screen;

[0087] (3) Extract the projected image area from the captured image, specifically:

[0088] Obtain a perspective transformation matrix for transforming the image from a first coordinate system of the captured image to a second coordinate system of the projector; and perform perspective transformation processing on the first image based on the perspective transformation matrix to obtain the projected image area located in the second coordinate system.

[0089] Specifically, obtaining the perspective transformation matrix used for image transformation from the first coordinate system of the captured image to the second coordinate system of the projector includes: detecting the coordinates of feature points in the feature point image to obtain the first coordinates; detecting the coordinates of feature points in the second image to obtain the second coordinates; and performing coordinate transformation calculations based on the second coordinates and the first coordinates to obtain the perspective transformation matrix.

[0090] (4) Extract the area outside the projected image from the captured image, specifically:

[0091] The region outside the feature point area corresponding to the feature point image in the second image is extracted to obtain the region outside the projection image; or, the region outside the pure white image area corresponding to the pure white image in the first image is extracted to obtain the region outside the projection image.

[0092] Specifically, extracting the region outside the feature point image area corresponding to the feature point image from the second image to obtain the region outside the projection image can include: performing linear fitting analysis based on the second coordinates to obtain the region corner coordinates of the feature point image area corresponding to the feature point image; and extracting the region outside the feature point image area from the second image based on the region corner coordinates of the feature point image area to obtain the region outside the projection image.

[0093] (5) Extracting features from the projected image area to obtain first image features representing the planar features of the plane located within the projected image area, and extracting features from the area outside the projected image to obtain second image features representing the environmental features of the projection environment, which may specifically include:

[0094] Perform pixel color clustering analysis on the projected image area to obtain the color with the highest area ratio within the projected image area; calculate the grayscale value of the area outside the projected image to obtain the grayscale value of the area outside the projected image.

[0095] (6) Adjust the projection parameters of the projector according to the characteristics of the first and second images, which may include:

[0096] Obtain the first parameter table; query the first target color temperature and first target brightness corresponding to the color with the highest area ratio in the projected image area and the grayscale value of the area outside the projected image area from the first parameter table to obtain the first parameter; adjust the color temperature and brightness of the projector to the first target color temperature and first target brightness according to the first parameter.

[0097] In this scenario, by comprehensively applying the aforementioned embodiments of this application to adjust the projected image, the projector can be controlled by combining the image features of both the projected image area and the area outside the projected image through the method of capturing the projected image. This can effectively improve the projection effect and enhance the user's viewing experience.

[0098] To facilitate better implementation of the projection screen adjustment method provided in the embodiments of this application, the embodiments of this application also provide a projection screen adjustment device based on the above-described projection screen adjustment method. The meanings of the terms used are the same as in the above-described projection screen adjustment method, and specific implementation details can be found in the description of the method embodiments. Figure 7 shows a block diagram of a projection screen adjustment device according to an embodiment of this application.

[0099] As shown in Figure 7, the projection screen adjustment device 700 may include: an image acquisition module 710, which can be used to: acquire a captured image of a test projection screen projected onto a plane by a projector; a region extraction module 720, which can be used to: extract the projection screen region from the captured image, and extract the region outside the projection screen from the captured image; a feature extraction module 730, which can be used to: extract features from the projection screen region to obtain a first image feature representing the planar features of the plane located within the projection screen region, and extract features from the region outside the projection screen to obtain a second image feature representing the environmental features of the projection environment; and a projection control module 740, which can be used to adjust the projection parameters of the projector according to the first image feature and the second image feature.

[0100] In some embodiments of this application, the test projection screen includes a pure white screen, and the captured image includes a first image obtained by capturing the pure white screen; the region extraction module is used to: obtain a perspective transformation matrix for image transformation from a first coordinate system of the captured image to a second coordinate system of the projector; and perform perspective transformation processing on the first image based on the perspective transformation matrix to obtain the projection screen region located in the second coordinate system.

[0101] In some embodiments of this application, the test projection screen further includes a feature point screen, and the captured image includes a second image obtained by capturing the feature point screen; the region extraction module is used to: detect the coordinates of feature points in the feature point screen to obtain first coordinates; detect the coordinates of feature points in the second image to obtain second coordinates; and perform coordinate transformation calculation based on the second coordinates and the first coordinates to obtain the perspective transformation matrix.

[0102] In some embodiments of this application, the region extraction module is used to: extract the region outside the feature point image region corresponding to the feature point image from the second image to obtain the region outside the projected image, or extract the region outside the pure white image region corresponding to the pure white image from the first image to obtain the region outside the projected image.

[0103] In some embodiments of this application, the region extraction module is used to: perform linear fitting analysis based on the second coordinates to obtain the region corner coordinates of the feature point image area corresponding to the feature point image; and extract the region outside the feature point image area from the second image based on the region corner coordinates of the feature point image area to obtain the region outside the projected image.

[0104] In some embodiments of this application, the first image feature includes the color with the highest area proportion within the projected image area, and the second image feature includes the grayscale value of the area outside the projected image area; the feature extraction module is used to: perform pixel color clustering analysis on the projected image area to obtain the color with the highest area proportion within the projected image area; and calculate the grayscale value of the area outside the projected image area to obtain the grayscale value of the area outside the projected image area.

[0105] In some embodiments of this application, the projection control module is configured to: obtain a first parameter table; query the first target color temperature and first target brightness corresponding to the color with the highest area ratio in the projection screen area and the grayscale value of the area outside the projection screen from the first parameter table to obtain a first parameter; and adjust the color temperature and brightness of the projector to the first target color temperature and first target brightness according to the first parameter.

[0106] In some embodiments of this application, the first image feature includes a first feature map, and the second image feature includes a second feature map; the feature extraction module is configured to: use a first convolutional neural network to extract features from the projected image area to obtain the first feature map; and use a second convolutional neural network to extract features from the area outside the projected image to obtain the second feature map.

[0107] In some embodiments of this application, the projection control module is configured to: fuse the first feature map and the second feature map to obtain a fused feature; analyze the fused feature using a parameter analysis model to obtain a second parameter; and adjust the projection parameters of the projector to the second parameter according to the second parameter.

[0108] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0109] Furthermore, this application also provides a device, as shown in FIG8, which is a block diagram of a device according to an embodiment of this application. Specifically:

[0110] The device may include components such as a processor 801 with one or more processing cores, a memory 802 with one or more computer-readable storage media, a power supply 803, and an input unit 804. Those skilled in the art will understand that the device structure shown in FIG8 does not constitute a limitation on the device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0111] The processor 801 is the control center of the device, connecting various parts of the computer device through various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 802, and by calling data stored in the memory 802, thereby providing overall monitoring of the device. Optionally, the processor 801 may include one or more processing cores; preferably, the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user page, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 801.

[0112] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 802 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0113] The device also includes a power supply 803 that supplies power to the various components. Preferably, the power supply 803 can be logically connected to the processor 801 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 803 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0114] The device may also include an input unit 804, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0115] Although not shown, the device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 801 in the device loads the executable files corresponding to the processes of one or more computer programs into the memory 802 according to the following instructions, and the processor 801 runs the computer programs stored in the memory 802, thereby realizing the various functions in the foregoing embodiments of this application. For example, the processor 801 can perform the following steps:

[0116] A photographic image is obtained by capturing a test projection image projected onto a plane by a projector; the projected image area and the area outside the projected image are extracted from the photographic image; a first image feature is obtained by performing feature extraction on the projected image area to obtain a planar feature representing the plane located within the projected image area, and a second image feature is obtained by performing feature extraction on the area outside the projected image to obtain an environmental feature representing the projection environment; the projection parameters of the projector are adjusted according to the first image feature and the second image feature.

[0117] In some embodiments of this application, the test projection screen includes a pure white screen, and the captured image includes a first image obtained by capturing the pure white screen; the step of extracting the projection screen region from the captured image includes: obtaining a perspective transformation matrix for transforming the image from a first coordinate system of the captured image to a second coordinate system of the projector; and performing perspective transformation processing on the first image based on the perspective transformation matrix to obtain the projection screen region located in the second coordinate system.

[0118] In some embodiments of this application, the test projection screen further includes a feature point screen, and the captured image includes a second image obtained by capturing the feature point screen; obtaining the perspective transformation matrix for image transformation from the first coordinate system of the captured image to the second coordinate system of the projector includes: detecting the coordinates of feature points in the feature point screen to obtain first coordinates; detecting the coordinates of feature points in the second image to obtain second coordinates; and performing coordinate transformation calculation based on the second coordinates and the first coordinates to obtain the perspective transformation matrix.

[0119] In some embodiments of this application, the step of extracting the projection area outside the captured image includes: extracting the area outside the feature point image area corresponding to the feature point image from the second image to obtain the projection area outside the image; or, extracting the area outside the pure white image area corresponding to the pure white image from the first image to obtain the projection area outside the image.

[0120] In some embodiments of this application, the step of extracting the region outside the feature point image area corresponding to the feature point image from the second image to obtain the region outside the projected image includes: performing linear fitting analysis based on the second coordinates to obtain the region corner coordinates of the feature point image area corresponding to the feature point image; and extracting the region outside the feature point image area from the second image based on the region corner coordinates of the feature point image area to obtain the region outside the projected image.

[0121] In some embodiments of this application, the first image feature includes the color with the highest area proportion within the projected image area, and the second image feature includes the grayscale value of the area outside the projected image area; the step of extracting features from the projected image area to obtain the first image feature representing the planar features of the plane located within the projected image area, and extracting features from the area outside the projected image area to obtain the second image feature representing the environmental features of the projection environment, includes: performing pixel color clustering analysis on the projected image area to obtain the color with the highest area proportion within the projected image area; and calculating the grayscale value of the area outside the projected image area to obtain the grayscale value of the area outside the projected image area.

[0122] In some embodiments of this application, adjusting the projection parameters of the projector based on the first image feature and the second image feature includes: obtaining a first parameter table; querying the first target color temperature and first target brightness corresponding to the color with the highest area ratio in the projected image area and the grayscale value of the area outside the projected image from the first parameter table to obtain a first parameter; and adjusting the color temperature and brightness of the projector to the first target color temperature and first target brightness based on the first parameter.

[0123] In some embodiments of this application, the first image feature includes a first feature map, and the second image feature includes a second feature map; the step of extracting features from the projected image area to obtain a first image feature representing the planar features of the plane located within the projected image area, and extracting features from the area outside the projected image to obtain a second image feature representing the environmental features of the projection environment, includes: using a first convolutional neural network to extract features from the projected image area to obtain the first feature map; and using a second convolutional neural network to extract features from the area outside the projected image to obtain the second feature map.

[0124] In some embodiments of this application, adjusting the projection parameters of the projector based on the first image feature and the second image feature includes: fusing the first feature map and the second feature map to obtain a fused feature; analyzing the fused feature using a parameter analysis model to obtain a second parameter; and adjusting the projection parameters of the projector to the second parameter based on the second parameter.

[0125] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a computer program, or by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0126] Therefore, embodiments of this application also provide a storage medium storing a computer program that can be loaded by a processor to execute the steps in any of the methods provided in embodiments of this application.

[0127] The storage medium can be a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0128] Since the computer program stored in the storage medium can execute the steps of any of the methods provided in the embodiments of this application, the beneficial effects that the methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0129] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0130] It should be understood that this application is not limited to the embodiments described above and shown in the accompanying drawings, but various modifications and changes can be made without departing from its scope.

Claims

1. A method for adjusting a projected image, wherein, include: Acquire images of the test projection screen projected onto a flat surface by a projector; Extract the projected image area from the captured image, and extract the area outside the projected image from the captured image; Feature extraction is performed on the projected image area to obtain a first image feature that characterizes the planar features of the plane located within the projected image area, and feature extraction is performed on the area outside the projected image to obtain a second image feature that characterizes the environmental features of the projection environment; The projection parameters of the projector are adjusted based on the first and second image features.

2. The method according to claim 1, wherein, The test projection screen includes a pure white screen, and the captured image includes a first image obtained by capturing the pure white screen; Extracting the projected image area from the captured image includes: Obtain the perspective transformation matrix used to transform the image from the first coordinate system of the captured image to the second coordinate system of the projector; The first image is subjected to perspective transformation processing based on the perspective transformation matrix to obtain the projected image area located in the second coordinate system.

3. The method according to claim 2, wherein, The test projection screen also includes a feature point screen, and the captured image includes a second image obtained by capturing the feature point screen; The step of obtaining the perspective transformation matrix for image transformation from the first coordinate system of the captured image to the second coordinate system of the projector includes: The coordinates of the feature points in the feature point image are detected to obtain the first coordinates; The coordinates of feature points in the second image are detected to obtain the second coordinates; The perspective transformation matrix is ​​obtained by performing coordinate transformation calculations based on the second coordinate and the first coordinate.

4. The method according to claim 3, wherein, Extracting the area outside the projected image from the captured image includes: The region outside the feature point image area corresponding to the feature point image is extracted from the second image to obtain the region outside the projected image; or, the region outside the pure white image area corresponding to the pure white image is extracted from the first image to obtain the region outside the projected image.

5. The method according to claim 4, wherein, The step of extracting the region outside the feature point image area corresponding to the feature point image from the second image to obtain the region outside the projected image includes: Based on the second coordinates, a linear fitting analysis is performed to obtain the coordinates of the corner points of the feature point image area corresponding to the feature point image. Based on the coordinates of the corner points of the feature point image area, the area outside the feature point image area is extracted from the second image to obtain the area outside the projected image.

6. The method according to claim 1, wherein, The first image feature includes the color with the highest area proportion within the projected image area, and the second image feature includes the grayscale value of the area outside the projected image. The first image feature, which extracts features from the projected image area to obtain planar features representing the plane located within the projected image area, and the second image feature, which extracts features from the area outside the projected image to obtain environmental features representing the projection environment, includes: Perform pixel color clustering analysis on the projected image area to obtain the color with the highest area ratio in the projected image area; The grayscale value of the area outside the projected image is calculated to obtain the grayscale value of the area outside the projected image.

7. The method according to claim 6, wherein, The step of adjusting the projection parameters of the projector based on the first image feature and the second image feature includes: Get the first parameter table; The first parameter is obtained by querying the color with the highest area ratio in the projected image area and the grayscale value of the area outside the projected image from the first parameter table; The color temperature and brightness of the projector are adjusted to the first target color temperature and the first target brightness according to the first parameter.

8. The method according to claim 1, wherein, The first image feature includes a first feature map, and the second image feature includes a second feature map; The first image feature, which extracts features from the projected image area to obtain planar features representing the plane located within the projected image area, and the second image feature, which extracts features from the area outside the projected image to obtain environmental features representing the projection environment, includes: The first feature map is obtained by extracting features from the projected image area using a first convolutional neural network. A second convolutional neural network is used to extract features from the area outside the projected image to obtain a second feature map.

9. The method according to claim 8, wherein, The step of adjusting the projection parameters of the projector based on the first image feature and the second image feature includes: The first feature map and the second feature map are fused to obtain the fused feature; The fusion features are analyzed using a parametric analysis model to obtain the second parameter; The projection parameters of the projector are adjusted to the second parameter according to the second parameter.

10. The method according to claim 6, wherein, The step of performing pixel color clustering analysis on the projected image area to obtain the color with the highest area proportion within the projected image area includes: Determine the number of clusters K; Randomly select K pixel color values ​​as the initial cluster centers; Data points are assigned to assign the color value of each pixel in the color data of the pixels within the projected image area to the nearest cluster center; Update the cluster centers to calculate the average of all data in each cluster, and update the cluster centers of each cluster to the average calculated for that cluster. Repeat the operations of allocating data points and updating cluster centers until the cluster center of each cluster no longer changes, then output the final cluster center and the number of color values ​​of the pixels included in each cluster; The cluster center of the cluster containing the most color values ​​is determined as the color with the highest area proportion within the projected image area.

11. The method according to claim 9, wherein, The step of fusing the first feature map and the second feature map to obtain the fused feature includes: splicing the first feature map and the second feature map together to obtain the fused feature.

12. The method according to claim 9, wherein, The step of fusing the first feature map and the second feature map to obtain fused features includes: fusing the first feature map and the second feature map through an attention mechanism to obtain fused features.

13. The method according to claim 9, wherein, The parameter analysis model includes a large model after fine-tuning and training of a large model; Before adjusting the projection parameters of the projector based on the first image feature and the second image feature, the method further includes: Obtain the fused feature samples and the corresponding parameter samples of the fused features; The fused feature samples and the corresponding parameter samples are fed into the selected large model for learning to obtain model parameters; the model parameters are then added to the selected large model to obtain the fine-tuned large model.

14. A projection screen adjustment device, wherein, include: The image acquisition module is used to: acquire images obtained by taking pictures of the test projection screen projected onto the plane by the projector; The region extraction module is used to: extract the projection area from the captured image, and extract the area outside the projection area from the captured image; The feature extraction module is used to: extract features from the projected image area to obtain a first image feature that characterizes the planar features of the plane located within the projected image area, and extract features from the area outside the projected image to obtain a second image feature that characterizes the environmental features of the projection environment; The projection control module is used to adjust the projection parameters of the projector based on the first image features and the second image features.

15. The apparatus according to claim 14, wherein, The test projection screen includes a pure white screen, and the captured image includes a first image obtained by capturing the pure white screen; the region extraction module is used to: obtain a perspective transformation matrix for image transformation from the first coordinate system of the captured image to the second coordinate system of the projector; The first image is subjected to perspective transformation processing based on the perspective transformation matrix to obtain the projected image area located in the second coordinate system.

16. The apparatus according to claim 15, wherein, The test projection screen also includes a feature point screen, and the captured image includes a second image obtained by capturing the feature point screen; the region extraction module is used to: detect the coordinates of feature points in the feature point screen to obtain a first coordinate; detect the coordinates of feature points in the second image to obtain a second coordinate; and perform coordinate transformation calculation based on the second coordinate and the first coordinate to obtain the perspective transformation matrix.

17. The apparatus according to claim 16, wherein, The region extraction module is used to: extract the region outside the feature point image region corresponding to the feature point image from the second image to obtain the region outside the projected image, or extract the region outside the pure white image region corresponding to the pure white image from the first image to obtain the region outside the projected image.

18. The apparatus according to claim 17, wherein, The region extraction module is used to: perform linear fitting analysis based on the second coordinates to obtain the region corner coordinates of the feature point image area corresponding to the feature point image; and extract the region outside the feature point image area from the second image based on the region corner coordinates of the feature point image area to obtain the region outside the projected image.

19. A storage medium, wherein, It stores a computer program that, when executed by the device's processor, causes the device to perform the method described in any one of claims 1 to 13.

20. A device, wherein, include: Memory, which stores computer programs; A processor reads a computer program stored in memory to perform the method described in any one of claims 1 to 13.

Images