Projection control method and projection device

Abstract

The present application relates to a projection control method and a projection device. The method comprises: when an instruction to restore a memory position of a projection device has been received, acquiring first environmental features of the current position of the projection device; on the basis of the matching between the first environmental features and second environmental features of the memory position, determining target environmental features in the second environmental features that match the first environmental features, wherein the first environmental features and the second environmental features are generated on the basis of multi-sensor data collected from a corresponding projection environment; on the basis of the first environmental features and the target environmental features, determining adjustment information of the projection device; and on the basis of the adjustment information, driving the projection device to rotate spatially, such that a projected image of the projection device is projected onto a memory position corresponding to the target environmental features. By means of the present application, a projected image of a projection device can be accurately restored after the position of the projection device changes.

Description

Projection control method and projection device

[0001] This application claims priority to the Chinese patent application No. 202411377711.0, filed on September 29, 2024, and entitled "Projection control method and projection device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of projection technology, in particular, to a projection control method and a projection device. BACKGROUND

[0003] With the rapid development of display technology, projection devices are increasingly widely used. A projection device is a device that can project images or videos onto a projection surface (such as a screen or a wall), and is currently widely used in homes, offices, schools, and entertainment venues.

[0004] When the position of a projection device changes, the user often needs to manually adjust the focal length, angle, and position of the projection device, so as to map the projection image to the original projection position. This manual adjustment method not only makes it difficult to accurately control the position and angle of the projection, but also requires time and cost, greatly affecting the user experience. SUMMARY

[0005] The present application discloses a projection control method and a projection device, which can solve the technical problem of difficulty in accurately restoring the projection image of a projection device.

[0006] In one aspect, the present application provides a projection control method, which includes: when a restoration instruction for a memory position of a projection device is received, acquiring a first environment feature of a current position of the projection device, determining a target environment feature in a second environment feature of the memory position based on a match between the first environment feature and the second environment feature; wherein the first environment feature and the second environment feature are generated based on multi-sensor data collected from a corresponding projection environment, determining adjustment information of the projection device based on the first environment feature and the target environment feature, and driving the projection device to rotate in space according to the adjustment information, so that the projection image of the projection device is projected to a memory position corresponding to the target environment feature.

[0007] In some embodiments of the present application, the memory position contains corresponding playback information, and the playback information includes one or more of playback content, playback type, and playback settings.

[0008] In some embodiments of the present application, the method for generating the first environmental feature comprises: fusing multi-sensor data corresponding to the current position to obtain a target image, determining feature points in the target image and descriptors of the feature points according to detection of the target image, and determining the first environmental feature according to the multi-sensor data and the descriptors of the feature points.

[0009] In some embodiments of the present application, the multi-sensor data corresponding to the current position comprises image information, depth information and illumination information of the projection environment, and the fusing of the multi-sensor data corresponding to the current position to obtain a target image comprises: registering the image information and the depth information to determine depth information corresponding to each pixel point in the image information, generating an initial image according to texture information corresponding to each pixel point in the image information and the depth information corresponding to each pixel point, and adjusting visual parameters of the initial image according to the illumination information to obtain the target image.

[0010] In some embodiments of the present application, the determining of the feature points in the target image and the descriptors of the feature points according to the detection of the target image comprises: performing edge detection on the target image to obtain an edge image, determining the feature points from pixel points in the edge image according to pixel values of the pixel points, dividing a neighborhood corresponding to the feature points in the edge image into a plurality of grids, generating a gradient histogram corresponding to each grid according to gradient information of each grid, determining a gradient amplitude vector corresponding to each grid according to a target gradient amplitude of each gradient direction in the gradient histogram, and aggregating the gradient amplitude vectors corresponding to the plurality of grids to obtain the descriptor of the feature points.

[0011] In some embodiments of the present application, the determining of the adjustment information of the projection device based on the first environmental feature and the target environmental feature comprises: determining a plurality of groups of initial feature point pairs by matching descriptors in the first environmental feature and the target environmental feature, mapping depth information of each group of initial feature point pairs to a first preset coordinate system to obtain updated depth information of the each group of initial feature point pairs, selecting a target feature point pair from the plurality of groups of initial feature point pairs according to the updated depth information, the target feature point pair comprising a first target feature point and a second target feature point, performing alignment processing on the first target feature point and the second target feature point to obtain a perspective transformation matrix, and determining the adjustment information based on the perspective transformation matrix.

[0012] In some embodiments of the present application, the adjustment information comprises attitude information, and the driving of the projection device to rotate in space according to the adjustment information comprises: controlling the projection device to rotate in space by using a gimbal of the projection device according to the attitude information.

[0013] In some embodiments of the present application, the adjustment information comprises correction information, and the method further comprises: correcting the projection picture according to the correction information, wherein the correction of the projection picture comprises picture scaling and trapezoidal correction.

[0014] In some embodiments of the present application, the method further comprises: adjusting the projection parameter of the projection device if it is detected that the current projection environment changes.

[0015] In another aspect, the present disclosure also provides a projection device, comprising: a processor; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the projection control method of the first aspect.

[0016] In the projection control method provided by the embodiments of the present application, since the multi-sensor data can comprehensively reflect the characteristics of the projection environment, the target environment characteristics of the memory position matched with the first environment characteristics can be accurately determined by matching the first environment characteristics with the environment characteristics of the memory position. The adjustment information of the projection device can be accurately determined through the first environment characteristics and the target environment characteristics, the projection picture can be accurately projected to the matched memory position by driving the projection device to rotate in the space through the accurate adjustment information, so that the projection picture of the projection device can be accurately restored after the position of the projection device changes. Since the restoration of the projection picture does not require complex manual adjustment by the user, the restoration of the projection picture is intelligent and simple, thereby improving the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can also be obtained by those skilled in the art without creative labor.

[0018] FIG. 1 is an application scenario diagram of the projection control method provided by the embodiments of the present application.

[0019] FIG. 2 is a flowchart of a target image generation method provided by an embodiment of the present application.

[0020] FIG. 3 is a flowchart of a descriptor generation method provided by another embodiment of the present application.

[0021] FIG. 4 is a flowchart of an adjustment information determination method provided by an embodiment of the present application.

[0022] FIG. 5 is a structural schematic diagram of a projection device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0023] Embodiments of the present disclosure will be described in more detail with reference to the drawings. Although some embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be implemented in various forms and should not be interpreted as being limited to the embodiments set forth herein, but rather these embodiments are provided so as to more thoroughly and completely understand the present disclosure. It is understood that the drawings and embodiments of the present disclosure are for exemplary purposes only and are not intended to limit the scope of protection of the present disclosure.

[0024] It should be understood that each step described in the method embodiments of the present disclosure can be performed in different order and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0025] The term "comprising" and variations thereof as used herein are open-ended, that is "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Related terms are defined in the following description.

[0026] It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0027] It should be noted that the modification of "one" or "multiple" mentioned in the present disclosure is illustrative rather than limiting, and those skilled in the art should understand that, unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0028] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present disclosure are only for illustrative purposes, and are not used to limit the scope of the messages or information.

[0029] In the related art, when the position of the projection device changes (for example, moves to a new position), if it is desired to restore the projection picture at the original projection position, the following methods are often used:

[0030] The first method relies on the user to re-adjust a series of complex adjustments of the projection device to achieve the restoration of the projection picture, such as adjusting the position and angle of the projection device, etc., to restore the original projection picture. This adjustment process is not only tedious and time-consuming, but also prone to errors, affecting the user experience.

[0031] The second way relies on the projection device being installed in a fixed position or a sensor (gyroscope and accelerometer, etc.) in the projection device to realize the recovery of the projection picture. However, this way cannot cope with complex environmental changes and is difficult to accurately recover the projection picture to the original projection position.

[0032] To solve the above problems, the embodiment of the present application provides a projection control method, which can accurately recover the projection picture of the projection device after the position of the projection device changes.

[0033] In the embodiment, the projection control method can be applied to one or more projection devices. For the projection device that needs to be controlled, the function for projection control provided by the method of the present application can be integrated directly on the projection device, or run on the projection device in the form of a software development kit (Software Development Kit, SDK).

[0034] As shown in FIG. 1, it is a flow chart of the projection control method provided by the embodiment of the present application. The order of the steps in the flow chart can be changed according to different needs, and some steps can be omitted. The projection control method is applied to a projection device. For example, the projection device in FIG. 5.

[0035] S11, when receiving the recovery instruction of the memory position of the projection device, acquiring the first environmental feature of the current position of the projection device.

[0036] In some embodiments of the present application, the projection device can be integrated with a gimbal. For example, the projection device can be located above the gimbal. The gimbal can have multiple degrees of freedom, so that the gimbal can be adjusted in multiple directions.

[0037] In some embodiments of the present application, since the projection position of the projection picture is determined by the pose information of the projection device, the memory position can be represented by the pose information of the projection device.

[0038] In some embodiments of the present application, the memory position can be obtained by multiple adjustments. For example, the initial pose information of the projection device is recorded as A, and the projection device can receive the adjustment operation of the user on the initial pose information A, so that the projection picture is projected to the projection surface (such as wall or curtain, etc.) expected by the user. The pose information obtained by adjusting the initial pose information A is recorded as B. In order to correct the projection picture, the projection device can adjust the pose information B, and the pose information obtained by adjusting the pose information B is recorded as C. In response to the save operation of the user, the projection device can take the pose information C as the memory position.

[0039] In other embodiments of the present application, the memory position can also correspond to the playing information, for example, setting the correspondence between the memory position and the playing information, the playing information including one or more of the playing content, the playing type and the playing setting. The playing type can be an image, a video, an audio, a playing channel, etc., the playing content can be the content corresponding to the playing type, and the playing setting can be a playing time, a playing order, and the like.

[0040] In some embodiments of the present application, the memory position can be multiple, and is set according to different use scenarios and needs of the user. For example, in some home use scenarios of the projection device, the living room, the bedroom and the study can each have a memory position.

[0041] When the memory position is multiple, each memory position has a corresponding second environment feature. The second environment feature corresponding to each memory position can be generated by the multi-sensor data corresponding to each memory position, and the multi-sensor data corresponding to each memory position can be obtained by the projection device collecting the projection environment through multiple sensors under the memory position. The description of the multi-sensor data corresponding to each memory position can refer to the multi-sensor data corresponding to the current position below, and the method of generating the second environment feature corresponding to each memory position according to the multi-sensor data of each memory position can refer to the description of the generation method of the first environment feature below, which will not be repeated here.

[0042] In some embodiments of the present application, the recovery instruction can be triggered in various ways. For example, the user can send the recovery instruction to the projection device through a remote controller or the like, or directly input the corresponding recovery instruction through any physical button or virtual control provided in the projection device.

[0043] In some embodiments of the present application, the first environment feature can be generated according to the multi-sensor data collected by the projection device from the projection environment of the current position. The multi-sensor data can include the data collected by the sensor of the projection device from the projection environment and / or the data processed from the data collected by the sensor.

[0044] The sensor includes but is not limited to a visual sensor, a depth sensor, a light sensor, an inertial measurement unit (IMU), etc. For example, the visual sensor can be a camera, a camera, etc., and the depth sensor can be a time of flight (ToF) sensor and a structured light sensor, etc.

[0045] The multi-sensor data can include, but is not limited to, image information of a projection surface (e.g., a screen or a wall) in a projection environment, depth information of the projection environment, illumination information, pose information of a projection device, and position information of a calibration point on a projection picture. The image information of the projection surface can be obtained by capturing the projection surface by a vision sensor. The texture information of the projection surface can include, but is not limited to, color, pattern shape, and roughness. The depth information of the projection environment can constitute a three-dimensional model of the projection environment. The illumination information can be obtained by a light sensor, and the illumination information can include brightness and light intensity, etc. The pose information of the projection device can be obtained by an inertial measurement unit, and the pose information of the projection device includes position information, attitude information, acceleration information, angular velocity information, etc. The position information can be the height of the projection device and the distance between the projection device and the projection surface, etc. The attitude information can be the yaw angle, the pitch angle, and the roll angle of the projection device, etc. The position information of the calibration point on the projection picture can be obtained by identifying the image information of the projection surface captured by the vision sensor by an image recognition algorithm. The image recognition algorithm can be a convolutional neural network and a target detection algorithm, etc. The image recognition method is not limited in the present application. The calibration point on the projection picture can be a plurality of corner points of the projection picture, and the corner points of the projection picture can be vertexes. The position information of the calibration point can be the coordinates of the corner points. The position information of the calibration point can be in the optical-mechanical coordinate system of the projection device. The optical-mechanical coordinate system can be a coordinate system corresponding to the optical system of the projection device. The optical system can include one or more optical components, such as a lens, a mirror, etc. For example, the optical-mechanical coordinate system can take the center point of the lens as the origin, take the width direction of the projection picture as the X-axis, take the height direction of the projection picture as the Y-axis, and take the direction perpendicular to the projection picture as the Z-axis.

[0046] In some embodiments of the present application, the multi-sensor data can be pre-processed before the first environment feature is generated. For example, the image information collected by the vision sensor can be filtered and corrected for distortion to reduce noise and distortion in the processed image information. The depth information collected by the depth sensor can be filtered and corrected to eliminate outliers and ensure the accuracy of the depth information. The attitude information collected by the inertial measurement unit can be filtered and standardized to enable comparison of attitude data collected at different times in the same coordinate system. The illumination information collected by the light sensor can be normalized to reflect the relative changes in illumination.

[0047] In some embodiments of the present application, the method for generating the first environment feature includes: the projection device fusing the multi-sensor data corresponding to the current position to obtain a target image, determining the feature points in the target image and the descriptors of the feature points according to the detection of the target image, and determining the first environment feature according to the multi-sensor data and the descriptors of the feature points.

[0048] The descriptor is used to describe the feature points and can be in the form of a vector, including the geometric, texture or gradient information of the pixel points around the feature points. The projection device can obtain the target image by fusing all or part of the multi-sensor data. For example, the projection device can obtain the target image by fusing the texture information, depth information and illumination information of the image information in the multi-sensor data.

[0049] For example, the projection device can determine the descriptors of the feature points and all the multi-sensor data corresponding to the current position as the first environment feature. Alternatively, the projection device can determine the descriptors of the feature points and part of the multi-sensor data corresponding to the current position as the first environment feature.

[0050] In some embodiments of the present application, the projection device can detect the feature points in the target image by various methods.

[0051] Exemplarily, the projection device can perform feature detection on the target image by using a feature detection algorithm, so as to determine the feature points. The feature points can be corner points, which can be pixel points with sharp changes in brightness and significant features in the target image, such as pixel points at the intersection of edges. The feature detection algorithm can be a corner point detection algorithm, which includes but is not limited to Harris corner point detection and FAST (Features from Accelerated Segment Test) algorithm.

[0052] In some embodiments of the present application, after detecting the feature points, the projection device can generate the descriptors of the feature points by using algorithms such as Scale-Invariant Feature Transform (SIFT) and Speeded-Up Robust Features (SURF).

[0053] In other embodiments of the present application, the projection device can detect the feature points in the target image and generate the descriptors of the feature points by using algorithms such as SIFT and SURF.

[0054] Exemplarily, the method for generating the descriptors of the feature points includes: for each detected feature point, the projection device can determine a corresponding feature region of the feature point in the target image or in a feature image obtained by performing feature detection on the target image, input the feature region into a preset neural network model, obtain feature maps output by multiple network layers in the neural network model, perform dimension reduction operation on each feature map to obtain a feature vector, and aggregate the feature vectors corresponding to the multiple feature maps to obtain a descriptor corresponding to the feature point.

[0055] The corresponding feature region of the feature point in the target image or in the feature image obtained by performing feature detection on the target image can be a region of a preset size centered on the feature point in the target image or in the feature image, and the preset size can be customized, which is not limited in the present application.

[0056] The neural network model includes but is not limited to ResNet, VGG or MobileNet. The neural network model can include multiple convolutional layers and pooling layers, which can extract features at different levels to obtain multiple feature maps, and the multiple feature maps can include shape, texture, color and depth information. By performing pooling processing on each feature map, the dimension of each feature map can be reduced. The pooling processing can be maximum pooling or average pooling, and the type of pooling is not limited in the present application.

[0057] The projection device can aggregate the feature vectors corresponding to the plurality of feature maps by using technologies such as average pooling, maximum pooling, or an attention mechanism, to obtain the descriptor corresponding to each feature point.

[0058] In view of the fact that single sensor data cannot cope with complex environmental changes and it is difficult to accurately restore the projection picture to the memory position, in the embodiment, since the multi-sensor data can comprehensively reflect the characteristics of the projection environment, the first environment feature and the second environment feature are generated by the multi-sensor data, so that the first environment feature and the second environment feature have accuracy, distinguishability, and stability, and can adapt to complex projection environments and cope with changes in the projection environment. Since the neural network model can generate multi-level and multi-scale descriptors, the descriptors have robustness.

[0059] In some embodiments, the projection device can optimize the generated descriptors to obtain optimized descriptors.

[0060] For example, the projection device can calculate the loss of the neural network model according to the descriptors of the generated feature points, adjust the network parameters of the neural network model according to the loss, until the loss converges, and determine the descriptors output by the neural network model when the loss converges as the optimized descriptors.

[0061] The loss can be a triplet loss and a contrastive loss. The network parameters can be a learning rate, a weight, a bias, and the like. The condition for determining the convergence of the neural network model can be customized, for example, the condition for convergence can be that the difference between consecutive loss values is within a preset range.

[0062] In the embodiment, by adjusting the network parameters of the neural network model by the loss, the similar descriptors in the descriptors generated by the neural network model are closer to each other, and the dissimilar descriptors are farther from each other, so that the similar descriptors and the dissimilar descriptors can be distinguished.

[0063] S12, determining a target environment feature in the second environment feature that matches the first environment feature based on matching between the first environment feature and the second environment feature of the memory position.

[0064] In some embodiments of the present application, the first environment feature and the second environment feature corresponding to each memory position each include a descriptor of a feature point.

[0065] In some embodiments of the present application, the projection device can match the first environment feature with the second environment feature corresponding to each memory position by using various methods, so as to determine the target environment feature matching the first environment feature in the second environment feature. The present application does not limit the matching method.

[0066] For example, the projection device can calculate the similarity between the first environment feature and the second environment feature corresponding to each memory position, and determine the second environment feature corresponding to the maximum similarity as the target environment feature. In this case, the projection device can determine the matching descriptors between the descriptor of the first environment feature and the descriptor of each second environment feature. The more the number of matching descriptors in the second environment feature, the more similar the second environment feature is to the first environment feature.

[0067] The above examples of matching the first environment feature with the second environment feature corresponding to each memory position are only examples, and the actual application is not limited thereto.

[0068] In this embodiment, since the first environment feature and the second environment feature have accuracy, distinguishability and stability, by calculating the similarity between the first environment feature and the second environment feature corresponding to each memory position, and determining the second environment feature corresponding to the maximum similarity as the target environment feature, the target environment feature can be accurately determined.

[0069] S13, determining the adjustment information of the projection device based on the first environment feature and the target environment feature.

[0070] In some embodiments of the present application, the projection device determines the adjustment information of the projection device based on the first environment feature and the target environment feature, which includes: determining a plurality of groups of initial feature point pairs by matching the descriptors in the first environment feature and the target environment feature; mapping the depth information of each group of initial feature point pairs to a first preset coordinate system to obtain updated depth information of each group of initial feature point pairs, selecting a target feature point pair from the plurality of groups of initial feature point pairs according to the updated depth information, the target feature point pair including a first target feature point and a second target feature point, aligning the first target feature point and the second target feature point to obtain a perspective transformation matrix, and determining the adjustment information based on the perspective transformation matrix.

[0071] In this case, the projection device can match the descriptors in the first environment feature and the target environment feature by using various methods, so as to determine a plurality of groups of initial feature point pairs. The present application does not limit the matching method.

[0072] Exemplarily, the projection device can calculate the distance (e.g., Euclidean distance and Hamming distance) between the descriptors of the feature points in the first environment feature and the descriptors of the feature points in the target environment feature, and determine the feature points corresponding to the descriptor pairs with the distance in the preset interval as the initial feature point pairs. The preset interval can be set as needed, and in order to ensure the matching degree of the initial feature point pairs, the preset interval can be set to be small, such as [0, 1].

[0073] In this embodiment, by determining the feature points corresponding to the descriptor pairs with the distance in the preset interval as the initial feature point pairs, the initial feature point pairs can be quickly determined.

[0074] Exemplarily, the projection device can determine the matched feature point pairs from the feature points of the first environment feature and the feature points of the second environment feature based on the descriptors of the feature points in the first environment feature and the descriptors of the feature points in the target environment feature by using a nearest neighbor search algorithm (NNS), and take the matched feature point pairs as the initial feature point pairs. The nearest neighbor search algorithm can be Fast Library for Approximate Nearest Neighbors (FLANN) or KD-Tree, etc.

[0075] In this embodiment, since the nearest neighbor search algorithm has high efficiency, the initial feature point pairs are determined by using the nearest neighbor search algorithm, which can improve the determination efficiency of the initial feature point pairs.

[0076] Exemplarily, the projection device can determine the matched feature point pairs from the feature points of the first environment feature and the feature points of the second environment feature by using a bidirectional matching method or a random sample consensus algorithm (RANSAC), and take the matched feature point pairs as the initial feature point pairs. The two feature points meeting or satisfying the bidirectional matching rule can be successfully matched in the forward direction and the reverse direction. By using RANSAC for optimization, the feature point pairs with matching errors in the initial feature point pairs can be reduced, and the accuracy of the initial feature point pairs can be improved.

[0077] In this embodiment, since the two feature points meeting or satisfying the bidirectional matching rule can be successfully matched in the forward direction and the reverse direction, the initial feature point pairs are determined by using the bidirectional matching method, which can improve the reliability of the initial feature point pairs.

[0078] In other embodiments of the present application, the projection device can match the descriptors in the first environment feature and the target environment feature by using a deep neural network, so as to determine the matched feature points in the feature points of the first environment feature and the feature points of the second environment feature, and pair the matched feature points as the initial feature point pairs. The deep neural network can include but is not limited to convolutional neural networks such as ResNet, VGG and Inception, Siamese Network, Attention Mechanism, and neural networks trained based on Contrastive Learning.

[0079] In other embodiments of the present application, the projection device can match the image information in the first environment feature and the image information in the second environment feature, and match the descriptors in the first environment feature and the descriptors in the second environment feature, so as to determine the feature points matched with each other in the feature points of the first environment feature and the feature points of the second environment feature as the initial feature point pairs.

[0080] For example, the projection device can match the first environment feature and the target environment feature by using a global feature matching algorithm, so as to determine the matched feature point pairs in the feature points of the first environment feature and the feature points of the second environment feature, and pair the matched feature point pairs as the initial feature point pairs. The global feature matching algorithm includes but is not limited to Image Pyramid Matching, Optical Flow Matching, and Global Descriptor Matching.

[0081] In this embodiment, by matching the image information and the descriptors in the first environment feature and the second environment feature, the stability and accuracy of the matching can be ensured.

[0082] In some embodiments of the present application, the first preset coordinate system can be a coordinate system corresponding to a visual sensor or a coordinate system corresponding to a depth sensor. The initial feature point pairs with a gap between the updated depth information in the multiple initial feature point pairs within a preset range can be determined as the target feature point pairs, wherein the preset range can be customized, and the present application does not limit this. The dimension of the perspective transformation matrix can be 4*4, including a 3*3 first rotation matrix and a 3*1 translation vector.

[0083] Since the first target feature point and the second target feature point have corresponding depth information, the first target feature point and the second target feature point can be regarded as point clouds in space.

[0084] Exemplarily, the projection device can utilize an Iterative Closest Point (ICP) to align the first target feature points with the second target feature points, and obtain the perspective transformation matrix. In the alignment process, the relationship between the depth information of the first target feature points and the second target feature points can be converted into a least square problem, and the target of the least square problem can be to minimize the distance or error between the first target feature points and the second target feature points. By utilizing Singular Value Decomposition (SVD) to solve the least square problem, the perspective transformation matrix for aligning the first target feature points to the second target feature points can be obtained.

[0085] It should be noted that the above examples of the method for aligning the first target feature points with the second target feature points are only examples, and the actual application is not limited thereto.

[0086] In the embodiment, by utilizing the Iterative Closest Point algorithm and the Singular Value Decomposition algorithm to calculate the perspective transformation matrix, the accuracy of the perspective transformation matrix can be ensured.

[0087] In some embodiments of the present application, the projection device can determine adjustment information according to the perspective transformation matrix, the first environment feature and the second environment feature. The determination method of the adjustment information will be described in detail below.

[0088] The adjustment information can include pose information corresponding to the projection device and / or correction information corresponding to the projection picture. The pose information can be a target Euler angle, and the target Euler angle includes one or more of a yaw angle, a pitch angle and a roll angle. The pose of the projection device can be adjusted through the pose information. Exemplarily, the pitch angle in the pose information can be used to control the angle of up and down inclination of the projection device, and / or the yaw angle in the pose information can be used to control the angle of left and right rotation of the projection device, and / or the roll angle in the pose information can be used to control the angle of left and right flipping of the projection device.

[0089] The correction information can include target coordinates of a plurality of corner points in the projection picture, and the correction information can be used to correct the projection picture of the projection device.

[0090] In the embodiment, since the perspective transformation matrix has accuracy, the first environment feature and the second environment feature have accuracy, distinguishability and stability, therefore, according to the perspective transformation matrix, the first environment feature and the second environment feature, the accuracy of the pose information of the projection device and the correction information of the projection picture can be ensured.

[0091] S14, driving the projection device to rotate in the space according to the adjustment information, so that the projection picture of the projection device is projected to the memory position corresponding to the target environment feature.

[0092] In some embodiments of the present application, taking the control of the gimbal according to the attitude information as an example, the projection device can convert the attitude information into control instructions of the gimbal, control the motor of the gimbal according to the control instructions, so as to drive the projection device to rotate in space, so that the projection picture is projected to the memory position corresponding to the target environmental feature.

[0093] The control instructions can include parameters such as the rotation direction, angle and speed of the gimbal.

[0094] In the present embodiment, the gimbal is controlled to rotate by accurate attitude information, so as to drive the projection device to rotate in space, and the projection picture can be accurately restored to the memory position corresponding to the target environmental feature.

[0095] In some embodiments of the present application, the projection device can correct the projection picture according to the correction information, wherein the correction of the projection picture includes picture scaling and trapezoidal correction.

[0096] The coordinates of the corner points in the projection picture can be converted into the corresponding target gradient coordinates in the correction information, so as to realize the trapezoidal correction and picture scaling of the projection picture.

[0097] In the present embodiment, by performing picture scaling and trapezoidal correction on the projection picture, the trapezoidal distortion can be eliminated, and better visual effect can be provided.

[0098] In other embodiments of the present application, if it is detected that the current projection environment changes, such as illumination change, wall texture change, etc., the projection parameters of the projection device can be adjusted. The projection parameters can include but are not limited to attitude information, correction information, etc.

[0099] In the present embodiment, by adjusting the projection parameters in real time according to the change of the projection environment, the stability and clarity of the projection picture can be ensured.

[0100] In other embodiments of the present application, if the memory position includes corresponding playing content, the projection device can determine the unplayed content in the playing content, and play or display the unplayed content in the projection picture.

[0101] In the projection control method provided in the embodiments of the present application, since the multi-sensor data can comprehensively reflect the characteristics of the projection environment, the target environment characteristics of the memory position matched with the first environment characteristics can be accurately determined by matching the first environment characteristics with the environment characteristics of the memory position. The adjustment information for the projection device can be accurately determined according to the first environment characteristics and the target environment characteristics, the projection device can be driven to rotate in the space according to the accurate adjustment information, and the projection picture can be accurately projected to the matched memory position, so that the projection picture of the projection device can be accurately restored after the position of the projection device changes. Since the restoration of the projection picture does not need complex manual adjustment by the user, the restoration of the projection picture is intelligent and simple, thereby improving the user experience.

[0102] As shown in FIG. 2, it is a flowchart of the target image generation method provided in an embodiment of the present application.

[0103] S111, register the image information and the depth information to determine the depth information corresponding to each pixel point in the image information.

[0104] In some embodiments of the present application, the projection device can register the image information and the depth information by using a nearest neighbor matching method such as Fast Library for Approximate Nearest Neighbors (FLANN) or KD-Tree, so as to quickly determine the depth information corresponding to each pixel point.

[0105] S112, generate an initial image according to the texture information corresponding to each pixel point in the image information and the depth information corresponding to each pixel point.

[0106] In some embodiments of the present application, the projection device can associate the texture information of each pixel point with the corresponding depth information, so as to generate a three-dimensional initial image.

[0107] The texture information can be the pixel value of the pixel point. For example, if the image information is color, the pixel value can be the RGB value. By fusing the RGB information and the depth information, an initial image in RGB-D format can be obtained.

[0108] In the present embodiment, by fusing the texture information and the depth information, the information amount of the initial image can be improved, thereby providing a basis for generating comprehensive, stable and accurate first environment characteristics and second environment characteristics.

[0109] S113, adjust the visual parameters of the initial image according to the illumination information to obtain a target image.

[0110] In some embodiments of the present application, the visual parameter can be luminance, contrast, color saturation, and the like.

[0111] In the present embodiment, the visual parameter of the initial image is adjusted according to the illumination information, so that a clear and natural target image can be obtained.

[0112] As shown in FIG. 3, it is a flow chart of a descriptor generation method according to another embodiment of the present application, including the following steps:

[0113] S114, performing edge detection on the target image to obtain an edge image.

[0114] In some embodiments of the present application, the projection device can perform edge detection on the target image by using an edge detection algorithm. The edge detection algorithm includes but is not limited to Canny edge detection, Sobel operator, and the like.

[0115] In the present embodiment, the binary edge image can be obtained by performing edge detection on the target image. The pixel point with a preset value (for example, 255) in the edge image represents an edge.

[0116] S115, determining a feature point from the pixel point according to the pixel value of the pixel point in the edge image.

[0117] In some embodiments of the present application, the projection device can determine a feature point from the pixel point by using a feature detection algorithm based on the pixel value of the pixel point in the edge image.

[0118] The feature point can be a corner point, and the corner point can be a pixel point with a sharp change in luminance and a significant feature in the target image, such as a pixel point at the intersection of edges. The feature detection algorithm can be a corner point detection algorithm, which includes but is not limited to Harris corner point detection and FAST (Features from Accelerated Segment Test) algorithm.

[0119] The above examples of the feature point detection method are only examples, and are not limited in actual applications.

[0120] S116, dividing the neighborhood corresponding to the feature point in the edge image into a plurality of grids, and generating a gradient histogram corresponding to each grid according to the gradient information of each grid.

[0121] In some embodiments of the present application, each feature point in the edge image has a corresponding neighborhood, and the size of the neighborhood corresponding to each feature point can be customized. For example, the neighborhood corresponding to each feature point can be a 16*16 region centered on each feature point.

[0122] The number of grids is not limited in the present application. For example, the 16*16 neighborhood can be divided into 16 grids according to the above embodiment.

[0123] The gradient histogram corresponding to each feature point can reflect the gradient intensity distribution of each feature point in a plurality of gradient directions, which can be obtained by dividing a preset angle. For example, the preset angle can be 360°, and each gradient direction has a corresponding angle interval.

[0124] The number of gradient directions obtained by dividing the preset angle can be set by the user, and the present application does not limit this. The gradient information of each pixel point can include the gradient direction and the gradient size of each pixel point.

[0125] The angle interval corresponding to the gradient direction of each pixel point is determined, the gradients of all pixel points corresponding to each angle interval are accumulated, and the target gradient amplitude of the gradient direction corresponding to each angle interval is obtained. The target gradient amplitudes corresponding to the plurality of gradient directions can constitute the gradient histogram corresponding to each feature point.

[0126] S117, according to the target gradient amplitude of each gradient direction in the gradient histogram, determining the gradient amplitude vector corresponding to each grid.

[0127] In some embodiments of the present application, the target gradient amplitudes of all gradient directions in the gradient histogram corresponding to each grid can be used as elements in the gradient amplitude vector corresponding to each grid, so as to obtain the gradient amplitude vector corresponding to each grid.

[0128] S118, aggregating the gradient amplitude vectors corresponding to the plurality of grids to obtain the descriptor of the feature point.

[0129] In some embodiments of the present application, the projection device can use methods such as average pooling, maximum pooling or attention mechanism to aggregate and process the gradient amplitude vectors corresponding to the plurality of grids, so as to obtain the descriptor corresponding to each feature point.

[0130] In some embodiments of the present application, the position information of the anchor points of the projection picture in the first environment feature can include a plurality of first anchor coordinates, and the position information of the anchor points of the projection picture in the second environment feature can include a plurality of second anchor coordinates. For example, the plurality of first anchor coordinates and the plurality of second anchor coordinates can be the three-dimensional space coordinates of four anchor points on the projection picture. As shown in FIG. 4, it is a flow chart of the method for determining adjustment information provided by an embodiment of the present application, which includes the following steps:

[0131] S131, transform each first calibration coordinate by using a perspective transformation matrix to obtain a third calibration coordinate corresponding to the each first calibration coordinate, map the third calibration coordinate to the second preset coordinate system to obtain a fourth calibration coordinate corresponding to the each first calibration coordinate.

[0132] In some embodiments of the present application, the method for generating the third calibration coordinate corresponding to the each first calibration coordinate can refer to formula (1):

[0133] p' = R * p + t; (1)

[0134] wherein p' represents the third calibration coordinate, R represents a rotation matrix in the perspective transformation matrix, p represents the first calibration coordinate, and t represents a translation vector in the perspective transformation matrix.

[0135] The second preset coordinate system can be an optical-mechanical coordinate system.

[0136] In the present embodiment, by transforming each first calibration coordinate by using a perspective transformation matrix, each first calibration coordinate can be mapped from the second preset coordinate system to the first preset coordinate system to obtain a third calibration coordinate corresponding to the each first calibration coordinate. Each third calibration coordinate can be mapped from the first preset coordinate system to the second preset coordinate system by using an inverse matrix of the perspective transformation matrix to obtain a fourth calibration coordinate corresponding to the each first calibration coordinate.

[0137] S132, transform each fourth calibration coordinate according to any second calibration coordinate to obtain a first transformation matrix between the each fourth calibration coordinate and the any second calibration coordinate.

[0138] In some embodiments of the present application, the method for obtaining the first transformation matrix according to the any second calibration coordinate can refer to formula (1) in step S131, which will not be repeated herein. The dimension of the first transformation matrix is the same as that of the perspective transformation matrix.

[0139] S133, calculate a second transformation matrix according to the pose information in the first environmental feature and the second environmental feature.

[0140] In some embodiments of the present application, the projection device can determine the pitch axis gravity variation GA1, the yaw axis variation GA2 and the calibration parameters of the gravity sensor (G-sensor) according to the pose information of the first environmental feature and the second environmental feature, and calculate the pitch axis gravity variation GA1, the yaw axis variation GA2 and the calibration parameters of the gravity sensor (G-sensor) by using the inverse kinematics solution to obtain the second transformation matrix.

[0141] When the pan-tilt unit of the projection device has multiple degrees of freedom, there can be multiple second transformation matrices. The method for generating the second transformation matrices can refer to the inverse kinematics solution method, which will not be repeated here. The dimension of each second transformation matrix is ​​the same as the dimension of the perspective transformation matrix.

[0142] S134, determine the third transformation matrix based on each first transformation matrix and the second transformation matrix corresponding to each first step coordinate.

[0143] In some embodiments of this application, a third transformation matrix can be obtained by multiplying each first transformation matrix corresponding to each first step coordinate with a second transformation matrix. The dimension of the third transformation matrix is ​​the same as the dimension of the perspective transformation matrix.

[0144] S135, calculate the initial Euler angles corresponding to each first step coordinate based on the rotation matrix in each third transformation matrix, and determine the polygon corresponding to each first step coordinate in the two-dimensional parameter space based on the multiple initial Euler angles corresponding to each first step coordinate.

[0145] In some embodiments of this application, the projection device can extract initial Euler angles from the rotation matrix in the third transformation matrix. The initial Euler angles include one or more of yaw angle, pitch angle, and roll angle. When there are four first-tier calibration coordinates and four second-tier calibration coordinates, each first-tier calibration coordinate can correspond to four initial Euler angles, and four first-tier calibration coordinates correspond to 16 initial Euler angles. The polygon corresponding to the four Euler angles of each first-tier calibration coordinate in the two-dimensional plane can be a rectangle.

[0146] S136, determine the two-dimensional coordinate system corresponding to the two-dimensional parameter space, and determine the overlapping area between the polygons corresponding to the plurality of first-tier coordinates.

[0147] In some embodiments of this application, the horizontal and vertical axes of the two-dimensional coordinate system can each represent a component of the initial Euler angle. For example, if the initial Euler angle includes yaw and pitch angles, the horizontal axis of the two-dimensional coordinate system can represent the yaw angle, and the vertical axis can represent the pitch angle.

[0148] S137. Based on the coordinates of the target point in the overlapping region in the two-dimensional coordinate system, determine the target Euler angles, and solve the target Euler angles in reverse to obtain the fourth transformation matrix.

[0149] In some embodiments of the present application, a point in the overlapping region can be randomly selected as the target point, and the coordinates of the target point can be determined as the target Euler angle. For example, if the horizontal axis of the two-dimensional coordinate system represents the yaw angle and the vertical axis represents the pitch angle, the target point can be the centroid of the overlapping region of the plurality of polygons, and the coordinates of the centroid can correspond to a yaw angle and a pitch angle. Therefore, the yaw angle and the pitch angle corresponding to the centroid can be determined as the target Euler angle.

[0150] The above examples of the selection method of the target point are only examples, and the actual application is not limited thereto.

[0151] According to the target Euler angle, the projection device can determine the rotation matrix in the fourth transformation matrix, determine the preset vector as the translation vector in the fourth rotation matrix, and thus obtain the fourth rotation matrix. The preset vector can be customized, or the translation vector in the first transformation matrix corresponding to the initial Euler angle that is the same as or similar to the target Euler angle can be used as the translation vector in the fourth transformation matrix, or the projection device can directly use the rotation matrix determined by the target Euler angle as the fourth transformation matrix. The dimension of the fourth transformation matrix is the same as that of the perspective transformation matrix.

[0152] S138, according to the fourth transformation matrix, the plurality of second homographic coordinates are transformed to obtain target homographic coordinates.

[0153] In some embodiments of the present application, the method of transforming the plurality of second homographic coordinates according to the fourth transformation matrix to obtain the target homographic coordinates can refer to the generation method of the third homographic coordinates corresponding to each first homographic coordinate, which will not be repeated here.

[0154] S139, the target Euler angle and the target homographic coordinates are determined as the adjustment information.

[0155] In some embodiments of the present application, the target Euler angle can be used to adjust the pose of the projection device, and the target homographic coordinates can be used to correct the projection picture of the projection device.

[0156] As shown in FIG. 5, it is a structural schematic diagram of the projection device provided by the embodiments of the present application.

[0157] The projection device 200 includes a projection part 210 and a driving part 220 for driving the projection part 210. The projection part 210 can form an optical image and project the optical image onto the imaging medium SC.

[0158] The projection part 210 includes a light source part 211, a light modulator 212, and an optical system 213. The driving part 220 includes a light source driving part 221 and a light modulator driving part 222.

[0159] The light source section 211 can include a solid light source such as a light emitting diode (LED), a laser, a pump lamp, and the like. The light source section 211 can include an optical element such as a lens, a polarizing plate, and the like for improving the optical characteristics of the projected light, and a dimming element and the like for adjusting the light flux.

[0160] The light source driving section 221 can control the operation of the light source in the light source section 211, including turning on and off, according to the instructions of the control section 250.

[0161] The light modulator 212 includes a display panel 215, which can be a transmissive liquid crystal display (LCD), a reflective liquid crystal on silicon (LCOS), or a digital micromirror device (DMD).

[0162] The light modulator 212 is driven by a light modulator driving section 222, which is connected to the image processing section 245.

[0163] The image processing section 245 inputs image data to the light modulator driving section 222. The light modulator driving section 222 converts the input image data into a data signal suitable for the operation of the display panel 215. The light modulator driving section 222 applies a voltage to each pixel of each display panel 215 according to the converted data signal, and draws an image on the display panel 215.

[0164] The optical system 213 includes a lens or a mirror and the like that images the incident image light PLA on the imaging medium SC. The optical system 213 can also include a zoom mechanism that magnifies or reduces the image projected onto the imaging medium SC, a focus adjustment mechanism that adjusts the focus, and the like.

[0165] The projection device 200 also includes an operation section 231, a signal receiving section 233, an input interface 235, a storage section 237, a data interface 241, an interface section 242, a frame memory 243, an image processing section 245, and a control section 250. The input interface 235, the storage section 237, the data interface 241, the interface section 242, the image processing section 245, and the control section 250 can communicate data with each other via an internal bus 207.

[0166] The operation section 231 can generate corresponding operation signals according to the operation of various buttons and switches acting on the surface of the housing of the projection device 200, and output to the input interface 235. The input interface 235 includes a circuit that outputs the operation signals input from the operation section 231 to the control section 250.

[0167] The signal receiving section 233 receives a signal (e.g., an infrared signal, a Bluetooth signal) transmitted from a control device 5 (e.g., a remote controller), and decodes the received signal to generate a corresponding operation signal. The signal receiving section 233 outputs the generated operation signal to the input interface 235. The input interface 235 outputs the received operation signal to the control section 250.

[0168] The storage section 237 can be a magnetic recording device such as a hard disk drive (HDD), or a storage device using a semiconductor storage element such as a flash memory. The storage section 237 stores a program executed by the control section 250, data processed by the control section 250, image data, and the like.

[0169] The data interface 241 includes a connector and an interface circuit, and can be connected to another electronic device 100 in a wired manner. The data interface 241 can be a communication interface that performs communication with another electronic device 100. The data interface 241 receives image data, sound data, and the like from another electronic device 100. In the present embodiment, the image data can be content image.

[0170] The interface section 242 is a communication interface that communicates with another electronic device 100 in accordance with an Ethernet standard. The interface section 242 includes a connector and an interface circuit that processes a signal transmitted by the connector. The interface section 242 is an interface substrate including the connector and the interface circuit, and is connected to a main substrate of the control section 250, which is a substrate on which the processor 253 and other components are mounted. The connector and the interface circuit that constitute the interface section 242 are mounted on the main substrate of the control section 250. The interface section 242 can receive setting information or instruction information transmitted from another electronic device 100.

[0171] The control section 250 includes a memory 251 and a processor 253.

[0172] The memory 251 is a storage device that non-volatile stores a program and data executed by the processor 253. The memory 251 is constituted by a magnetic storage device, a flash read-only memory (ROM), a semiconductor storage element, or another kind of non-volatile storage device. The memory 251 can also include a random access memory (RAM) that constitutes a work area of the processor 253. The memory 251 stores data processed by the control section 250, and a control program executed by the processor 253.

[0173] The processor 253 can be constituted by a single processor or a combination of a plurality of processing units. The processor 253 executes a control program to control the respective parts of the projection device 200. For example, the processor 253 performs a corresponding interactive projection control according to an operation signal generated by the operation section 231, and outputs a parameter used in the interactive projection control, such as a parameter for trapezoidal correction of an image, to the image processing section 245. In addition, the processor 253 can control the light source section 211 to turn on, turn off, or adjust the brightness of the light source by controlling the light source driving section 221.

[0174] The image processing section 245 and the frame memory 243 can be constituted by an integrated circuit. The integrated circuit includes a Large Scale Integration (LSI), an Application Specific Integrated Circuit (ASIC), a Programmable Logic Device (PLD), which can include a Field-Programmable Gate Array (FPGA). The integrated circuit can also include a part of an analog circuit, or a combination of a processor and an integrated circuit. The combination of the processor and the integrated circuit is referred to as a Microcontroller Unit (MCU), a System on Chip (SoC), a system LSI, a chipset, or the like.

[0175] The image processing section 245 can store the image data received from the data interface 241 in the frame memory 243. The frame memory 243 includes a plurality of memory banks each including a storage capacity in which one frame of image data can be written. The frame memory 243 can be constituted by a Synchronous Dynamic Random Access Memory (SDRAM) or a Dynamic Random Access Memory (DRAM).

[0176] The image processing section 245 can perform interactive projection control on the image data stored in the frame memory 243, including resolution conversion, size adjustment, distortion correction, shape correction, digital zooming, image tone adjustment, and image brightness adjustment.

[0177] The image processing section 245 can also convert the input frame frequency of the vertical synchronization signal to a drawing frequency, and generate a vertical synchronization signal having the drawing frequency, which is referred to as an output synchronization signal. The image processing section 245 then outputs the output synchronization signal to the light modulator driving section 222.

[0178] In some embodiments of the present application, a computer readable storage medium is provided, which stores a computer program. The computer program is loaded by a processor, so that the processor executes the steps of the projection control method described above. The steps of the projection control method can be the steps of the projection control method in each of the embodiments described above.

[0179] The above description is merely exemplary of the present disclosure and the applied technical principles. It should be understood by those skilled in the art that the disclosed scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the technical features described above, and should also cover other technical solutions formed by any combinations of the technical features described above or their equivalent features without departing from the disclosed concept. For example, the technical solutions formed by replacing the above-described features with the technical features disclosed in the present disclosure (but not limited to) having similar functions.

[0180] In addition, although each operation is described in a specific order, this should not be understood as requiring the operations to be performed in the specific order shown or in a sequential order. In certain circumstances, multitasking and parallel processing can be advantageous. Similarly, although several implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments can also be combined in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented separately or in any suitable subcombination.

[0181] Although the subject matter has been described in language specific to structural features, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely exemplary forms of implementing the claims. As for the devices in the above-described embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

Claims

1. A projection control method, wherein, The method comprises: When receiving a recovery instruction of a memory position of a projection device, acquiring a first environment feature of a current position of the projection device; Based on the matching between the first environment feature and a second environment feature of the memory position, determining a target environment feature in the second environment feature that matches the first environment feature; wherein the first environment feature and the second environment feature are generated based on multi-sensor data collected from a corresponding projection environment; Based on the first environment feature and the target environment feature, determining adjustment information of the projection device; According to the adjustment information, driving the projection device to rotate in space, so that the projection picture of the projection device is projected to a memory position corresponding to the target environment feature.

2. The projection control method according to claim 1, wherein The memory position contains corresponding playing information, and the playing information includes one or more of playing content, playing type and playing setting.

3. The projection control method according to claim 1, wherein The generation method of the first environment feature comprises: Fusing multi-sensor data corresponding to the current position to obtain a target image; According to the detection of the target image, determining feature points in the target image and descriptors of the feature points; According to the multi-sensor data and the descriptors of the feature points, determining the first environment feature.

4. The projection control method according to claim 3, wherein The multi-sensor data corresponding to the current position includes image information, depth information and illumination information of the projection environment, and the fusion of the multi-sensor data corresponding to the current position to obtain a target image comprises: Registering the image information and the depth information to determine the depth information corresponding to each pixel point in the image information; According to the texture information corresponding to each pixel point in the image information and the depth information corresponding to each pixel point, generating an initial image; According to the illumination information, adjusting the visual parameters of the initial image to obtain the target image.

5. The projection control method according to claim 3, wherein According to the detection of the target image, determining the feature points in the target image and the descriptors of the feature points comprises: Performing edge detection on the target image to obtain an edge image; According to the pixel values of the pixel points in the edge image, determining the feature points from the pixel points; Dividing the neighborhood corresponding to the feature points in the edge image into multiple grids, and generating a gradient histogram corresponding to each grid according to the gradient information of each grid; According to the target gradient amplitude of each gradient direction in the gradient histogram, determining a gradient amplitude vector corresponding to each grid; Aggregating the gradient amplitude vectors corresponding to the multiple grids to obtain the descriptor of the feature points.

6. The projection control method according to claim 3, wherein The determination of the adjustment information of the projection device based on the first environment feature and the target environment feature comprises: By matching the descriptors in the first environment feature and the target environment feature, a plurality of groups of initial feature point pairs are determined; Mapping the depth information of each group of initial feature point pairs to a first preset coordinate system to obtain updated depth information of each group of initial feature point pairs; According to the updated depth information, selecting a target feature point pair from the plurality of groups of initial feature point pairs, the target feature point pair comprising a first target feature point and a second target feature point; align the first target feature point and the second target feature point, to obtain a perspective transformation matrix; determine the adjustment information based on the perspective transformation matrix.

7. The projection control method according to claim 6, wherein The determining a plurality of initial feature point pairs by matching the descriptors in the first environment feature and the target environment feature comprises: calculating distances between descriptors of feature points in the first environment feature and descriptors of feature points in the target environment feature; determining feature points corresponding to descriptor pairs with distances in a preset interval as the plurality of initial feature point pairs.

8. The projection control method according to claim 6, wherein The determining a plurality of initial feature point pairs by matching the descriptors in the first environment feature and the target environment feature comprises: determining, as the plurality of initial feature point pairs, mutually matched feature points from feature points between the first environment feature and the target environment feature based on matching image information between the first environment feature and the target environment feature and matching descriptors between the first environment feature and the target environment feature.

9. The projection control method according to claim 6, wherein The determining a plurality of initial feature point pairs by matching the descriptors in the first environment feature and the target environment feature comprises: determining, as the plurality of initial feature point pairs, matched feature point pairs from feature points between the first environment feature and the target environment feature by using a bidirectional matching method or a random sample consensus algorithm.

10. The projection control method according to claim 6, wherein The determining a plurality of initial feature point pairs by matching the descriptors in the first environment feature and the target environment feature comprises: determining, as the plurality of initial feature point pairs, matched feature point pairs from feature points between the first environment feature and the target environment feature by matching the first environment feature and the target environment feature through a global feature matching algorithm, wherein the global feature matching algorithm comprises at least one of image pyramid matching, optical flow method, and global descriptor matching.

11. The projection control method according to claim 6, wherein The determining a plurality of initial feature point pairs by matching the descriptors in the first environment feature and the target environment feature comprises: determining, as the plurality of initial feature point pairs, matched feature point pairs from feature points between the first environment feature and the target environment feature based on matching, by a deep neural network, descriptors between the first environment feature and the target environment feature.

12. The projection control method according to claim 6, wherein The first environment feature comprises a plurality of first gradient coordinates, the target environment feature comprises a plurality of second gradient coordinates, and the determining the adjustment information based on the perspective transformation matrix comprises: transforming each first gradient coordinate by using the perspective transformation matrix to obtain a third gradient coordinate corresponding to the each first gradient coordinate, and mapping the third gradient coordinate to a second preset coordinate system to obtain a fourth gradient coordinate corresponding to the each first gradient coordinate; transforming each fourth gradient coordinate according to any one second gradient coordinate to obtain a first transformation matrix between the each fourth gradient coordinate and the any one second gradient coordinate; calculating a second transformation matrix according to pose information in the first environment feature and the target environment feature; and determining the adjustment information based on the first transformation matrix and the second transformation matrix. determining a third transformation matrix according to each first transformation matrix corresponding to each first keystone coordinate and a second transformation matrix; calculating initial Euler angles corresponding to each first keystone coordinate according to a rotation matrix in each third transformation matrix, and determining a polygon corresponding to each first keystone coordinate in a two-dimensional parameter space according to the initial Euler angles corresponding to each first keystone coordinate; determining the adjustment information based on the polygons corresponding to the first keystone coordinates.

13. The projection control method according to claim 12, wherein The determination of the adjustment information based on the polygons corresponding to the first keystone coordinates comprises: determining a two-dimensional coordinate system corresponding to the two-dimensional parameter space, and determining an overlapping area between the polygons corresponding to the first keystone coordinates; determining a target Euler angle according to coordinates of a target point in the overlapping area in the two-dimensional coordinate system, inversely solving the target Euler angle to obtain a fourth transformation matrix; transforming the second keystone coordinates according to the fourth transformation matrix to obtain target keystone coordinates; determining the target Euler angle and the target keystone coordinates as the adjustment information.

14. The projection control method according to claim 3, wherein The method for generating the descriptor of the feature point comprises: for each detected feature point, determining a feature region corresponding to the feature point in the target image or in a feature image obtained by performing feature detection on the target image; inputting the feature region into a preset neural network model to obtain feature maps output by multiple network layers in the neural network model; performing dimension reduction on each feature map to obtain a feature vector; aggregating the feature vectors corresponding to the multiple feature maps to obtain a descriptor corresponding to the feature point.

15. The projection control method according to claim 14, wherein The method further comprises: optimizing the descriptor by using the neural network model to obtain an optimized descriptor.

16. The projection control method of claim 15, wherein, The optimization of the descriptor by using the neural network model to obtain the optimized descriptor comprises: calculating a loss of the neural network model according to the descriptor, adjusting network parameters of the neural network model according to the loss until the neural network model converges, and determining a descriptor output by the converged neural network model as the optimized descriptor.

17. The projection control method of claim 1, wherein, The adjustment information comprises attitude information, and the driving of the projection device to rotate in space according to the adjustment information comprises: controlling the projection device to rotate in space by using a gimbal of the projection device according to the attitude information.

18. The projection control method of claim 1, wherein, The adjustment information comprises correction information, and the method further comprises: correcting the projection picture according to the correction information, wherein the correction of the projection picture comprises picture scaling and keystone correction.

19. The projection control method of claim 1, wherein, The method further comprises: if it is detected that the current projection environment changes, adjusting projection parameters of the projection device.

20. A projection apparatus, wherein, The projection device comprises: a processor; a memory; and an application program stored in the memory and configured to be executed by the processor to implement the projection control method in any one of claims 1 to 19.

Images