Obstacle recognition method, projection method and projection device

Abstract

The application discloses an obstacle recognition method, a projection method and a projection device. Firstly, environment information is collected at a target projection position by an image sensor; three-dimensional space data of a current projection environment is constructed based on the environment information, and depth obstacle feature points are extracted based on the three-dimensional space data; an infrared segmentation map of a projection plane is constructed based on the environment information, and infrared obstacle feature points are extracted based on the infrared segmentation map; the depth obstacle feature points and the infrared obstacle feature points are filtered, and the depth obstacle feature points and the infrared obstacle feature points meeting conditions are marked as target obstacle points. The application combines depth data and infrared data to achieve a non-sensing obstacle avoidance projection effect based on depth obstacle avoidance features and infrared obstacle avoidance features.

Description

Technical Field

[0001] This invention relates to the field of projection obstacle avoidance technology, and more specifically, to an obstacle recognition method, a projection method, and a projection device. Background Technology

[0002] Due to the nature of the usage scenarios, projectors are often moved during use. After the entire unit is moved, the image projected onto the screen or wall becomes distorted. Obstacles such as paintings, ornaments, and light switches often exist on the viewing surface, affecting the viewing experience.

[0003] Obstacles are usually identified by visual observation. This requires a thorough examination of the projection surface, such as the screen, before setting up the projector. While this method can address obvious obstacles, it is not always possible to accurately identify uneven projection surfaces with the naked eye under certain special projection conditions. As a result, obstacles can affect the projection effect.

[0004] Currently, the common practice is to use the projected UI image function to achieve obstacle avoidance calculation for the projected image. This requires the projector to project an image with distinctive features, which is then captured by the projector's camera. Image processing algorithms are then used to calculate the space of the projected lower point. This method interrupts the user's normal use after projecting the image with distinctive features, and the processing time is relatively long, usually taking several seconds, which is not a very user-friendly experience. Summary of the Invention

[0005] In view of this, the present invention provides an obstacle recognition method, a projection method, and a projection device to combine depth data and infrared data to more effectively identify obstacles based on depth obstacle avoidance features and infrared obstacle avoidance features.

[0006] To address the above technical problems, the present invention provides an obstacle recognition method, comprising:

[0007] Environmental information is acquired at the target projection location using an image sensor;

[0008] Based on environmental information, construct three-dimensional spatial data of the current projection environment, and extract feature points of deep obstacles based on the three-dimensional spatial data;

[0009] Infrared segmentation maps of the projection plane are constructed based on environmental information, and infrared obstacle feature points are extracted based on the infrared segmentation maps.

[0010] The feature points of depth obstacles and infrared obstacles are filtered, and those that meet the criteria are marked as target obstacle points.

[0011] As an optional method, acquiring environmental information at the target projection location using an image sensor includes:

[0012] Depth data of the current projection environment is collected using a depth sensor; infrared data of the current projection environment is collected using an infrared sensor.

[0013] Based on environmental information, a 3D spatial data of the current projection environment is constructed, and feature points of deep obstacles are extracted based on the 3D spatial data, including:

[0014] 3D point cloud data is obtained from depth data, the projection plane is fitted by the least squares method to obtain the plane equation, the distance from each point to the plane is calculated, and pixels whose distance from the plane is greater than a preset threshold are marked as depth obstacle feature points.

[0015] An infrared segmentation map of the projection plane is constructed based on environmental information, and infrared obstacle feature points are extracted based on the infrared segmentation map, including:

[0016] Infrared data is fed into a trained infrared wall segmentation network to obtain an infrared segmentation map, and obstacles in the projected background are marked as infrared obstacle feature points.

[0017] As an optional method, depth obstacle feature points and infrared obstacle feature points are filtered, and those that meet the criteria are marked as target obstacle points, including:

[0018] Feature points that belong to both deep obstacle feature points and infrared obstacle feature points are marked, and all marked feature points are regarded as target obstacle points.

[0019] Area region filtering is performed on unmarked depth obstacle feature points. Points with an area region smaller than or equal to a first preset area are removed based on the filtering results, and the remaining points are regarded as target obstacle points.

[0020] Unmarked infrared obstacle feature points are filtered by area region. Points with an area region smaller than or equal to a second preset area are removed based on the filtering results. Interest regions are established around the remaining infrared obstacle feature points. The average infrared intensity of the interest region is compared with the average pixel intensity of the projection plane. Based on the comparison results, infrared obstacle feature points that meet the preset conditions are marked as target obstacle points.

[0021] As an optional method, area region filtering is performed on unmarked infrared obstacle feature points, and points with an area region smaller than or equal to a second preset area are removed based on the filtering results, including:

[0022] Find each connected region in the image that is only marked as infrared obstacle feature points; calculate the area of ​​each infrared obstacle connected region, and remove infrared obstacle feature points whose area is less than or equal to the second preset threshold.

[0023] Unmarked depth obstacle feature points are filtered by area region. Based on the filtering results, points with an area region smaller than or equal to a first preset area are removed, including:

[0024] Find all connected regions in the image that are marked only as deep obstacle feature points; calculate the area of ​​each deep obstacle connected region, and remove deep obstacle feature points whose area is less than or equal to a first preset threshold.

[0025] As an optional approach, a region of interest is established around the remaining infrared obstacle feature points. The average infrared intensity of this region of interest is compared with the average pixel intensity of the projection plane. Based on the comparison result, infrared obstacle feature points that meet preset conditions are marked as target obstacle points, including:

[0026] Regions of interest are set near the connected domains of the remaining infrared obstacle feature points, and the average infrared intensity of the region of interest is calculated. Then, the average infrared intensity is compared with the average pixel intensity in the connected domain. If the comparison value meets the preset interval, the infrared obstacle feature point is removed and considered as the projected background. Infrared obstacle feature points that do not meet the preset interval are regarded as target obstacle points.

[0027] On the other hand, the present invention also provides a projection method, which uses the obstacle recognition method described above to perform obstacle recognition;

[0028] And based on the obstacle recognition results, avoid the recognized obstacles and project them.

[0029] As an alternative approach, projection can avoid identified obstacles based on obstacle recognition results, including:

[0030] Obstacle feature maps in the image sensor coordinate system are obtained based on obstacle recognition results;

[0031] Based on the homography mapping relationship between the infrared segmentation map and the projected image, the obstacle feature map is mapped to the projection coordinate system to obtain the obstacle avoidance mask in the projection coordinate system.

[0032] Based on the obstacle avoidance masking results, the final lower point value of the projection is found according to the preset projection ratio and then projected.

[0033] As an optional approach, methods for calculating homography include:

[0034] Select a preset number of points in the plane and convert them into a three-dimensional point set; combine the calibration information of the projection device and convert them into three-dimensional points in the projection coordinate system; then map the three-dimensional points in the projection coordinate system back to the projection map to obtain the corresponding two-dimensional projection coordinates.

[0035] Based on projected two-dimensional coordinates, the homography matrix is ​​calculated using two-dimensional points in the infrared sensor and the corresponding two-dimensional points in the projected image.

[0036] On the other hand, the present invention also provides a projection device, comprising:

[0037] The projection module is equipped with a lens to receive electrical signals and convert them into optical signals for projection.

[0038] The sensing module includes an inertial sensor, a depth sensor, and / or an infrared sensor, used to acquire environmental information at the target projection location;

[0039] The image recognition module is used to perform image recognition based on feedback from the sensing module, construct three-dimensional spatial data of the current projection environment based on environmental information, and extract deep obstacle feature points based on the three-dimensional spatial data; construct an infrared segmentation map of the projection plane based on environmental information, and extract infrared obstacle feature points based on the infrared segmentation map;

[0040] The filtering module is used to filter deep obstacle feature points and infrared obstacle feature points, and mark the deep obstacle feature points and infrared obstacle feature points that meet the conditions as target obstacle points.

[0041] The filtering module filters for depth obstacle feature points and infrared obstacle feature points, including:

[0042] Feature points that belong to both deep obstacle feature points and infrared obstacle feature points are marked, and all marked feature points are regarded as target obstacle points.

[0043] Area region filtering is performed on unmarked depth obstacle feature points. Points with an area region smaller than or equal to a first preset area are removed based on the filtering results, and the remaining points are regarded as target obstacle points.

[0044] Unmarked infrared obstacle feature points are filtered by area region. Points with an area region smaller than or equal to a second preset area are removed based on the filtering results. Interest regions are established around the remaining infrared obstacle feature points. The average infrared intensity of the interest region is compared with the average pixel intensity of the projection plane. Based on the comparison results, infrared obstacle feature points that meet the preset conditions are marked as target obstacle points.

[0045] The beneficial effects of this invention are as follows:

[0046] This invention utilizes feature extraction in both the infrared and depth dimensions to jointly filter depth obstacle feature points and infrared obstacle feature points, enabling more effective obstacle identification and automatic, seamless obstacle avoidance. It eliminates the need to interrupt the user to project a pre-defined image for obstacle avoidance calculations, thus improving the user experience. Attached Figure Description

[0047] Figure 1 This is a schematic flowchart of the obstacle recognition method provided in an embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram of the obstacle avoidance lower point projection after the segmentation result is mapped according to an embodiment of the present invention. Detailed Implementation

[0049] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to specific embodiments.

[0050] Example

[0051] Please see Figure 1 This embodiment provides an obstacle recognition method, including:

[0052] Environmental information is collected at the target projection location using an image sensor; three-dimensional spatial data of the current projection environment is constructed based on the environmental information, and depth obstacle feature points are extracted based on the three-dimensional spatial data; an infrared segmentation map of the projection plane is constructed based on the environmental information, and infrared obstacle feature points are extracted based on the infrared segmentation map; the depth obstacle feature points and infrared obstacle feature points are filtered, and those that meet the conditions are marked as target obstacle points. Specifically, data collection can be initiated either by first determining whether the preconditions for data collection are met using the built-in inertial measurement unit (such as an inertial sensor) or by user triggering. In one optional method, when the projector (model and type are not limited in this embodiment) is moved to the viewing position, the stationary status of the projection is determined using data from the built-in IMU. If the projection is stationary or the function is triggered by an interface button, the required data collection begins.

[0053] The aforementioned method of collecting environmental information at the target projection location using an image sensor is implemented in this embodiment by collecting depth data of the current projection environment using a depth sensor. Then, based on the environmental information, three-dimensional spatial data of the current projection environment is constructed, and depth obstacle feature points are extracted based on the three-dimensional spatial data. Specifically, this includes collecting depth data of the current environment using a depth sensor and collecting acceleration and angular velocity data of the current device using an inertial measurement unit. The depth data is then used to first fit the plane equation of the projection plane. The plane can be represented by the equation Ax + By + Cz + D = 0. Next, three-dimensional point cloud data is obtained from the depth data, with the coordinates of each point being (x, y, z). The plane is fitted using the least squares method to minimize the sum of the distances from all points to the plane. The fitted plane equation is obtained using the fitted plane equation parameters A, B, C, and D, where A, B, and C are the plane's normal vectors, and D is the plane's offset.

[0054] Calculate the distance d_pt from each point to the plane.

[0055]

[0056] Pixels with a distance greater than a preset threshold from d_pt are marked as depth obstacle feature points. The preset threshold can be selected within the range of 30-70mm, such as 30mm, 50mm, 60mm or 70mm. The flatness weight of the projection plane can be determined according to the actual projection environment and angle, such as the projection effect affected by wall protrusions, the screen being pushed up from behind, etc.

[0057] The process of acquiring infrared obstacle feature points includes: firstly, collecting infrared data of the current projection environment through an infrared sensor, then inputting the infrared data into a trained infrared wall segmentation network to obtain an infrared segmentation map, and marking obstacles in the projection background as infrared obstacle feature points to construct the infrared segmentation map of the current projection environment.

[0058] To achieve the above objectives, an infrared wall segmentation network is first trained using a pre-designed deep learning network structure and collected infrared background annotation data. During use, infrared data is input into the segmentation network, and the resulting infrared segmentation map is output. Then, obstacles in the projected background are marked as infrared obstacle feature points along the dimension of infrared features.

[0059] The infrared and depth obstacle feature points obtained above cannot be used directly to avoid errors or misidentification. Instead, they can be subjected to joint depth-infrared re-judgment to improve the accuracy and comprehensiveness of obstacle point identification.

[0060] In this embodiment, firstly, feature points that simultaneously belong to both depth and infrared obstacle features are marked, and all marked feature points are considered as target obstacle points. This step only completes the first elimination of obstacle points with overlapping features; it does not yet eliminate specific obstacle points based on their feature attribute information. Therefore, for the remaining infrared or depth obstacle feature attribute points, this embodiment uses small-area filtering on both the infrared and depth dimensions to complete the second elimination.

[0061] Next, area region filtering is performed on unmarked infrared obstacle feature points and depth obstacle feature points. Points with an area region smaller than or equal to the preset area are removed based on the filtering results, in order to reduce errors and increase the fault tolerance effect of obstacle avoidance recognition.

[0062] As an alternative approach, unmarked depth obstacle feature points can be filtered by area region. Points with an area region smaller than or equal to a first preset area are removed based on the filtering results. This first preset area can be 2-10 mm, such as 2 mm, 5 mm, 8 mm, or 10 mm. This is because different scenarios may indeed have uneven projection planes, but considering projection requirements and the projection range of the projector, a small number of less severe depth obstacle feature points can be ignored.

[0063] Unmarked infrared obstacle feature points are filtered by area region, and points with an area region smaller than or equal to a second preset area are removed based on the filtering results. The second preset area can be 20-60mm, such as 20mm, 30mm, 50mm, or 60mm. This is because different scenarios may indeed have noise on the projection plane, and since projectors generally have high light intensity, a small number of infrared obstacle feature points with small areas can be ignored.

[0064] Regions of interest (ROIs) are established around the remaining infrared obstacle feature points. The average infrared intensity of these ROIs is compared with the average pixel intensity of the projection plane. Based on the comparison results, infrared obstacle feature points that meet preset conditions are marked as target obstacle points. This can be achieved by calculating the average infrared intensity I_ROI in the adjacent ROIs of the connected region (e.g., 15*15, 10*10, 20*20, etc., depending on the actual projection size and scale), and then comparing it with the pixels within the connected infrared region. If the compared value falls within a preset range, the infrared obstacle feature point is discarded and considered part of the projection background. Specifically:

[0065] Abs(I_(Infrared Barrier)–I_ROI) <th_ir

[0066] Where Abs represents the operation of taking the absolute value, and th_ir represents the infrared intensity approaching the preset threshold. Infrared obstacle feature points that satisfy the formula are deleted and considered as projected background, thus completing the third removal.

[0067] It should be understood that in some other embodiments, only one or two of the above-described filtering methods may be used to filter obstacle feature points. For example, only feature points that simultaneously belong to depth obstacle feature points and infrared obstacle feature points may be considered as target obstacle points; area region filtering may also be performed, etc. Based on the above three filtering methods, this can eliminate the possibility of incorrectly identified non-obstacle points to the greatest extent possible.

[0068] On the other hand, this embodiment also provides a projection method that uses the obstacle recognition method described above to identify obstacles, and then projects the image around the identified obstacles based on the obstacle recognition results. This achieves the effect of seamless obstacle avoidance, eliminating the need for separate image obstacle avoidance.

[0069] In this embodiment, the above-mentioned projection that avoids identified obstacles based on obstacle recognition results includes:

[0070] Obstacle feature maps in the image sensor coordinate system are obtained based on obstacle recognition results;

[0071] Based on the homography mapping relationship between the infrared segmentation map and the projected image, the obstacle feature map is mapped to the projection coordinate system to obtain the obstacle avoidance mask in the projection coordinate system.

[0072] Based on the obstacle avoidance masking results, the final lower point value of the projection is found according to the preset projection ratio and then projected.

[0073] Optionally, the homography mapping relationship between the infrared segmentation image and the projected image can be obtained by constructing a homography matrix from the projection device to the projection plane, thus obtaining the required segmented image mapping. Then, by calculating the obstacle feature map on the projection plane and the projection coordinate system based on the homography mapping relationship, an obstacle avoidance mask in the projection coordinate system is obtained. Finally, based on the result of this obstacle avoidance mask, the final bottom point value of the projection is found according to a preset projection ratio for display.

[0074] This embodiment uses the following method to perform homography matrix operations:

[0075] Select N points in a plane and convert them into a three-dimensional point set P.

[0076]

[0077] in K is the index of the selected point in the projection plane. t These are the intrinsic parameters of the infrared sensor, obtained through prior calibration; [u i v i ] represents the x and y coordinates of a point on a plane in the infrared coordinate system, Z. i It is the depth value of that point.

[0078] By combining the calibration information of the equipment, the corresponding points are converted into three-dimensional points in the projected coordinate system by combining the extrinsic parameters obtained from the calibration.

[0079]

[0080] in R pi and T pi These represent the rotation matrix and translation vector from the infrared sensor to the projection, respectively. It is the three-dimensional point of the point with index number i in the projected coordinate system.

[0081] Then, the three-dimensional points in the projected coordinate system are mapped back to the projected image to obtain their corresponding two-dimensional projected coordinates.

[0082]

[0083] Among them, K P These are intrinsic parameters of the projection, obtained through calibration; Z pro The depth values ​​of the transformed points are used for normalization; These are the projected two-dimensional coordinates of the point with index number i.

[0084] Then, the homography matrix is ​​calculated using two-dimensional points in the infrared sensor and their corresponding two-dimensional points in the projected image. It is obtained using the least squares method, and the formula is as follows:

[0085]

[0086] Where H is a 3x3 homography matrix.

[0087] The homography matrix H represents the homography mapping between the infrared segmentation image and the projected image. By mapping the obstacle avoidance segmentation result onto the coordinate system of the projected image, the obstacle avoidance mask I is obtained. p (res):

[0088] I p (res) = H*I t

[0089] Among them, I t This represents the obstacle feature map obtained after the review and filtering steps. It is a pixel representation in the depth-infrared coordinate system and needs to be mapped to the projection coordinate system through homography to obtain the final obstacle avoidance mask I in the projection coordinate system. p (res).

[0090] Finally, according to the set ratio, such as 16:9 or 4:3, the obstacle avoidance mask I... p (res) calculates the final projection point value, finds the optimal projection area, displays the final image, and completes the seamless projection.

[0091] Thus, this embodiment combines depth sensors, infrared sensors, IMUs, and other sensors to propose depth obstacle avoidance features and infrared obstacle avoidance features, and combines data from various sources for re-judgment and filtering to achieve a projection scheme that ultimately achieves seamless obstacle avoidance (e.g., Figure 2 (As shown).

[0092] In addition, this embodiment also provides a projection device with non-sensory obstacle avoidance, including:

[0093] The projection module is equipped with a lens to receive electrical signals and convert them into optical signals for projection.

[0094] The sensing module includes an inertial sensor, a depth sensor, and / or an infrared sensor, used to acquire environmental information at the target projection location;

[0095] The image recognition module is used to perform image recognition based on feedback from the sensing module, construct three-dimensional spatial data of the current projection environment based on environmental information, and extract deep obstacle feature points based on the three-dimensional spatial data; construct an infrared segmentation map of the projection plane based on environmental information, and extract infrared obstacle feature points based on the infrared segmentation map;

[0096] The filtering module is used to filter deep obstacle feature points and infrared obstacle feature points, and mark the deep obstacle feature points and infrared obstacle feature points that meet the conditions as target obstacle points.

[0097] As an optional method, the filtering module filters depth obstacle feature points and infrared obstacle feature points, including:

[0098] Feature points that belong to both deep obstacle feature points and infrared obstacle feature points are marked, and all marked feature points are regarded as target obstacle points.

[0099] Area region filtering is performed on unmarked depth obstacle feature points. Points with an area region smaller than or equal to a first preset area are removed based on the filtering results, and the remaining points are regarded as target obstacle points.

[0100] Unmarked infrared obstacle feature points are filtered by area region. Points with an area region smaller than or equal to a second preset area are removed based on the filtering results. Interest regions are established around the remaining infrared obstacle feature points. The average infrared intensity of the interest region is compared with the average pixel intensity of the projection plane. Based on the comparison results, infrared obstacle feature points that meet the preset conditions are marked as target obstacle points.

[0101] The above are merely preferred embodiments of the present invention. It should be noted that the above preferred embodiments should not be considered as limitations on the present invention, and the scope of protection of the present invention should be determined by the scope defined in the claims. For those skilled in the art, several improvements and modifications can be made without departing from the spirit and scope of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An obstacle recognition method characterized by, include: Environmental information is collected at the target projection location using a sensing module; Based on the environmental information, three-dimensional spatial data of the current projection environment is constructed, and feature points of deep obstacles are extracted based on the three-dimensional spatial data. An infrared segmentation map of the projection plane is constructed based on the environmental information, and infrared obstacle feature points are extracted based on the infrared segmentation map. Feature points that belong to both deep obstacle feature points and infrared obstacle feature points are marked, and all marked feature points are regarded as target obstacle points. Unmarked infrared obstacle feature points are filtered by area region. Points with an area region smaller than or equal to a second preset area are removed based on the filtering results. Interest regions are established around the remaining infrared obstacle feature points. The average infrared intensity of the interest region is compared with the average pixel intensity of the projection plane. Based on the comparison results, infrared obstacle feature points that meet the preset conditions are marked as target obstacle points.

2. The obstacle recognition method according to claim 1, wherein The step of collecting environmental information at the target projection location via a sensing module includes: The depth data of the current projection environment is collected by the depth sensor in the sensing module; the infrared data of the current projection environment is collected by the infrared sensor in the sensing module. The step of constructing three-dimensional spatial data of the current projection environment based on the environmental information, and extracting depth obstacle feature points based on the three-dimensional spatial data, includes: Three-dimensional point cloud data is obtained from the depth data, the projection plane is fitted by the least squares method to obtain the plane equation, the distance from each point to the plane is calculated, and pixels whose distance from the plane is greater than a preset threshold are marked as depth obstacle feature points. The step of constructing an infrared segmentation map of the projection plane based on the environmental information, and extracting infrared obstacle feature points based on the infrared segmentation map, includes: The infrared data is fed into a trained infrared wall segmentation network to obtain an infrared segmentation map, and obstacles in the projected background are marked as infrared obstacle feature points.

3. The obstacle recognition method according to claim 1, characterized in that, Unmarked depth obstacle feature points are filtered by area region. Points with an area region smaller than or equal to a first preset area are removed based on the filtering results, and the remaining points are regarded as target obstacle points.

4. The obstacle recognition method according to claim 3, characterized in that, The step of filtering unmarked infrared obstacle feature points by area, and removing points whose area is less than or equal to a second preset area based on the filtering results, includes: Find each connected region in the image that is only marked as infrared obstacle feature points; calculate the area of ​​each infrared obstacle connected region, and remove infrared obstacle feature points whose area is less than or equal to the second preset threshold. The step of filtering unmarked depth obstacle feature points by area region, and removing points whose area region is less than or equal to a first preset area based on the filtering result, includes: Find all connected regions in the image that are marked only as deep obstacle feature points; calculate the area of ​​each deep obstacle connected region, and remove deep obstacle feature points whose area is less than or equal to a first preset threshold.

5. The obstacle recognition method according to claim 1, characterized in that, The step of establishing a region of interest around the remaining infrared obstacle feature points, comparing the average infrared intensity of the region of interest with the average pixel intensity of the projection plane, and marking the infrared obstacle feature points that meet the preset conditions as target obstacle points based on the comparison results includes: Regions of interest are set near the connected domains of the remaining infrared obstacle feature points, and the average infrared intensity of the region of interest is calculated. Then, the average infrared intensity is compared with the average pixel intensity in the connected domain. If the comparison value meets the preset interval, the infrared obstacle feature point is removed and considered as the projected background. Infrared obstacle feature points that do not meet the preset interval are regarded as target obstacle points.

6. A projection method, characterized in that, include: Obstacle identification is performed using the obstacle identification method described in any one of claims 1-5 above; Projecting avoids the identified obstacles based on the obstacle recognition results.

7. A projection method according to claim 6, characterized in that, The projection based on obstacle recognition results to avoid recognized obstacles includes: Obstacle feature maps in the depth-infrared coordinate system are obtained based on the obstacle recognition results; Based on the homography mapping relationship between the infrared segmentation map and the projected image, the obstacle feature map is mapped to the projection coordinate system to obtain the obstacle avoidance mask in the projection coordinate system. Based on the obstacle avoidance masking results, the final lower point value of the projection is found according to the preset projection ratio and then projected.

8. A projection method according to claim 7, characterized in that, The method for calculating the homography mapping relationship includes: Select a preset number of points in the infrared coordinate system plane and convert them into a three-dimensional point set; combine the calibration information of the projection device and convert them into three-dimensional points in the projection coordinate system; then map the three-dimensional points in the projection coordinate system back to the projection map to obtain the corresponding two-dimensional projection coordinates. Based on projected two-dimensional coordinates, the homography matrix is ​​calculated using two-dimensional points in the infrared sensor and the corresponding two-dimensional points in the projected image.

9. A projection device, characterized in that, include: A projection module, which is equipped with a lens, is used to receive electrical signals and convert them into optical signals for projection; A sensing module, comprising an inertial sensor, a depth sensor, and / or an infrared sensor, for acquiring environmental information at the target projection location; An image recognition module is used to perform image recognition based on feedback from the sensing module, construct three-dimensional spatial data of the current projection environment based on the environmental information, and extract depth obstacle feature points based on the three-dimensional spatial data; construct an infrared segmentation map of the projection plane based on the environmental information, and extract infrared obstacle feature points based on the infrared segmentation map; The filtering module is used to mark feature points that belong to both deep obstacle feature points and infrared obstacle feature points, and to regard all marked feature points as target obstacle points. Unmarked infrared obstacle feature points are filtered by area region. Points with an area region smaller than or equal to a second preset area are removed based on the filtering results. Interest regions are established around the remaining infrared obstacle feature points. The average infrared intensity of the interest region is compared with the average pixel intensity of the projection plane. Based on the comparison results, infrared obstacle feature points that meet the preset conditions are marked as target obstacle points.

10. A projection device according to claim 9, characterized in that, The filtering module also includes: Unmarked depth obstacle feature points are filtered by area region. Points with an area region smaller than or equal to a first preset area are removed based on the filtering results, and the remaining points are regarded as target obstacle points.

Images