Obstacle recognition method, projection method and projection apparatus

Abstract

Disclosed in the present invention are an obstacle recognition method, a projection method and a projection apparatus. The method comprises: first, collecting environmental information by means of an image sensor at a target projection position; constructing three-dimensional spatial data of a current projection environment on the basis of the environmental information, and extracting depth obstacle feature points on the basis of the three-dimensional spatial data; constructing an infrared segmentation map of a projection plane on the basis of the environmental information, and extracting infrared obstacle feature points on the basis of the infrared segmentation map; and filtering the depth obstacle feature points and the infrared obstacle feature points, and labeling as target obstacle points the depth obstacle feature points and the infrared obstacle feature points which satisfy conditions. The present invention takes the depth data and the infrared data into account to achieve a seamless obstacle avoidance projection effect based on depth obstacle avoidance features and infrared obstacle avoidance features.

Description

Obstacle identification method, projection method and projection device Technical Field

[0001] The present invention relates to the technical field of projection obstacle avoidance, and in particular to an obstacle recognition method, a projection method and a projection device. Background Art

[0002] Due to the characteristics of the usage scenarios, the position of the projector is often moved during use. After the entire machine is moved, the image projected on the screen or wall is deformed. When the projected image is projected onto the viewing surface, there are often obstacles such as hanging paintings, ornaments, switches, etc., which affect the viewing experience.

[0003] Obstacles are usually identified by human eyes. Before setting up the projector, it is necessary to carefully examine the projection surface such as the screen. Although obvious obstacles can be eliminated, in some special projection environments, uneven projection surfaces cannot be accurately identified by the naked eye, causing obstacles to affect the projection effect.

[0004] Currently, the common practice is to use the projected UI image function to implement obstacle avoidance calculations for the projected image. This requires the projector to project a feature-rich image, which is then captured by the device's camera. Using image processing algorithms, the projected image is then calculated to determine the optimal location for the projection. This method interrupts the user's normal use and takes a long time to process, typically several seconds, making it a less user-friendly experience. Summary of the Invention

[0005] In view of this, the present invention provides an obstacle recognition method, a projection method, and a projection device to combine depth data and infrared data to more effectively recognize obstacles based on depth obstacle avoidance features and infrared obstacle avoidance features.

[0006] To solve the above technical problems, the present invention provides an obstacle recognition method, comprising:

[0007] Collect environmental information through image sensors at the target projection position;

[0008] Construct the three-dimensional spatial data of the current projection environment based on the environmental information, and extract the depth obstacle feature points based on the three-dimensional spatial data;

[0009] Construct an infrared segmentation map of the projection plane based on environmental information, and extract infrared obstacle feature points based on the infrared segmentation map;

[0010] The depth obstacle feature points and infrared obstacle feature points are filtered, and the depth obstacle feature points and infrared obstacle feature points that meet the conditions are marked as target obstacle points.

[0011] As an optional method, collecting environmental information at the target projection location through an image sensor includes:

[0012] The depth data of the current projection environment is collected through the depth sensor; the infrared data of the current projection environment is collected through the infrared sensor;

[0013] Construct the three-dimensional spatial data of the current projection environment based on the environmental information, and extract the depth obstacle feature points based on the three-dimensional spatial data, including:

[0014] Obtain 3D point cloud data from the depth data, fit the projection plane using the least squares method to obtain the plane equation, and calculate the distance from each point to the plane; and mark the pixels whose distance from the point to the plane is greater than a preset threshold as depth obstacle feature points;

[0015] An infrared segmentation map of the projection plane is constructed based on environmental information, and infrared obstacle feature points are extracted based on the infrared segmentation map, including:

[0016] The infrared data is passed into the trained infrared wall segmentation network to obtain the infrared segmentation map, and the obstacles in the projected background are marked as infrared obstacle feature points.

[0017] As an optional method, deep obstacle feature points and infrared obstacle feature points are filtered and qualified deep obstacle feature points and infrared obstacle feature points are marked as target obstacle points, including:

[0018] Mark the feature points that are both depth obstacle feature points and infrared obstacle feature points, and regard all marked feature points as target obstacle points;

[0019] Performing area filtering on the unmarked depth obstacle feature points, eliminating points whose area is less than or equal to the first preset area according to the filtering results, and treating the remaining points as target obstacle points;

[0020] Perform area filtering on the unmarked infrared obstacle feature points, eliminate points with an area less than or equal to a second preset area based on the filtering results, establish a region of interest around the remaining infrared obstacle feature points, compare the average infrared intensity of the region of interest with the average pixel intensity of the projection plane, and mark the infrared obstacle feature points that meet the preset conditions as target obstacle points based on the comparison results.

[0021] As an optional method, performing area filtering on unmarked infrared obstacle feature points and eliminating points with an area less than or equal to a second preset area according to the filtering result includes:

[0022] Finding connected regions in the image that are marked only as infrared obstacle feature points; calculating the area of ​​each infrared obstacle connected region, and removing infrared obstacle feature points whose area is less than or equal to a second preset threshold;

[0023] Performing area filtering on unmarked depth obstacle feature points, and eliminating points whose area is less than or equal to a first preset area according to the filtering result, including:

[0024] Find the connected regions in the image that are marked only as depth obstacle feature points; calculate the area of ​​each depth obstacle connected region, and remove depth obstacle feature points whose area is less than or equal to a first preset threshold.

[0025] As an optional method, a region of interest is set up around the remaining infrared obstacle feature points. The average infrared intensity of the region of interest is compared with the average pixel intensity of the projection plane. Based on the comparison results, the infrared obstacle feature points that meet the preset conditions are marked as target obstacle points, including:

[0026] An interest region is set near the connected domain of the remaining infrared obstacle feature points, and the average infrared intensity of the interest region is calculated. The average infrared intensity is then compared with the average pixel intensity in the connected domain. If the compared value meets the preset interval, the infrared obstacle feature point is eliminated and considered to be the projection background. The infrared obstacle feature points that do not meet the preset interval are regarded as target obstacle points.

[0027] On the other hand, the present invention also provides a projection method, which uses the above-mentioned obstacle recognition method to perform obstacle recognition;

[0028] And projection is performed based on the obstacle recognition results to avoid the identified obstacles.

[0029] As an optional method, projection is performed to avoid identified obstacles based on the obstacle recognition results, including:

[0030] Obtain an obstacle feature map in the image sensor coordinate system based on the obstacle recognition result;

[0031] Based on the homography mapping relationship between the infrared segmentation map and the projection image, the obstacle feature map is mapped to the projection coordinate system to obtain the obstacle avoidance mask in the projection coordinate system;

[0032] Based on the obstacle avoidance mask result, the final lower point value of the projection is found according to the preset projection scale for projection.

[0033] As an optional method, the calculation method of the homography mapping relationship includes:

[0034] Select a preset number of points in the plane and convert them into a 3D point set; combine the calibration information of the projection device to convert them into 3D points in the projection coordinate system, and then map the 3D points in the projection coordinate system back to the projection image to obtain the corresponding projected 2D coordinates;

[0035] Based on the projected 2D coordinates, the homography matrix is ​​calculated by the 2D points in the infrared sensor and the corresponding 2D points in the projected image.

[0036] On the other hand, the present invention further provides a projection device, comprising:

[0037] The projection module is provided with a lens for receiving electrical signals and converting them into optical signals for projection;

[0038] A sensing module, which includes an inertial sensor, a depth sensor, and / or an infrared sensor, and is used to collect environmental information at the target projection location;

[0039] The image recognition module is used to perform image recognition based on the feedback from the sensor module, construct three-dimensional spatial data of the current projection environment based on the environmental information, and extract deep obstacle feature points based on the three-dimensional spatial data; construct an infrared segmentation map of the projection plane based on the environmental information, and extract infrared obstacle feature points based on the infrared segmentation map;

[0040] The screening module is used to filter the depth obstacle feature points and infrared obstacle feature points, and mark the depth obstacle feature points and infrared obstacle feature points that meet the conditions as target obstacle points.

[0041] The filtering module filters depth obstacle feature points and infrared obstacle feature points, including:

[0042] Mark the feature points that are both depth obstacle feature points and infrared obstacle feature points, and regard all marked feature points as target obstacle points;

[0043] Performing area filtering on the unmarked depth obstacle feature points, eliminating points whose area is less than or equal to the first preset area according to the filtering results, and treating the remaining points as target obstacle points;

[0044] Perform area filtering on the unmarked infrared obstacle feature points, eliminate points with an area less than or equal to a second preset area based on the filtering results, establish a region of interest around the remaining infrared obstacle feature points, compare the average infrared intensity of the region of interest with the average pixel intensity of the projection plane, and mark the infrared obstacle feature points that meet the preset conditions as target obstacle points based on the comparison results.

[0045] The beneficial effects of the present invention are:

[0046] This invention uses infrared and depth feature extraction to jointly filter depth and infrared obstacle feature points, effectively identifying obstacles and enabling automatic, seamless obstacle avoidance. This eliminates the need to interrupt users to project a pre-defined image for obstacle avoidance calculations, improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] FIG1 is a schematic diagram of a flow chart of an obstacle identification method provided by an embodiment of the present invention;

[0048] FIG2 is a schematic diagram of a projection of a lower point of obstacle avoidance after mapping of a segmentation result provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with specific implementation methods.

[0050] Example

[0051] Referring to FIG. 1 , this embodiment provides an obstacle recognition method, including:

[0052] Collect environmental information through the image sensor at the target projection position; construct the three-dimensional spatial data of the current projection environment based on the environmental information, and extract the depth obstacle feature points based on the three-dimensional spatial data; construct an infrared segmentation map of the projection plane based on the environmental information, and extract the infrared obstacle feature points based on the infrared segmentation map; filter the depth obstacle feature points and infrared obstacle feature points, and mark the qualified depth obstacle feature points and infrared obstacle feature points as target obstacle points. Specifically, it can be judged according to the built-in inertial measurement unit (such as an inertial sensor) that the acquisition preconditions are met before acquisition or it can be triggered by the user. In an optional manner, when the projector (the model and type are not limited in this embodiment) is moved to the viewing position, the data of the built-in IMU is used to judge whether the projection is still. If it is still or the function is triggered by the interface button. Then start collecting the required data.

[0053] The above-mentioned environmental information is collected by the image sensor at the target projection position. In this embodiment, the depth data of the current projection environment is collected by the depth sensor, and then the three-dimensional spatial data of the current projection environment is constructed based on the environmental information, and the depth obstacle feature points are extracted based on the three-dimensional spatial data. Specifically, it includes collecting the depth data of the current environment through the depth sensor and collecting the acceleration and angular velocity data of the current device through the inertial measurement unit. Then, the depth data is used to first fit the plane equation of the projection plane. The plane can be expressed by the equation Ax+By+Cz+D=0. Then, the three-dimensional point cloud data is obtained from the depth data, and the coordinates of each point are (x, y, z). The plane is fitted by the least squares method so that the sum of the distances from all points to the plane is minimized. The plane equation parameters A, B, C, and D obtained by fitting can be used to obtain the fitted plane equation, where A, B, and C are the normal vectors of the plane, and D is the offset of the plane.

[0054] Calculate the distance d_pt from each point to the plane,

[0055] Pixel points whose distance d_pt is greater than a preset threshold are marked as depth obstacle feature points. The preset threshold can be selected within 30-70mm, such as 30mm, 50mm, 60mm or 70mm. The flatness weight of the projection plane can be determined according to the actual projection environment and angle, such as the projection effect affected by the protrusion of the wall or the curtain being lifted up from the back.

[0056] The process of acquiring infrared obstacle feature points includes: first, collecting infrared data of the current projection environment through an infrared sensor, passing the infrared data to a trained infrared wall segmentation network to obtain an infrared segmentation map, marking obstacles in the projection background as infrared segmentation maps to construct the current projection environment, and marking obstacles in the projection background as infrared obstacle feature points.

[0057] To achieve this, we first train an infrared wall segmentation network using a pre-designed deep learning network structure and collected infrared background annotated data. During use, infrared data is fed into the segmentation network, which outputs an infrared segmentation map. Obstacles in the projected background are then marked as infrared obstacle feature points in the infrared feature dimension.

[0058] The infrared obstacle feature points and depth obstacle feature points obtained above cannot be used directly to avoid errors or misidentification. Instead, they can be subjected to depth-infrared joint re-judgment to improve the accuracy and comprehensiveness of obstacle point identification.

[0059] In this embodiment, feature points that are both depth and infrared obstacle feature points are first marked, and all marked feature points are considered target obstacle points. This step only completes the first obstacle point elimination process for overlapping features; it does not yet eliminate specific obstacle points based on their characteristic attribute information. Therefore, for the remaining infrared or depth obstacle feature attribute points, this embodiment uses small area filtering in both the infrared and depth dimensions to complete the second elimination process.

[0060] Afterwards, the unmarked infrared obstacle feature points and depth obstacle feature points are filtered by area, and points with an area less than or equal to the preset area are eliminated according to the filtering results to reduce errors and increase the fault tolerance of obstacle avoidance recognition.

[0061] As an optional method, unmarked depth obstruction feature points are filtered by area. Based on the filtering results, points with an area less than or equal to a first preset area are removed. The first preset area can be 2-10mm, such as 2mm, 5mm, 8mm, or 10mm. This is because different scenes may indeed have uneven projection planes, but considering the projection requirements and the projection range of the projector, a small number of depth obstruction feature points with less severe problems can be ignored.

[0062] Unmarked infrared obstacle feature points are filtered by area, and points with an area less than or equal to a second preset area are removed based on the filtering results. The second preset area can be 20-60 mm, such as 20 mm, 30 mm, 50 mm, or 60 mm. This is because different scenes may indeed cause noise to appear on the projection plane. Due to the generally high illumination intensity of projectors, a small number of smaller infrared obstacle feature points can be ignored.

[0063] Set up a region of interest around the remaining infrared obstacle feature points, compare the average infrared intensity of the region of interest with the average pixel intensity of the projection plane, and mark the infrared obstacle feature points that meet the preset conditions as target obstacle points based on the comparison results. What can be achieved is that the average infrared intensity mean I_ROI is calculated in the area adjacent to the region of interest (such as 15*15 size, or 10*10, 20*20, etc.. It can be confirmed based on the actual projection size and ratio) of the connected area, and then compared with the pixels in the infrared connected area. If the compared value meets the preset interval, the infrared obstacle feature point is eliminated and considered to be the projection background. Specifically:

[0064] Abs(I_(infrared obstacle)–I_ROI) <th_ir

[0065] Abs represents the absolute value operation, th_ir represents the infrared intensity threshold close to the preset value, and the infrared obstacle feature points that meet the formula are deleted and considered to be the projection background, completing the third elimination.

[0066] It should be understood that in other embodiments, only one or two of the above filtering methods may be used to filter obstacle feature points. For example, only feature points that are both depth obstacle feature points and infrared obstacle feature points may be used as target obstacle points; or only area filtering may be performed. Based on the above three filtering methods, the misidentification of non-obstacle points can be eliminated to the greatest extent possible.

[0067] On the other hand, this embodiment also provides a projection method that uses the above-mentioned obstacle recognition method to identify obstacles, and then avoids the identified obstacles based on the obstacle recognition results during projection. This eliminates the need for separate image obstacle avoidance and achieves the effect of seamless obstacle avoidance.

[0068] In this embodiment, the above-mentioned projection based on the obstacle recognition result while avoiding the recognized obstacles includes:

[0069] Obtain an obstacle feature map in the image sensor coordinate system based on the obstacle recognition result;

[0070] Based on the homography mapping relationship between the infrared segmentation map and the projection image, the obstacle feature map is mapped to the projection coordinate system to obtain the obstacle avoidance mask in the projection coordinate system;

[0071] Based on the obstacle avoidance mask result, the final lower point value of the projection is found according to the preset projection scale for projection.

[0072] Alternatively, the required segmented image mapping can be obtained by constructing a homography matrix from the projection device to the projection plane, obtaining a homography mapping relationship between the infrared segmentation map and the projected image. Furthermore, an obstacle avoidance mask in the projected coordinate system is calculated by calculating the projection coordinate system based on the obstacle feature map on the projection plane and the homography mapping relationship. Based on the resulting obstacle avoidance mask, the final lower point of the projection is found according to a preset projection scale for display.

[0073] This embodiment uses the following method to perform the calculation of the homography matrix:

[0074] Select N points in the plane and convert them into a three-dimensional point set P.

[0075] in is the index of the selected point in the projection plane, K t It is the internal reference of the infrared sensor, which is obtained by calibration in advance; is the horizontal and vertical coordinates of the plane point in the infrared coordinate system, Z i is the depth value of the point.

[0076] Combined with the calibration information of the device, the corresponding points are then combined with the external parameters obtained by calibration to convert them into three-dimensional points in the projected coordinate system.

[0077] in R pt and T pt They represent the rotation matrix and translation vector from the infrared sensor to the projection respectively; It is the three-dimensional point of the point with index number i in the projected coordinate system.

[0078] Then the three-dimensional point in the projected coordinate system is mapped back to the projection image to obtain its corresponding projected two-dimensional coordinate.

[0079] Among them, K p is the internal parameter of the projection, obtained by calibration; Z pro is the depth value of the transformed point for normalization; It is the projected two-dimensional coordinate corresponding to the point with index number i.

[0080] Then, the homography matrix is ​​calculated by the two-dimensional points in the infrared sensor and the corresponding two-dimensional points in the projected image. The least squares method is used to calculate it, and the calculation formula is:

[0081] Where H is a 3*3 homography matrix.

[0082] The homography matrix H is the homography mapping relationship between the infrared segmentation image and the projection image. The obstacle avoidance segmentation result is mapped to the coordinate system of the projection image, and then the obstacle avoidance mask I is obtained. p (res): I p (res) = H * I t Among them, I t It represents the obstacle feature map obtained after the re-judgment and filtering steps. It is a pixel representation in the depth-infrared coordinate system and needs to be mapped to the projected coordinate system through the homography relationship to obtain the final obstacle avoidance mask I in the projected coordinate system. p (res).

[0083] Finally, according to the set ratio, such as 16:9, 4:3 and other common ratios, the obstacle avoidance mask I p (res) calculates the final projection point value, finds the best projection area, displays the final image, and completes the seamless projection.

[0084] In this way, this embodiment combines depth sensors, infrared sensors, IMU and other sensors to propose depth obstacle avoidance features and infrared obstacle avoidance features, and combines data from all parties for re-judgment and filtering to achieve the final projection solution with a senseless obstacle avoidance effect (as shown in Figure 2).

[0085] In addition, this embodiment also provides a projection device with seamless obstacle avoidance, including:

[0086] The projection module is provided with a lens for receiving electrical signals and converting them into optical signals for projection;

[0087] A sensing module, which includes an inertial sensor, a depth sensor, and / or an infrared sensor, and is used to collect environmental information at the target projection location;

[0088] The image recognition module is used to perform image recognition based on the feedback from the sensor module, construct three-dimensional spatial data of the current projection environment based on the environmental information, and extract deep obstacle feature points based on the three-dimensional spatial data; construct an infrared segmentation map of the projection plane based on the environmental information, and extract infrared obstacle feature points based on the infrared segmentation map;

[0089] The screening module is used to filter the depth obstacle feature points and infrared obstacle feature points, and mark the depth obstacle feature points and infrared obstacle feature points that meet the conditions as target obstacle points.

[0090] As an optional method, the screening module filters depth obstacle feature points and infrared obstacle feature points, including:

[0091] Mark the feature points that are both depth obstacle feature points and infrared obstacle feature points, and regard all marked feature points as target obstacle points;

[0092] Performing area filtering on the unmarked depth obstacle feature points, eliminating points whose area is less than or equal to the first preset area according to the filtering results, and treating the remaining points as target obstacle points;

[0093] Perform area filtering on the unmarked infrared obstacle feature points, eliminate points with an area less than or equal to a second preset area based on the filtering results, establish a region of interest around the remaining infrared obstacle feature points, compare the average infrared intensity of the region of interest with the average pixel intensity of the projection plane, and mark the infrared obstacle feature points that meet the preset conditions as target obstacle points based on the comparison results.

[0094] The above are merely preferred embodiments of the present invention. It should be noted that the above preferred embodiments should not be construed as limiting the present invention, and the scope of protection of the present invention should be determined by the scope defined in the claims. Persons skilled in the art will appreciate that improvements and modifications may be made without departing from the spirit and scope of the present invention, and such improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An obstacle recognition method, characterized in that, Including: Collecting environmental information through an image sensor at a target projection position; Constructing three-dimensional spatial data of the current projection environment based on the environmental information, and extracting depth obstacle feature points based on the three-dimensional spatial data; Constructing an infrared segmentation map of the projection plane based on the environmental information, and extracting infrared obstacle feature points based on the infrared segmentation map; Filtering the depth obstacle feature points and the infrared obstacle feature points, and marking the depth obstacle feature points and the infrared obstacle feature points that meet the conditions as target obstacle points.

2. The obstacle recognition method according to claim 1, characterized in that, The collecting environmental information through an image sensor at a target projection position includes: Collecting depth data of the current projection environment through a depth sensor; collecting infrared data of the current projection environment through an infrared sensor; The constructing three-dimensional spatial data of the current projection environment based on the environmental information, and extracting depth obstacle feature points based on the three-dimensional spatial data, includes: Obtaining three-dimensional point cloud data from the depth data, fitting a projection plane by the least squares method to obtain a plane equation, and calculating the distance from each point to the plane; and marking the pixel points whose distance from the plane is greater than a preset threshold as depth obstacle feature points; The constructing an infrared segmentation map of the projection plane based on the environmental information, and extracting infrared obstacle feature points based on the infrared segmentation map, includes: Inputting the infrared data into a trained infrared wall segmentation network to obtain an infrared segmentation map, and marking the obstacles in the projection background as infrared obstacle feature points.

3. The obstacle recognition method according to claim 1, characterized in that, The filtering the depth obstacle feature points and the infrared obstacle feature points, and marking the depth obstacle feature points and the infrared obstacle feature points that meet the conditions as target obstacle points, includes: Marking the feature points that belong to both the depth obstacle feature points and the infrared obstacle feature points, and regarding all the marked feature points as target obstacle points; Performing area region filtering on the unmarked depth obstacle feature points, removing the points whose area region is less than or equal to a first preset area according to the filtering result, and regarding the remaining points as target obstacle points; Performing area region filtering on the unmarked infrared obstacle feature points, removing the points whose area region is less than or equal to a second preset area according to the filtering result, setting an interest region around the remaining infrared obstacle feature points, comparing the average infrared intensity of the interest region with the average pixel intensity of the projection plane, and marking the infrared obstacle feature points that meet the preset conditions as target obstacle points according to the comparison result.

4. The obstacle recognition method according to claim 3, wherein The performing area region filtering on the unmarked infrared obstacle feature points, removing the points whose area region is less than or equal to a second preset area according to the filtering result, includes: Finding out each connected region in the image that is only marked as an infrared obstacle feature point; calculating the area of each infrared obstacle connected region, and removing the infrared obstacle feature points whose area is less than or equal to a second preset threshold; The performing area region filtering on the unmarked depth obstacle feature points, removing the points whose area region is less than or equal to a first preset area according to the filtering result, includes: Find out each connected region in the image that is only marked as a depth obstacle feature point; calculate the area of each connected region of the depth obstacle, and remove the depth obstacle feature points whose area is less than or equal to the first preset threshold.

5. The obstacle recognition method according to claim 3, characterized in that The step of setting an interest region around the remaining infrared obstacle feature points and comparing the average infrared intensity of the interest region with the average pixel intensity of the projection plane, and marking the infrared obstacle feature points that meet the preset conditions as target obstacle points according to the comparison result includes: Set an interest region near the adjacent connected domain of the remaining infrared obstacle feature points, and calculate the average infrared intensity mean value of the interest region; then compare the average infrared intensity mean value with the average pixel intensity value within the connected domain. If the comparison value meets the preset interval, remove the infrared obstacle feature point, considering it as the projection background, and regard the infrared obstacle feature points that do not meet the preset interval as target obstacle points.

6. A projection method, characterized in that, Including: Use the obstacle recognition method described in any one of the above claims 1-5 for obstacle recognition; Avoid the recognized obstacles based on the obstacle recognition result for projection.

7. A projection method according to claim 6, characterized in that, The step of avoiding the recognized obstacles based on the obstacle recognition result for projection includes: Obtain an obstacle feature map in the image sensor coordinate system based on the obstacle recognition result; Map the obstacle feature map to the projection coordinate system based on the homography mapping relationship between the infrared segmentation map and the projection screen, and obtain an obstacle avoidance mask in the projection coordinate system; Based on the result of the obstacle avoidance mask, find the final projection down-point value for projection according to the preset projection ratio.

8. A projection method according to claim 7, characterized in that, The calculation method of the homography mapping relationship includes: Select a preset number of points in the plane, convert them into a three-dimensional point set; combine the calibration information of the projection device, convert them into three-dimensional points in the projection coordinate system, and then map the three-dimensional points in the projection coordinate system back to the projection image to obtain the corresponding projection two-dimensional coordinates; Based on the projection two-dimensional coordinates, calculate the homography matrix through the two-dimensional points in the infrared sensor and the two-dimensional points in the corresponding projection image.

9. A projection device, characterized in that, Including: A projection module, the projection module is provided with a lens for receiving an electrical signal and converting it into an optical signal for projection; A sensing module, the sensing module includes a depth sensor and / or an infrared sensor for collecting environmental information at the target projection position; An image recognition module, the image recognition module is used for image recognition according to the feedback of the sensing module, constructing three-dimensional space data of the current projection environment based on the environmental information, and extracting depth obstacle feature points based on the three-dimensional space data; constructing an infrared segmentation map of the projection plane based on the environmental information, and extracting infrared obstacle feature points based on the infrared segmentation map; A screening module, the screening module is used for filtering depth obstacle feature points and infrared obstacle feature points, and marking the depth obstacle feature points and infrared obstacle feature points that meet the conditions as target obstacle points.

10. A projection device according to claim 9, characterized in that, The sensing module further includes an inertial measurement unit, and starts to collect the required data after judging that the acquisition precondition is reached according to the inertial measurement unit.

11. A projection device according to claim 10, characterized in that, The inertial measurement unit is an inertial sensor.

12. A projection device according to any one of claims 9-11, characterized in that, The screening module filters the depth obstacle feature points and the infrared obstacle feature points, including: Marking the feature points that belong to both the depth obstacle feature points and the infrared obstacle feature points, and regarding all the marked feature points as target obstacle points; Performing area filtering on the unmarked depth obstacle feature points, removing the points with an area less than or equal to the first preset area according to the filtering result, and regarding the remaining points as target obstacle points; Performing area filtering on the unmarked infrared obstacle feature points, removing the points with an area less than or equal to the second preset area according to the filtering result, setting an interest area around the remaining infrared obstacle feature points, comparing the average infrared intensity of the interest area with the average pixel intensity of the projection plane, and marking the infrared obstacle feature points that meet the preset conditions as target obstacle points according to the comparison result.

Images