A door detection method and storage medium combining laser and visual images

Abstract

The present invention discloses a door detection method and storage medium for fusion of laser and visual image, comprising the following steps: acquiring original laser data and visual image data; converting the laser data into a grayscale image; using a first detection model to detect the laser grayscale image respectively and extract a door characteristic curve, wherein the characteristic curve is composed of a door horizontal reference line characteristic curve and a door curve characteristic curve, and the door characteristic curve and the door horizontal reference line characteristic curve are composed of point cloud data; using a second detection model to detect the visual image data and extract door features, wherein the door features include a visual image of the area where the door is located; based on the characteristic curve, respectively obtaining the tilted frame position of the characteristic curve in the laser data grayscale image and the visual image data; using a fusion module to fuse the visual image and laser image results; the detection results of the visual image and the laser image will be matched one-to-one, and unmatched detection results will be discarded.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a door detection method and a storage medium for fusing laser and visual images. Background Art

[0002] Most common laser door detection methods are based on the original laser data and use some traditional machine learning algorithms for identification, or convert the laser into a grayscale image, and then use a convolutional neural network-based prediction model for detection, and directly output the detection results. Summary of the invention

[0003] The technical solution of the present invention to solve the above technical problem is as follows: a door detection method integrating laser and visual image, comprising the following steps:

[0004] Acquire raw laser data and visual image data;

[0005] Converting the laser data into a grayscale image;

[0006] Using the first detection model to detect the laser grayscale image and extract the door characteristic curve, wherein the characteristic curve is composed of a door horizontal reference line characteristic curve and a door curve characteristic curve, and the door characteristic curve and the door horizontal reference line characteristic curve are composed of point cloud data;

[0007] Using a second detection model to detect and extract door features from the visual image data, wherein the door features include a visual image of an area where the door is located;

[0008] Based on the characteristic curve, respectively obtain the tilted frame position of the characteristic curve in the laser data grayscale image and the visual image data;

[0009] Use the fusion module to fuse the visual image and laser image results;

[0010] The detection results of the visual image and the laser image will be matched one-to-one, and unmatched detection results will be discarded.

[0011] Furthermore, the specific steps of extracting the door characteristic curve include:

[0012] Obtaining the tilt frame position information of the laser grayscale image, wherein the tilt frame position information includes a center point (x, y), a length and width (w, h) and a first angle orientation;

[0013] Obtaining the tilted frame position information of the visual image detection part will output the coordinates of the upper left (x1, y1) and lower right (x2, y2) corners, as well as a second angle orientation, wherein the second angle orientation is calculated by calibrating the internal and external parameters of the camera.

[0014] Furthermore, the specific fusion step includes: matching the first angle orientation with the second angle orientation to determine the specific position of the door.

[0015] Furthermore, the specific fusion steps include: the fusion module consists of two parts: angle interval matching and linear increase threshold.

[0016] Furthermore, after the laser gate is detected, the position of the laser gate line is detected through some posture detection modules, and the angle values ​​θ1 and θ2 of the two end points on the gate line are calculated. The matching range is set to this interval. When the image detection angle θ img When it falls into it, the match is considered successful.

[0017] Further, after obtaining the angles θ1 and θ2 of the gate line endpoints, the angle interval can be increased according to a certain ratio t to calculate half the width of the gate line endpoint angle interval Then take the midpoint of the interval As the starting point, expand to both sides by t times. The calculation formula of the expansion angle value is as follows:

[0018]

[0019]

[0020] After obtaining the new expanded angle value, the angle matching threshold interval is reset based on the expanded angle value and then matched with the angle of the visual image detection result.

[0021] Furthermore, the expansion ratio t is dynamically updated according to the orientation of the laser gate.

[0022] Furthermore, when the simple laser gate is oriented to [0°, 90°], the expansion ratio t is calculated as follows:

[0023]

[0024] Where θ is the orientation angle of the laser gate, θ start is the starting point of the angle expansion.

[0025] Further, it comprises: a first obtaining unit, a second obtaining unit, a third obtaining unit, a fourth obtaining unit and a fusion unit;

[0026] The first obtaining unit is used to obtain the laser image to be identified;

[0027] The second obtaining unit is used to input the obtained laser image to be recognized into a preset recognition model to obtain a recognition and marking tilt frame;

[0028] The third obtaining unit is used to obtain the visual image to be recognized;

[0029] The fourth obtaining unit is used to input the obtained visual image into a preset recognition model to obtain recognition and identification results;

[0030] The fusion unit is used to fuse the results of the second obtaining unit and the fourth obtaining unit.

[0031] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the door detection method of laser and visual image fusion described above.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] (1) This method first converts the laser data into a grayscale image, uses the detection model to predict the laser grayscale image, and obtains the tilted frame position of the laser door on the image. Then, a similar detection model is used to make predictions based on the corresponding visual image data to obtain the door position information on the visual image. The fusion module is then used to fuse the visual image and laser image results. During the fusion process, the detection results of the visual image and the laser image will be matched one-to-one, and the unmatched detection results will be discarded. This solution can effectively remove false detections on laser images, greatly improve the accuracy of the detected laser door position, and facilitate use in other tasks;

[0034] (2) By increasing the threshold range linearly, it is possible to effectively avoid the situation where the image angle does not match the laser in special scenes, and by increasing the angle starting point, it is also possible to ensure that the accurate results under normal circumstances are not disturbed;

[0035] (3) This technical solution can solve the problem of false detection caused by relying solely on laser data by fusing the image results with the laser image detection results, greatly improving the accuracy of laser gates in actual scenes. The two methods of angle interval matching and linear increase of threshold in the fusion solution can not only ensure the accuracy of angle matching during the fusion process, but also improve the matching rate of detection results in the two data in special scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0037] Figure 1 The present invention provides a flow chart of a door detection method by integrating laser and visual images;

[0038] Figure 2 An embodiment of the present invention provides an effect diagram of a fusion result of a laser elevator door;

[0039] Figure 3 The embodiment provides an algorithm flow chart;

[0040] Figure 4 is a schematic diagram of an angle matching method provided by an embodiment;

[0041] Figure 5 The embodiment provides a schematic diagram of a matching angle threshold;

[0042] Figure 6 The embodiment provides a comparison of the effect of angle interval matching;

[0043] Figure 7 This embodiment provides a comparison of the effect of linearly increasing the threshold. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0045] It should be noted that, if there is no conflict, the various features in the embodiments of the present invention can be combined with each other, all within the scope of protection of the present invention. In addition, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a different order from the module division in the device or the flow chart. Furthermore, the words "first", "second", "third", etc. used in the present invention do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0046] In some embodiments, Figure 1 A flow chart of a door detection method for laser and visual image fusion is provided, comprising the following steps:

[0047] S1. Obtaining raw laser data and visual image data. There are many ways to obtain laser data. The following provides three situations for using 3D laser radar. The specific usage situations are as follows: If the robot scans while walking, it takes time to process the image. In order to reduce the time for the robot to process the image, it is necessary to manually collect and process the image before the robot moves. The user walks in the target area with the 3D laser radar in hand. The user drives the 3D laser radar to move to the target position. The 3D laser radar will detect and map the ground in real time. There are multiple users, and multiple users scan in different areas until all target areas are covered; or, the 3D laser radar is installed on a mobile robot, and the robot is allowed to move and scan autonomously, which can save the user's working time;

[0048] Or, if the robot walks while scanning, it takes time to process the image. In order to reduce the time for the robot to process the image, it is necessary to manually collect and process the image before the robot moves, which consumes manpower. If multiple laser scanning users are set in the target area, a schematic diagram of a target area setting device is provided. Schematically, the above-mentioned device sets three stations in the target area. The allocation scheme is: the three equipments are triangles, and the preferred triangle angle is 60°. The device establishes contact with the robot through a wireless communication device, and three-dimensional laser raw data;

[0049] S2, converting the laser data into a grayscale image;

[0050] S3, using the first detection model to detect the laser grayscale image and extract the door characteristic curve, wherein the characteristic curve is composed of a door horizontal reference line characteristic curve and a door curve characteristic curve, and the door characteristic curve and the door horizontal reference line characteristic curve are composed of point cloud data;

[0051] S4, using a second detection model to detect and extract door features from the visual image data, wherein the door features include a visual image of an area where the door is located;

[0052] S5, obtaining the tilted frame position of the characteristic curve in the laser data grayscale image and the visual image data respectively based on the characteristic curve;

[0053] S6, using a fusion module to fuse the visual image and the laser image results;

[0054] S7. The detection results of the visual image and the laser image will be matched one-to-one, and unmatched detection results will be discarded.

[0055] With the help of the detection results of the two sensor data fused in this solution, this technical solution can be applied to various behaviors of the robot in the elevator hall, such as repositioning in the elevator hall, taking the elevator, etc. When the robot loses its position in the elevator hall, it can autonomously infer its own global positioning based on the elevator position detected during the robot's movement and combined with the elevator's existing calibrated position in the global map, thereby reducing the number of manual interventions after the positioning is lost. While the robot is riding the elevator, accurate elevator door status information can greatly facilitate the smoothness of the elevator riding action. The elevator door position information helps the robot move quickly to the elevator entrance, and the elevator door angle information allows the robot to adjust its elevator entry posture at the elevator door and enter the elevator more safely.

[0056] It needs to be explained that fusion processing is to use image fusion technology to process the image data of a target collected by multiple source channels through image processing and computer technology, etc., to maximize the extraction of useful information in each channel, and finally synthesize it into a high-quality image to improve the utilization rate of image information, improve the accuracy and reliability of computer interpretation, and enhance the spatial resolution and spectral resolution of the original image, which is conducive to detection. The present invention uses the weighted fusion method in image fusion technology to add the pixel values ​​of the splicing area according to the preset full-time, so that the splicing area is more natural and convenient for observation. Specifically, the preset human leg recognition model is obtained by machine learning of the preset training image, and the preset training image is mainly an image with human leg features and an image without human leg features. The preset human leg model mainly uses the expression of the preset training image at different scales and aspect ratios to create an image pyramid, and the same training image of different scales and aspect ratios is expressed at different sizes and aspect ratios to create an image pyramid, and the human leg area in the human leg measurement response map at different sizes and aspect ratios is mapped to the preset training map of the original resolution, thereby identifying the human leg area of ​​the input preset training image. After training the preset human leg model with a large number of preset training images, a secondary classifier will be generated in the preset human leg model, which is used to determine whether there are human legs in the image to be identified. Among them, the image pyramid is a kind of multi-scale expression of the image, which is an effective but conceptually simple structure for explaining the image with multiple resolutions. Among them, the image pyramid is an effective but conceptually simple structure for explaining the image by arranging the resolution in a pyramid shape. The pyramid of an image is a series of resolutions arranged in a pyramid shape that gradually decrease and come from the same original image set. It is obtained by gradient downsampling until a certain termination condition is reached. We compare the layers of images to pyramids. The higher the level, the smaller the image and the lower the resolution.

[0057] In this embodiment, Figure 2A rendering of the fusion result of a laser elevator door is provided in the article. Taking the laser elevator door as an example, Figure 2 (a) and (b) show the fusion results of the visual image and the laser image. The elevator door in the visual image on the left and the result of the laser on the right are bound together by angle matching. Figure 2 (c) and (d) show the removal of false positives on the laser image by fusion with the visual image. The red false positive box on the right is discarded because the angle is not successfully matched, while the green one that is successfully matched is retained.

[0058] In this embodiment, specifically, Figure 3 An algorithm flow chart is provided. The algorithm consists of two parts: the detection stage and the fusion module. In the detection stage, the laser and visual image will predict the position of the door through two detection models respectively. The laser part will output the tilted frame position of the laser door, including the center point (x, y), length and width (w, h) and angle θ, while the visual image detection part will output the coordinates of the upper left (x1, y1) and lower right (x2, y2) corners, as well as the angle θ, where the θ value is calculated by calibrating the internal and external parameters of the camera. The fusion module binds the outputs of the two through angle matching, and finally outputs more accurate position information on the laser image. The fusion module consists of two parts: angle interval matching and linear increase threshold.

[0059] In this embodiment, Figure 4 Provides a schematic diagram of the angle matching method, which is usually used when matching lasers with visual images. Figure 4 (a) is allocated by the angle difference matching method. When the angle of laser detection is obtained, the threshold angle difference threshold (-10° to 10° in the figure) is set manually to obtain the matching interval. When the angle of the visual image detection result is θ img When it falls into the matching interval, it is considered a successful match. However, this method of manually setting the threshold will cause more missed matches, such as θ in (a) img It will be discarded because it does not meet the threshold requirement. If you blindly increase the angle difference threshold, it can indeed make up for the missed match, but it will increase the potential risk of false detection.

[0060] The angle interval matching method in the present invention is as follows: Figure 4 As shown in (b), after the laser gate is detected, the position of the laser gate line can be detected by some posture detection modules, and the angle values ​​θ1 and θ2 of the two end points on the gate line are calculated. The matching range is set to this interval. When the image detection angle θ img If it falls into the range, the match is considered successful. Figure 4 (b) shows that the same image angle θ imgHowever, it can be matched very well. Compared with the general difference matching, this angle interval matching method greatly improves the matching success rate in the process of image and laser gate detection fusion.

[0061] In this embodiment, generally speaking, due to the overhead view of the laser image, the calculation of position and angle will be more accurate than that of the visual image. Moreover, when calculating the direction angle and spatial position on the visual image, it depends largely on the accuracy of the camera calibration. Due to the calibration parameters and image distortion, the direction angle calculated based on the detection results often has errors. This will cause a certain deviation between the direction angle of the visual image detection result and the angle range of the laser door line in specific scenarios. For example, when the elevator door is vertical, the angle deviation will appear to be large. Generally speaking, the threshold range can be artificially increased based on the angle range, but directly expanding the range may lead to false detection. Taking the elevator door as an example, in Figure 5 In (a), when the elevator door’s orientation angle is close to 90°, the image angle θ img The gate line angle intervals θ1 and θ2 of the laser gate will not be satisfied.

[0062] The linear increase threshold value in the present invention is as follows Figure 5 As shown in (b), after obtaining the angles θ1 and θ2 of the gate line endpoints, the angle interval can be increased according to a certain ratio t. Specifically, first calculate half the width of the gate line endpoint angle interval Then take the midpoint of the interval As the starting point, expand to both sides by t times. The calculation formula of the expansion angle value is as follows:

[0063]

[0064]

[0065] After obtaining the new expanded angle value, the angle matching threshold interval can be reset based on this, and then matched with the angle of the visual image detection result. Since the above-mentioned situation where the visual image angle is very different from the laser angle will not always exist, and will only occur when the door is close to vertical, the expansion ratio t cannot be set to a fixed value, but needs to be dynamically updated according to the direction of the laser door. Taking the simplest laser door direction of [0°, 90°] as an example, the calculation formula of the expansion ratio t is as follows:

[0066]

[0067] Where θ is the orientation angle of the laser gate, θ start This means that the angle expansion does not always exist, but will increase linearly after the laser gate direction reaches a certain condition. In actual scenes, θstart Set to 30° or 45°, etc. The linearly increasing threshold range can effectively avoid the situation where the image angle does not match the laser in special scenes, and the increase in the angle starting point can also ensure that the accurate results in normal situations are not disturbed.

[0068] This technical solution can solve the problem of false detection caused by relying solely on laser data by fusing the image results with the laser image detection results, greatly improving the accuracy of laser gates in actual scenes. The two methods of angle interval matching and linear increase of threshold in the fusion solution can not only ensure the accuracy of angle matching during the fusion process, but also improve the matching rate of detection results in the two data in special scenes.

[0069] In this embodiment, Figure 6 The following figure shows the effect comparison of angle range matching. The image angle in (a) is -19.78°, while the center angle of the laser gate is -35.79°. If 10° is used as the threshold for difference matching, the matching will fail. If angle range matching is used, the calculated angle range is -81.81° to -13.74°, which can be successfully matched. The image angle in (b) is 30.67°, the center angle of the laser gate is 15.74°, and the angle range is 2.77° to 41.82°, which can also be successfully matched.

[0070] like Figure 7 As shown in Figure 1, (a) and (b) show typical examples of matching failures due to large differences in image angles. In Figure (a), the image angle is -38.38°, the original laser range is -56.10° to -39.21°, and the matching angle range after expansion by a ratio of 1.3 is -58.64° to -36.67°, which can just match the image detection results. In Figure (b), the angles between the image and the laser are 26.59° and 27.98° to 44.12°, respectively, and the expanded range is 25.56° to 46.54°. It can be seen that the matching rate can be effectively improved by using the method of linearly increasing the threshold.

[0071] In this embodiment, it includes: a first obtaining unit, a second obtaining unit, a third obtaining unit, a fourth obtaining unit and a fusion unit;

[0072] The first obtaining unit is used to obtain the laser image to be identified;

[0073] The second obtaining unit is used to input the obtained laser image to be recognized into a preset recognition model to obtain a recognition and marking tilt frame;

[0074] The third obtaining unit is used to obtain the visual image to be recognized;

[0075] The fourth obtaining unit is used to input the obtained visual image into a preset recognition model to obtain recognition and identification results;

[0076] The fusion unit is used to fuse the results of the second obtaining unit and the fourth obtaining unit.

[0077] 1 is a circuit structure diagram of a controller provided by an embodiment of the present invention. The controller includes one or more processors and a memory. In the figure, one processor is taken as an example.

[0078] The processor and the memory may be connected via a bus or other means, with connection via a bus being taken as an example.

[0079] As a non-volatile computer-readable storage medium, the memory can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the door detection method of the robot by laser and visual image fusion in the embodiment of the present invention. By running the non-volatile software programs, instructions and modules stored in the memory, various functional applications and data processing of the door detection method device of the robot by laser and visual image fusion are executed, that is, the door detection method of the robot by laser and visual image fusion provided by the above method embodiment and the functions of each module or unit of the above device embodiment are realized.

[0080] The memory may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may be remotely located and connected to the remote memory via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0081] The program instructions / modules are stored in, and when executed by the one or more processors, the door detection method of laser and visual image fusion in any of the above method embodiments is executed.

[0082] An embodiment of the present invention also provides a non-volatile computer storage medium, which stores computer executable instructions, and the computer executable instructions are executed by one or more processors. A processor can enable the above-mentioned one or more processors to execute the door detection method of laser and visual image fusion in any of the above-mentioned method embodiments.

[0083] An embodiment of the present invention also provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by an electronic device, the electronic device executes any one of the door detection methods for fusion of laser and visual images.

[0084] The above described device or equipment embodiments are merely illustrative, wherein the unit modules described as separate components may or may not be physically separated, and the components displayed as module units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network module units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0085] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may also be combined, the steps may be implemented in any order, and there are many other changes in different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they may still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A door detection method integrating laser and visual image, characterized in that: The steps include: Acquire raw laser data and visual image data; Converting the laser data into a grayscale image; Using the first detection model to detect the laser grayscale image and extract the door characteristic curve, wherein the characteristic curve is composed of a door horizontal reference line characteristic curve and a door curve characteristic curve, and the door characteristic curve and the door horizontal reference line characteristic curve are composed of point cloud data; Using a second detection model to detect and extract door features from the visual image data, wherein the door features include a visual image of an area where the door is located; Based on the characteristic curve, respectively obtain the tilted frame position of the characteristic curve in the laser data grayscale image and the visual image data; Use the fusion module to fuse the visual image and laser image results; The detection results of the visual image and the laser image will be matched one-to-one, and unmatched detection results will be discarded; After detecting the laser gate, the position of the laser gate line is detected through some posture detection modules, and the angle values ​​θ1 and θ2 of the two end points on the gate line are calculated. The matching range is set to this interval. When the image detection angle θ img When it falls into it, it is considered a successful match; After obtaining the angles θ1 and θ2 of the gate line endpoints, the angle interval can be increased according to a certain ratio t to calculate half the width of the gate line endpoint angle interval Then take the midpoint of the interval As the starting point, expand to both sides by t times. The calculation formula of the expansion angle value is as follows: After obtaining the new expanded angle value, the angle matching threshold interval is reset based on the expanded angle value and then matched with the angle of the visual image detection result.

2. The door detection method of laser and visual image fusion according to claim 1 is characterized in that: The specific steps of extracting the door characteristic curve include: Obtaining the tilt frame position information of the laser grayscale image, wherein the tilt frame position information includes a center point (x, y), a length and width (w, h) and a first angle orientation; Obtaining the tilted frame position information of the visual image detection part will output the coordinates of the upper left (x1, y1) and lower right (x2, y2) corners, as well as a second angle orientation, wherein the second angle orientation value is calculated by calibrating the internal and external parameters of the camera.

3. The door detection method of laser and visual image fusion according to claim 2 is characterized in that: The specific fusion step includes: matching the first angle orientation with the second angle orientation to determine the specific position of the door.

4. The door detection method of laser and visual image fusion according to claim 1 is characterized in that: The specific fusion steps include: the fusion module consists of two parts: angle interval matching and linear increase threshold.

5. The door detection method of laser and visual image fusion according to claim 1, characterized in that: The expansion ratio t is dynamically updated according to the orientation of the laser gate.

6. The door detection method of laser and visual image fusion according to claim 1 is characterized in that: When the simple laser gate is oriented to [0°, 90°], the expansion ratio t is calculated as follows: Where θ is the orientation angle of the laser gate, θ start is the starting point of the angle expansion.

7. A door detection system integrating laser and visual images, characterized in that: include: The first acquisition unit, the second acquisition unit, the third acquisition unit, the fourth acquisition unit and the fusion unit implement the door detection method of laser and visual image fusion as described in any one of claims 1 to 6; The first obtaining unit is used to obtain the laser image to be identified; The second obtaining unit is used to input the obtained laser image to be recognized into a preset recognition model to obtain a recognition and marking tilt frame; The third obtaining unit is used to obtain the visual image to be recognized; The fourth obtaining unit is used to input the obtained visual image into a preset recognition model to obtain recognition and identification results; The fusion unit is used to fuse the results of the second obtaining unit and the fourth obtaining unit.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the door detection method of laser and visual image fusion as described in any one of claims 1 to 6 is implemented.

Images