Automatic walking device and door-passing method based on laser radar and depth camera
Through the combination of lidar and depth camera, the door is identified and the equipment offset distance is calculated, and the problem of intelligent automatic walking equipment passing through doors indoors is solved, and the accurate door passing capability with low cost and low computing needs is achieved.
Patent Information
- Application Number
- CN202110340529.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-03-30
- Publication Date
- 2024-12-13
- Estimated Expiration
- 2041-03-30
AI Technical Summary
The problem of intelligent automatic walking equipment realizing autonomous navigation and safe passage of doors in indoor environments. The existing technologies such as magnetic navigation and machine learning-based methods have limitations, which cannot adapt to the recognition and aesthetic design of various doors, and are costly. .
Using a combination of lidar and depth camera, the door is identified through the depth camera and the coordinates of the center point of the door are obtained. Combined with the lidar, the door width is measured, the offset distance of the automatic walking device is calculated, and the equipment position is adjusted to pass through the door accurately.
It realizes automatic correction of equipment position in low-cost scenarios, ensures safe and accurate passage from the middle of the door, adapts to various door designs, and reduces calculation and sample requirements.
Smart Images
Figure CN114035563B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of intelligent walking devices, and in particular to an automatic walking device and a door passing method of the automatic walking device based on a laser radar and a depth camera. Background Art
[0002] At present, unmanned driving technology is developing rapidly, but its main application scenario is outdoors. As for indoor intelligent automatic walking equipment (such as automatic walking equipment), the most widely used is the magnetic navigation AGV car. This technology is mature and reliable, but it also has obvious disadvantages. Indoor navigation is performed by sticking magnetic strips on the ground. This navigation method will be mechanically damaged by hard objects such as metal passing through the loop, which has a certain impact on navigation and is only suitable for use in some factory environments.
[0003] However, unmanned vehicles are also needed in other occasions, such as hospitals, restaurants, offices, etc., but magnetic navigation AGVs are not competent for this purpose. Outdoor unmanned vehicles are expensive and their algorithms and logic are not competent for indoor scenes. For example, a very inconspicuous but difficult to overcome problem: for intelligent automatic walking equipment to achieve autonomous navigation in indoor environments, an important issue that is inevitably important is how to safely pass through various doors indoors.
[0004] At present, many people have proposed different solutions to the problem of intelligent automatic walking equipment passing through doors, and these methods have certain limitations. For example, the traditional method is to use sensors (such as ultrasound, infrared, etc.) to detect the distance between the intelligent automatic walking equipment and the two sides of the door frame during the movement of the robot, so that the robot can avoid the door frame and pass through the door smoothly. The limitation of this method is that it is sensitive to the initial position and environmental map of the intelligent automatic walking equipment. There are also machine learning-based methods proposed to identify doors and complete the door passing according to the relative position of the door and the intelligent automatic walking equipment. This method requires a large number of door samples and a lot of computing time. If the sample size is not rich enough, it cannot adapt to the recognition of various doors, and it is difficult to deal with some non-mainstream door frame designs that pay attention to design aesthetics. Summary of the invention
[0005] The embodiments of the present invention provide an automatic walking device and a method for the automatic walking device to pass through a door based on a laser radar and a depth camera, which can automatically correct its own position in a scenario with low hardware cost, thereby achieving safe and accurate passage through the center of the door.
[0006] To achieve the above object, the embodiments of the present invention adopt the following technical solutions:
[0007] In a first aspect, an automatic walking device provided by an embodiment of the present invention includes:
[0008] USB cable (1), main control board (2), battery (3), depth camera (4), lidar (5), hub motor (6), encoder (7), steering mechanism (8), wheel (9) and chassis (10);
[0009] Connect the laser radar (5) and the depth camera (4) to the main control board (2) via USB cables (1);
[0010] The battery (3) is connected to the main control board (2), the depth camera (4), the laser radar (5), the wheel hub motor (6), the encoder (7) and the steering mechanism (8) through wires;
[0011] A bracket is arranged on the upper surface of the chassis (10), a depth camera (4) is installed on the top of the bracket, the bracket is located on the central axis of the chassis (10), and the laser radar (5) and the depth camera (4) are arranged along the central axis of the chassis (10);
[0012] The encoder (7) is connected to the wheel hub motor (6), and the wheel hub motor (6) is connected to the main control board (2).
[0013] In a second aspect, an embodiment of the present invention provides a door-passing method based on a laser radar and a depth camera, comprising:
[0014] Step 1: After the autonomous walking device moves to a position close to the door, it uses a depth camera to collect images;
[0015] Step 2: Identify the door and obtain the door width pixel distance and door center point coordinates based on the collected image;
[0016] Step 3: According to the door center coordinates, obtain the horizontal pixel distance between the door center and the image center;
[0017] Step 4: Obtain the actual distance of the door width through the laser radar;
[0018] Step 5. According to the door width pixel distance and the door width actual distance, calculate the actual distance represented by each pixel distance in the image, that is, the pixel equivalent. When the automatic walking device is completely located at the center point of the door, the door center point identified in the image at this moment is also located at the center point of the entire image. When the position of the automatic walking device is offset, the door center point position identified in the image will also be relatively offset and will not be at the center point of the entire image. Therefore, according to the pixel distance calculated by the offset in step 3 and multiplied by the pixel equivalent, the actual distance of the automatic walking device offset from the door center can be obtained;
[0019] Step six: issue a control command to the automatic walking device according to the actual distance to adjust the position of the automatic walking device until the automatic walking device is adjusted to a position facing the center of the door, and then issue a straight-ahead command until it passes the door.
[0020] The automatic walking device and the method for automatic walking device to pass through a door based on a laser radar and a depth camera provided by the embodiments of the present invention use a combination of a laser radar and a depth camera. The depth camera is installed in the center of the base of the automatic walking device. The position of the door in the image is used to determine the horizontal deflection position of the automatic walking device at this moment. The width of the door is calculated in combination with the laser radar information. Finally, the distance of the automatic walking device offset from the center of the door in the horizontal direction is calculated. Based on this distance, an instruction is given to the automatic walking device to adjust the position of the automatic walking device until the automatic walking device is completely aligned with the center of the door, and then it goes straight to achieve accurate door passing. In this process, the automatic walking device only needs to use a low-cost main control to calculate the relative position relationship between the door and the automatic walking device based on the detection results of the depth camera and the laser radar, so as to control the automatic walking device to achieve accurate door passing. The automatic walking device of the present invention can automatically correct its own position, and achieve safe and accurate passage through the center of the door. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0022] Figure 1 A schematic diagram of a process flow of a door-passing method based on a laser radar and a depth camera provided in an embodiment of the present invention;
[0023] Figure 2 A flow chart of door recognition based on a laser radar and a depth camera according to an embodiment of the present invention;
[0024] Figure 3a A schematic diagram of the structure of an automatic walking device provided by an embodiment of the present invention;
[0025] Figure 3b A partial enlarged view of the radar and camera installation structure in the structure of the automatic walking device provided by an embodiment of the present invention;
[0026] Figure 4 A rear view of the structure of the automatic walking device provided according to the embodiment;
[0027] Figure 5 The figure is a top view of the structure of the automatic walking device provided according to the embodiment. DETAILED DESCRIPTION
[0028] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present invention will be described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention. It can be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "one", "said" and "the" used herein may also include plural forms. It should be further understood that the term "including" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used here may include wireless connection or coupling. The term "and / or" used herein includes any unit and all combinations of one or more associated listed items. It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as generally understood by those of ordinary skill in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.
[0029] The design purpose of this embodiment is to solve the door-passing problem of automatic walking equipment during indoor navigation, and proposes a method of combining laser radar and depth camera. The laser radar measures the distance, and the depth camera identifies the door. According to the recognition result, instructions are given to the automatic walking device to finally complete the door-passing. Specifically, for indoor automatic walking equipment, especially larger-sized automatic walking equipment, such as electric wheelchairs, walkers, etc., or sweeping robots, AGV carts, etc., if you want to pass through smaller doors, you basically need to align the center of the car body with the center of the door to pass through the door. Therefore, the present invention divides the door-passing problem into two parts. First, adjust the position of the automatic walking device until it is completely facing the center of the door, and then go straight through.
[0030] The embodiment of the present invention provides an automatic walking device, such as Figure 3a , Figure 3b , Figure 4 , Figure 5 As shown, including:
[0031] USB cable (1), main control board (2), battery (3), depth camera (4), lidar (5), hub motor (6), encoder (7), steering mechanism (8), wheel (9) and chassis (10).
[0032] Connect the laser radar (5) and the depth camera (4) to the main control board (2) via the USB cable (1).
[0033] The battery (3) is connected to the main control board (2), the depth camera (4), the laser radar (5), the wheel hub motor (6), the encoder (7) and the steering mechanism (8) through wires. The encoder (7) adopts the OMRON E6B2-CWZ6C encoder; the main control board (2) adopts the RK3399 development board.
[0034] A bracket is arranged on the upper surface of the chassis (10), a depth camera (4) is installed on the top of the bracket, the bracket is located on the central axis of the chassis (10), and the laser radar (5) and the depth camera (4) are arranged along the central axis of the chassis (10).
[0035] The encoder (7) is connected to the wheel hub motor (6), and the wheel hub motor (6) is connected to the main control board (2).
[0036] The upper surface of the chassis (10) is provided with a main control board (2) and a battery (3) in sequence along the central axis of the chassis (10) from the head to the tail of the chassis (10). Two pairs of running mechanisms are respectively provided on the two sides of the chassis (10), and each pair of running mechanisms includes: two groups of wheels (9) symmetrically distributed along the central axis of the chassis (10), the wheels (9) are connected to a steering mechanism (8), and the steering mechanism (8) is connected to a wheel hub motor (6). The laser radar (5) is installed in the center of the chassis of the automatic walking device and is located in front of the depth camera (4).
[0037] Specifically,
[0038] The lens of the depth camera (4) is directed toward the front of the automatic walking device along the central axis of the chassis (10); the field of view of the depth camera (4) covers the front of the automatic walking device; the depth range of the depth camera is 0.6 m to 8 m, and the FOV angle is: H66.1°V40.2°;
[0039] The laser emitting antenna of the laser radar (5) is directed toward the front of the automatic walking device along the central axis of the chassis (10); the detection distance range of the laser radar (5) is 0.15m-12m, and the scanning angle is 0°-360°.
[0040] The encoder (7) is used to record the position information of the automatic walking device. The depth camera (4) is used to collect depth images. The laser radar (5) is used to collect distance and angle information.
[0041] The main control board (2) is used to process the depth image, laser radar information and encoder (7) information, and is used to transmit speed control instructions to the wheel hub motor (6), and to transmit direction control instructions to the steering mechanism (8).
[0042] Based on the above-mentioned automatic walking device, the embodiment of the present invention also provides a door-passing method based on laser radar and depth camera, such as Figure 1 As shown, including:
[0043] Step 1: After the automatic walking device moves to a position close to the door, the depth camera is used to collect images. The position close to the door can be a position at a specified distance from the door. For example: the automatic walking device is controlled to move near the door, the depth camera is turned on, and the depth image is collected; wherein, the automatic walking device is at a certain distance from the door to ensure that the door can appear in the camera's field of view; the depth camera is installed in the center of the automatic walking device's forward direction.
[0044] Step 2: According to the collected image, identify the door and obtain the door width pixel distance and the door center point coordinates.
[0045] Step 3: According to the door center coordinates, obtain the horizontal pixel distance between the door center and the image center.
[0046] Step 4: Obtain the actual distance of the door width through LiDAR.
[0047] Step 5: Obtain the actual distance of the automatic walking device offset from the door center according to the door width pixel distance, the door width actual distance and the horizontal pixel distance of the door center point offset from the image center point.
[0048] Step six: issue a control command to the automatic walking device according to the actual distance to adjust the position of the automatic walking device until the automatic walking device is adjusted to a position facing the center of the door, and then issue a straight-ahead command until it passes the door.
[0049] In this embodiment, if Figure 2 As shown, the step 2 specifically includes:
[0050] S21. The main control reads the pixel value of the image captured by the depth camera and performs binarization on the obtained pixel value, wherein the size of the pixel value corresponds to the current distance between the depth camera and the obstacle in front, and when the distance corresponding to the pixel value is within a preset range, the pixel value is set to 0, and when the distance corresponding to the pixel value is not within the preset range, the pixel value is set to 255.
[0051] S22, removing the ground interference area from the image after the binarization processing.
[0052] S23. Based on the result outputted by S21, a morphological closing operation is performed on the binarized image using a rectangular structure element.
[0053] S24. Perform ROI area extraction on the result output by S22, extract the door area therein, and further perform contour extraction on the door area to obtain the door outline, and determine the minimum circumscribed rectangle in the door outline, wherein the width of the minimum circumscribed rectangle is used as the pixel length of the door width, and the center point of the minimum circumscribed rectangle is used as the center point of the door.
[0054] For example: Based on a geometric model, the ground interference area is removed. The height H of the camera optical center from the ground is known. (X p ,Y p ,Z p ) is the three-dimensional coordinate of a pixel point in the image corresponding to the scene point P, then the distance from point P to the ground is: h p =HY p , set the threshold Δh, when h p When <Δh, the pixel is considered to belong to the ground and is directly removed (set to 0). Use the rectangular structure element to perform morphological closing operation on the binary image to eliminate small black holes in the binary image. Use the template image to extract the ROI region of the above binary image. Find the outline of the extracted ROI region and find its minimum circumscribed rectangle. The width of the circumscribed rectangle (D p ) as the pixel length of the door width, and the center point of the circumscribed rectangle (x d ,y d ) as the center point of the door.
[0055] Optionally, the following settings may be performed in step three:
[0056] The image size is M x N, the image center is (M / 2, N / 2), and the horizontal pixel distance of the offset is L p =|M / 2–x m |, where M represents the number of pixels in the horizontal direction of the image, N represents the number of pixels in the vertical direction of the image, and L p Indicates the pixel distance of the offset, L represents the distance, p represents the pixel, x m Indicates the horizontal pixel coordinates of the center point of the door, X represents the pixel coordinates, and m represents the center point.
[0057] In this embodiment, in step 4, it is necessary to turn on the laser radar and use the laser radar information to calculate the actual distance of the door width, which specifically includes:
[0058] The laser radar is a two-dimensional laser radar.
[0059] The information obtained by the laser radar scanning includes: a one-dimensional distance array [d0, d1, d2, ..., d 359 ], where the element d in the array i is the distance between the laser radar and the obstacle, and i represents the angle of the radar scanning.
[0060] According to d i The change in the value determines the distance between the radar and the two sides of the door frame.
[0061] Among them, the i The change of the value determines the distance between the radar and the two sides of the door frame, including:
[0062] Query the positive distance array to get the first distance mutation point d l , where d l Corresponding to one side of the door frame, the scanning angle at this moment is l; query the reverse distance array to get the first distance mutation point d r , where d r Corresponding to the other side of the door frame, the scanning angle at this moment is r; determine that the distance between the laser radar and the two sides of the door frame is d l and d r , the angles are l and r respectively, so the angle θ with the two sides of the door frame is θ = |l–r|, and the door width D is obtained, where D = d l 2 +d r 2 -2*d l *d r *cosθ.
[0063] Furthermore, in step five: according to the door width pixel distance and the door width actual distance, the actual distance represented by each pixel distance in the image is calculated as the pixel equivalent; according to the calculated horizontal pixel distance of the door center point offset from the image center point multiplied by the pixel equivalent, the actual distance of the door center offset by the automatic walking device is obtained, wherein, when the automatic walking device is completely located at the door center point, the door center point identified in the image at this moment is also located at the center point of the entire image, and when the position of the automatic walking device is offset, the position of the door center point identified in the resulting image is also relatively offset and is not at the center point of the entire image. For example: Get pixel equivalent P e =D / D p , where D represents the actual distance of the door width, D p Indicates the door width pixel distance. Use the offset horizontal pixel distance L p , get the actual horizontal distance D of the current automatic walking device offset from the door center d =P e *L pFor example: According to the actual distance D of the automatic walking device offset from the door center calculated in step 5 d , the offset direction is (M / 2-x d ) sign, if the sign is positive, the automatic walking device is biased to the left, otherwise it is biased to the right. Based on this, a command is issued to adjust the position of the automatic walking device to the center of the door, and a straight-ahead command is issued to pass the door.
[0064] The step six includes: determining the offset direction according to Dd, wherein the offset direction is determined by the sign of (M / 2-xd), if the sign is positive, the current automatic walking device is biased to the left, and if the sign is negative, the current automatic walking device is biased to the right.
[0065] The solution of this embodiment uses a combination of laser radar and depth camera. The depth camera is installed in the center of the base of the automatic walking device. The position of the door in the image is used to determine the horizontal deflection position of the automatic walking device at this moment. The door width is calculated in combination with the laser radar information, and finally the distance of the automatic walking device offset from the center of the door in the horizontal direction is calculated. According to this distance, the automatic walking device is given instructions to adjust the position of the automatic walking device until the automatic walking device is completely aligned with the center of the door, and then goes straight to achieve accurate door passing. In this process, the automatic walking device only needs to use a low-cost main control to calculate the relative position relationship between the door and the automatic walking device based on the detection results of the depth camera and the laser radar, so as to control the automatic walking device to achieve accurate door passing. The automatic walking device of the present invention can automatically correct its own position, so that it can pass safely and accurately through the center of the door.
[0066] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The above is only a specific implementation method of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or replacements that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. An automatic walking device, characterized in that: include: USB cable (1), main control board (2), battery (3), depth camera (4), lidar (5), wheel hub motor (6), encoder (7), steering mechanism (8), wheel (9) and chassis (10); Connect the laser radar (5) and the depth camera (4) to the main control board (2) via USB cables (1); The battery (3) is connected to the main control board (2), the depth camera (4), the laser radar (5), the wheel hub motor (6), the encoder (7) and the steering mechanism (8) through wires; A bracket is arranged on the upper surface of the chassis (10), a depth camera (4) is installed on the top of the bracket, the bracket is located on the central axis of the chassis (10), and the laser radar (5) and the depth camera (4) are arranged along the central axis of the chassis (10); The encoder (7) is connected to the wheel hub motor (6), and the wheel hub motor (6) is connected to the main control board (2); The automatic walking device implements the following steps, including: Step 1: After the autonomous walking device moves to a position close to the door, it uses a depth camera to collect images; Step 2: Identify the door and obtain the door width pixel distance and door center point coordinates based on the collected image; Step 3: According to the door center coordinates, obtain the horizontal pixel distance between the door center and the image center; Step 4: Obtain the actual distance of the door width through the laser radar; Step 5: According to the door width pixel distance, the door width actual distance and the horizontal pixel distance of the door center point offset from the image center point, the actual distance of the door center point offset from the door center point is obtained; Step 6: issuing a control command to the automatic walking device according to the actual distance to adjust the position of the automatic walking device until the automatic walking device is adjusted to a position facing the center of the door, and then issuing a straight-ahead command until the automatic walking device passes the door; In step 3: the image size is M x N, the image center is (M / 2, N / 2), and the horizontal pixel distance of the offset is L p =|M / 2 –x m |, where M represents the number of pixels in the horizontal direction of the image, N represents the number of pixels in the vertical direction of the image, and L p Indicates the pixel distance of the offset, L represents the distance, p represents the pixel, x m Indicates the horizontal pixel coordinates of the center point of the door, X represents the pixel coordinates, and m represents the center point; In the step 5: according to the door width pixel distance and the door width actual distance, the actual distance represented by each pixel distance in the image is calculated as the pixel equivalent; according to the calculated horizontal pixel distance of the door center point offset from the image center point multiplied by the pixel equivalent, the actual distance of the door center offset by the automatic walking device is obtained, wherein, when the automatic walking device is completely located at the door center point, the door center point identified in the image at this moment is also located at the center point of the entire image, and when the position of the automatic walking device is offset, the position of the door center point identified in the image is also relatively offset and is not at the center point of the entire image; The step six comprises: according to D d Determine the offset direction, where the offset direction is given by (M / 2 – x m ) is determined by the sign of , if the sign is positive, the current automatic walking device is biased to the left; if the sign is negative, the current automatic walking device is biased to the right.
2. The automatic walking device according to claim 1, characterized in that: Encoder (7) adopts OMRON E6B2-CWZ6C encoder; The main control board (2) uses the RK3399 development board.
3. The automatic walking device according to claim 1, characterized in that: A main control board (2) and a battery (3) are sequentially mounted on the upper surface of the chassis (10) along the central axis of the chassis (10) from the head to the tail of the chassis (10); Two pairs of running mechanisms are respectively installed on two side surfaces of the chassis (10), each pair of running mechanisms comprising: two groups of wheels (9) axially symmetrically distributed along the central axis of the chassis (10), the wheels (9) being connected to the steering mechanism (8), and the steering mechanism (8) being connected to the wheel hub motor (6); The lens of the depth camera (4) is directed toward the front of the automatic walking device along the central axis of the chassis (10); the field of view of the depth camera (4) covers the front of the automatic walking device; the depth range of the depth camera is 0.6 m to 8 m, and the FOV angle is: H66.1° V40.2°; The laser transmitting antenna of the laser radar (5) is directed toward the front of the automatic walking device along the central axis of the chassis (10); the detection distance range of the laser radar (5) is 0.15m-12m, and the scanning angle is 0°-360°; The laser radar (5) is installed in the center of the chassis of the automatic walking device and is located in front of the depth camera (4).
4. A door-passing method based on laser radar and depth camera, characterized in that: include: Step 1: After the autonomous walking device moves to a position close to the door, it uses a depth camera to collect images; Step 2: Identify the door and obtain the door width pixel distance and door center point coordinates based on the collected image; Step 3: According to the door center coordinates, obtain the horizontal pixel distance between the door center and the image center; Step 4: Obtain the actual distance of the door width through the laser radar; Step 5: According to the door width pixel distance, the door width actual distance and the horizontal pixel distance of the door center point offset from the image center point, the actual distance of the door center point offset from the door center point is obtained; Step 6: issuing a control command to the automatic walking device according to the actual distance to adjust the position of the automatic walking device until the automatic walking device is adjusted to a position facing the center of the door, and then issuing a straight-ahead command until the automatic walking device passes the door; In step three: The image size is M x N, the image center is (M / 2, N / 2), and the horizontal pixel distance of the offset is L p =|M / 2 –x m |, where M represents the number of pixels in the horizontal direction of the image, N represents the number of pixels in the vertical direction of the image, and L p Indicates the pixel distance of the offset, L represents the distance, p represents the pixel, x m Indicates the horizontal pixel coordinates of the center point of the door, X represents the pixel coordinates, and m represents the center point; In step five: According to the door width pixel distance and the door width actual distance, the actual distance represented by each pixel distance in the image is calculated as a pixel equivalent; According to the calculated horizontal pixel distance of the door center point offset from the image center point multiplied by the pixel equivalent, the actual distance of the automatic walking device offset from the door center is obtained, wherein, when the automatic walking device is completely located at the door center point, the door center point identified in the image at this moment is also located at the center point of the entire image, and when the position of the automatic walking device is offset, the position of the door center point identified in the image is also relatively offset and is not at the center point of the entire image; The step six comprises: According to D d Determine the offset direction, where the offset direction is given by (M / 2 – x m ) is determined by the sign of , if the sign is positive, the current automatic walking device is biased to the left; if the sign is negative, the current automatic walking device is biased to the right.
5. The door-passing method based on laser radar and depth camera according to claim 4 is characterized in that: The second step comprises: S21, the main control reads the pixel value of the image acquired by the depth camera, and performs binarization processing on the obtained pixel value, wherein the size of the pixel value corresponds to the current distance between the depth camera and the obstacle in front, and the pixel value is set to 0 when the distance corresponding to the pixel value is within the preset range, and the pixel value is set to 255 when the distance corresponding to the pixel value is not within the preset range; S22, removing the ground interference area from the image after the binary processing; S23, for the result outputted by S21, using a rectangular structure element, performing a morphological closing operation on the image after the binary processing; S24. Perform ROI area extraction on the result output by S22, extract the door area therein, and further perform contour extraction on the door area to obtain the door outline, and determine the minimum circumscribed rectangle in the door outline, wherein the width of the minimum circumscribed rectangle is used as the pixel length of the door width, and the center point of the minimum circumscribed rectangle is used as the center point of the door.
6. The door-passing method based on laser radar and depth camera according to claim 4, characterized in that: In step 4: The laser radar is a two-dimensional laser radar; The information obtained by the laser radar scanning includes: a one-dimensional distance array [d0, d1, d2, ..., d 359 ], where the element d in the array i is the distance between the laser radar and the obstacle, i represents the angle of the radar scan; According to d i The change in the value determines the distance between the radar and the two sides of the door frame.
7. The door-passing method based on laser radar and depth camera according to claim 6, characterized in that: According to d i The change of the value determines the distance between the radar and the two sides of the door frame, including: Query the positive distance array to get the first distance mutation point d l , where d l Corresponding to one side of the door frame, the scanning angle at this moment is l; Query the reverse distance array to get the first distance mutation point d r , where d r Corresponding to the other side of the door frame, the scanning angle at this moment is r; Determine that the distances between the laser radar and the two sides of the door frame are d l and d r , the angles are l and r respectively, so the angle θ=|l–r| with the two sides of the door frame, and the door width D is obtained, where D=d l 2 +d r 2 -2*d l *d r *cosθ.