Method, medium and device for detecting opening degree of robot-based sliding door
By training a key point detection model for swing doors, key points of swing doors are obtained and opening and closing angles are calculated, solving the problem that existing technologies cannot quantify the degree of opening and closing of swing doors, and achieving high-precision detection and real-time monitoring.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGZHOU GOSUNCN ROBOTICS CO LTD
- Filing Date
- 2023-09-26
- Publication Date
- 2026-06-05
AI Technical Summary
Existing technologies cannot quantify the opening and closing degree of swing doors and are costly and lack precision.
By training a key point detection model for swing doors, key points in sample images of swing doors are obtained, and the opening and closing angles are calculated to quantitatively represent the degree of opening and closing of swing doors.
It reduces implementation and maintenance costs, improves detection accuracy, and enables quantitative representation and real-time monitoring of the opening and closing degree of swing doors.
Smart Images
Figure CN117197440B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of detection technology, and in particular to a method, medium, and equipment for detecting the opening degree of a swing door based on a robot. Background Technology
[0002] With technological advancements, robots have matured and are widely used in various scenarios. When a robot needs to pass through a swing door, it's typically necessary to pre-determine whether the robot can pass based on the degree of opening and closing of the door. If the swing door is closed, the robot cannot pass and measures are required; if the door is only slightly open, the robot still cannot pass and other measures are needed; if the door is open enough for the robot to pass, no further measures are needed, and the robot can pass smoothly. Therefore, determining whether a swing door is closed is of great importance in many application scenarios. In existing technologies, the following are some commonly used methods for determining whether a swing door is closed:
[0003] (1) Paper Strip Method: This method is suitable for hinged doors with two panels. A paper strip is pasted in the middle of each door in the closing direction, so that the paper strips on the two doors form a longer strip. When the door is open, the long paper strip splits into two; when the door is closed, the two paper strips form one long strip. The state of the door can be determined by detecting the state of the paper strip. This method is simple and intuitive, but it has high requirements for the placement and state of the paper strip.
[0004] (2) Classification Method: This method uses image classification algorithms based on machine learning and computer vision to determine whether a door is open or closed. A classification model is pre-trained, and given an image of a swing door, the model outputs the corresponding open or closed label. The classification model includes, but is not limited to, a convolutional neural network (CNN). This method requires sufficient labeled data, and the robustness and generalization ability of the model need to be fully considered.
[0005] The above two methods have the following technical drawbacks:
[0006] The first method requires a sticker on the door, as well as additional sensors and wiring. These external auxiliary devices increase implementation and maintenance costs and also affect the normal operation and visual appearance of the door. The second method requires building and training a model with a certain degree of generalization ability; for doors with special shapes, more training data and adjustments may be needed. In summary, both of these methods can only determine whether a swing door is open or closed, neither can quantify the degree of opening or closing, and their detection accuracy is low. Summary of the Invention
[0007] In view of this, the present invention provides a method, medium and device for detecting the opening degree of a swing door based on a robot, so as to solve the problems of existing technology being unable to quantitatively express the opening degree of a swing door and having high cost and poor accuracy.
[0008] On the one hand, this invention provides a robot-based method for detecting the opening degree of a swing door.
[0009] Acquire sample images of swing doors, and train a key point detection model for swing doors based on the sample images of swing doors;
[0010] The image to be detected is acquired, and the key points of the swing door in the image to be detected are obtained by the swing door key point detection model. The image to be detected is acquired by the robot from the target shooting perspective.
[0011] The opening degree of the swing door is calculated based on the key points of the swing door and the target shooting angle. The opening degree represents the observed value of the opening angle of the swing door panel around the rotation axis under the target shooting angle.
[0012] Optionally, calculating the opening degree of the swing door based on the key points of the swing door and the target shooting angle includes:
[0013] Based on the key points of the swing door, obtain the upper opening angle and the lower opening angle of the swing door respectively;
[0014] Obtain the height of the swing door panel and the line-of-sight height above the ground corresponding to the target shooting angle of the robot;
[0015] The opening degree of the swing door is calculated based on the door panel height, the line of sight from the ground, and the upper and lower opening angles.
[0016] Optionally, calculating the opening degree of the swing door based on the door panel height, the line-of-sight height from the ground, and the upper and lower opening angles includes:
[0017] When the line of sight is 0 degrees above the ground, the opening degree of the swing door is the upward opening angle;
[0018] When the line of sight is at any value less than the height of the door panel, calculate the ratio of the door panel height to the line of sight at the ground. Based on the ratio, perform a weighted summation of the upper opening angle and the lower opening angle to obtain the degree of opening of the swing door.
[0019] When the line of sight is at the height of the door panel, the opening degree of the swing door is the downward opening angle.
[0020] Optionally, obtaining the upper and lower opening angles of the swing door based on the key points of the swing door includes:
[0021] Obtain the key points of the swing door, which include the upper left corner of the door frame, the upper right corner of the door frame, the upper right corner of the door panel, the lower left corner of the door frame, the lower right corner of the door frame, and the lower right corner of the door panel;
[0022] The upward opening angle of the swing door is calculated based on the upper left corner of the door frame, the upper right corner of the door frame, and the upper right corner of the door panel.
[0023] The downward opening angle of the swing door is calculated based on the lower left corner of the door frame, the lower right corner of the door frame, and the lower right corner of the door panel.
[0024] Optionally, the formula for calculating the upward opening angle of the swing door is:
[0025]
[0026] Where θ1 represents the opening angle, (x1,y1) represents the coordinates of the upper left corner of the door frame, (x2,y2) represents the coordinates of the upper right corner of the door frame, and (x3,y3) represents the coordinates of the upper right corner of the door panel.
[0027] The formula for calculating the downward opening angle of the swing door is:
[0028]
[0029] Where θ2 represents the downward opening angle, (x4,y4) represents the coordinates of the lower left corner of the door frame, (x5,y5) represents the coordinates of the lower right corner of the door panel, and (x6,y6) represents the coordinates of the lower right corner of the door frame.
[0030] Optionally, acquiring sample images of swing doors and training a key point detection model for swing doors based on the sample images includes:
[0031] Acquire an initial image of a swing door, the initial image of which includes a swing door in various open and closed states;
[0032] The initial image of the swing door is annotated with key points to obtain a sample image of the swing door;
[0033] The key point detection model is trained using the sample images of the swing door to obtain the key point detection model for the swing door.
[0034] Optionally, the keypoint detection model includes the YOLOv5-pose model.
[0035] Optionally, the target shooting angle is such that the robot's line of sight is perpendicular to the plane where the door frame is located and directly opposite the center line of the door frame.
[0036] According to a second aspect of the present invention, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the robot-based method for detecting the degree of opening and closing of a swing door as described above.
[0037] According to a third aspect of the present invention, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the robot-based method for detecting the degree of opening and closing of a swing door as described above.
[0038] This invention acquires sample images of a swing door and trains a key point detection model based on these images. It then acquires an image to be detected and uses the key point detection model to obtain the key points of the swing door within that image. The image is captured by a robot from the target shooting perspective. Compared to detecting paper strips, this method eliminates the need for paper strips on the door, additional sensors, and wiring, effectively reducing implementation and maintenance costs. Next, the opening degree of the swing door is calculated based on the key points and the target shooting perspective. This degree represents the observed opening angle of the swing door panel around its rotation axis from the target shooting perspective, thus quantifying the opening degree of the swing door. This invention is applicable to robots of various heights and enables the robot to detect the opening degree of the swing door at different line-of-sight heights, improving detection accuracy. It allows for real-time monitoring and feedback without human intervention. Attached Figure Description
[0039] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0040] Figure 1 A flowchart of a robot-based method for detecting the opening degree of a swing door, provided in an embodiment of the present invention;
[0041] Figure 2 This is a schematic diagram of a single swing door captured by a robot from the target shooting perspective, as provided in an embodiment of the present invention.
[0042] Figure 3 This is a schematic diagram of a double-leaf swing door captured by a robot from the target shooting perspective, as provided in an embodiment of the present invention.
[0043] Figure 4 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation
[0044] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.
[0045] This invention is applied to patrol scenarios using patrol robots. The patrol robot captures images of the swing door to be inspected. The swing door is a common type of door, typically used for indoor or outdoor entrances. In this invention, the door panel is on the same plane as the door frame when closed and is connected to the door frame via hinges or bearings, allowing the door panel to move smoothly between opening and closing.
[0046] This invention acquires sample images of a swing door and trains a key point detection model based on these images. It then acquires an image to be detected and uses the key point detection model to obtain the key points of the swing door within that image. This image is captured by a robot from the target shooting perspective. Compared to detecting paper strips, this method eliminates the need for paper strips on the door panel, additional sensors, and wiring, effectively reducing implementation and maintenance costs. Next, the opening degree of the swing door is calculated based on the key points and the target shooting perspective. This opening degree represents the observed value of the opening angle of the swing door panel around its rotation axis from the target shooting perspective. This achieves a quantitative representation of the swing door's opening degree, is applicable to robots of various heights, and enables the robot to detect the opening degree of the swing door at different line-of-sight heights, improving detection accuracy. It also allows for real-time monitoring and feedback without human intervention.
[0047] The following uses a single-leaf swing door as an example to describe in detail the robot-based swing door opening degree detection method provided by this invention. Figure 1 As shown, the robot-based method for detecting the opening degree of a swing door includes:
[0048] Step S101: Obtain a sample image of a swing door, and train a key point detection model for the swing door based on the sample image of the swing door.
[0049] Step S102: Obtain the image to be detected. Use the swing door key point detection model to obtain the swing door key points in the image to be detected. The image to be detected is acquired by the robot from the target shooting perspective.
[0050] Step S103: Calculate the opening degree of the swing door based on the key points of the swing door and the target shooting angle. The opening degree represents the observed value of the opening angle of the swing door panel around the rotation axis under the target shooting angle.
[0051] The swing door sample image refers to a pre-acquired swing door image with key point annotation information. The swing door key point detection model is used to detect the swing door and its key point information in the image.
[0052] As an example, in step S101, this embodiment first obtains a sample image of a swing door, and then uses the sample image of the swing door to train a preset key point detection model, thereby obtaining a key point detection model for the swing door.
[0053] The image to be detected refers to an image acquired by the robot from the target shooting perspective, and the key points of the swing door refer to the corner points used to calculate the opening degree of the swing door, including but not limited to the corner points of the door panel and the corner points of the door frame.
[0054] As an example, in step S102, when the robot detects a swing door during its patrol and needs to pass through it, it first adjusts the camera to the target shooting angle, and then acquires an image from the target shooting angle to obtain the image to be detected. In this embodiment, the swing door key point detection model is used to detect the key points of the swing door in the image to be detected, and to obtain the corner points of the swing door panel and the corner points of the door frame.
[0055] The opening / closing degree refers to the observed value of the opening / closing angle of the hinged door panel around the rotation axis from the target shooting perspective. In other words, the opening / closing angle is the angle by which the door panel deviates from the door frame as presented in the image obtained from the target shooting perspective. The target shooting perspective is the plane in which the robot's line of sight is perpendicular to the door frame and directly opposite the center line of the door frame.
[0056] As an example, in step S103, after obtaining the key points of the swing door, this embodiment combines line segments based on the key points and calculates the opening degree of the swing door based on vector logic. The opening degree is a quantified value, which can more scientifically reflect the opening degree of the swing door and is used to subsequently determine whether the robot can pass through the swing door.
[0057] Compared to paper strip detection, this embodiment eliminates the need for paper strips on the door, additional sensor installation, and wiring, effectively reducing implementation and maintenance costs. Furthermore, it achieves a quantitative representation of the opening degree of the swing door. Compared to existing technologies that can only determine whether a door is open or closed but cannot determine the degree of opening or closing, this embodiment achieves precise monitoring and control of the door's opening and closing status. It is applicable to robots of various heights, enabling the robot to detect the opening degree of the swing door at different line-of-sight heights, improving detection accuracy, and providing real-time monitoring and feedback without human intervention.
[0058] In one embodiment, step S103, which involves calculating the opening degree of the swing door based on the key points of the swing door and the target shooting angle, includes:
[0059] Step S1031: Obtain the upper opening angle and lower opening angle of the swing door according to the key points of the swing door;
[0060] Step S1032: Obtain the height of the swing door panel and the line-of-sight height above the ground corresponding to the target shooting angle of the robot;
[0061] Step S1033: Calculate the opening degree of the swing door based on the door panel height, line-of-sight height from the ground, and upper and lower opening angles.
[0062] The upper opening angle refers to the angle at which the upper edge of the door panel deviates from the upper edge of the door frame, and the lower opening angle refers to the angle at which the lower edge of the door panel deviates from the lower edge of the door frame.
[0063] As an example, in step S1031, this embodiment obtains the key points for calculating the upper opening angle and the key points for calculating the lower opening angle from the key points of the swing door, and then calculates the upper opening angle and the lower opening angle based on the obtained key points.
[0064] For example, as a preferred embodiment of the present invention, step S1031, which involves obtaining the upper opening angle and lower opening angle of the swing door based on the key points of the swing door, includes:
[0065] Step S311: Obtain the key points of the swing door, which include the upper left corner of the door frame, the upper right corner of the door frame, the upper right corner of the door panel, the lower left corner of the door frame, the lower right corner of the door frame, and the lower right corner of the door panel.
[0066] Step S312: Calculate the upward opening angle of the swing door based on the upper left corner of the door frame, the upper right corner of the door frame, and the upper right corner of the door panel;
[0067] Step S313: Calculate the downward opening angle of the swing door based on the lower left corner of the door frame, the lower right corner of the door frame, and the lower right corner of the door panel.
[0068] Typically, the key points of a hinged door refer to the edges where the door panel and frame meet and their intersections. The door panel edges are divided into four segments: top, bottom, left, and right. Each edge consists of at least two points. The door frame edges are also divided into corresponding segments, corresponding to the door panel edges. The intersection points of these corresponding edge segments are then identified as the key points of the hinged door. For example... Figure 2 As shown, Figure 2 This is a schematic diagram of a swing door obtained by the robot from the target shooting perspective provided in this embodiment. The target shooting perspective is where the robot's line of sight is perpendicular to the plane containing the door frame and directly opposite the center line of the door frame. Directly opposite the center line of the door frame means that the vertical distance from the point where the line of sight intersects the door frame to the left and right edges of the door frame is the same. Figure 2 In the case of a hinged door, key points include the upper left corner A of the door frame, the upper right corner D of the door frame, the upper right corner E of the door panel, the lower left corner B of the door frame, the lower right corner F of the door panel, and the lower right corner C of the door frame. AE represents the upper edge of the door panel, BF represents the lower edge of the door panel, AB represents the left edge of the door panel, EF represents the right edge of the door panel, AD represents the upper edge of the door frame, BC represents the lower edge of the door frame, AB represents the left edge of the door frame, and DC represents the right edge of the door frame. Optionally, the opening angle of the hinged door is obtained by calculating the angle between the vectors formed by the line segments AE and AD, using the following formula:
[0069]
[0070] Where θ1 represents the opening angle, (x1,y1) represents the coordinates of the upper left corner A of the door frame, (x2,y2) represents the coordinates of the upper right corner D of the door frame, and (x3,y3) represents the coordinates of the upper right corner E of the door panel;
[0071] The downward opening angle of the swing door is obtained by calculating the angle between the vectors formed by the two line segments BF and BC. The calculation formula is as follows:
[0072]
[0073] Where θ2 represents the downward opening angle, (x4,y4) represents the coordinates of the lower left corner B of the door frame, (x5,y5) represents the coordinates of the lower right corner F of the door panel, and (x6,y6) represents the coordinates of the lower right corner C of the door frame.
[0074] Here, the door panel height refers to the actual height of the door panel, and the line of sight above the ground corresponding to the target shooting angle of the robot refers to the vertical height of the acquisition device on the robot relative to the ground, usually referring to the vertical height of the camera relative to the ground.
[0075] As an example, in step S1032, this embodiment pre-establishes a database of swing doors and their corresponding door panel heights in different scenarios. When obtaining the door panel height of the swing door, this embodiment obtains the robot's 3D point cloud data to obtain the robot's positioning information, and then searches for swing doors and their corresponding door panel heights in the corresponding scenarios based on the positioning information. When obtaining the line-of-sight height above the ground corresponding to the robot's target shooting angle, for robots that cannot adjust the camera height, the pre-stored line-of-sight height above the ground is directly called; for robots that can adjust the camera height, the robot's 3D point cloud data is obtained, and the camera's line-of-sight height above the ground is calculated based on the 3D point cloud data to obtain the line-of-sight height above the ground.
[0076] The degree of opening / closing refers to the opening / closing angle of the hinged door panel around its rotation axis as observed by the robot from the target shooting perspective. Therefore, the degree of opening / closing varies depending on the target shooting perspective. The target shooting perspective is defined as the robot's line of sight being perpendicular to the plane containing the door frame and directly facing the center line of the door frame. Directly facing the center line of the door frame means that the vertical distance from the point where the line of sight intersects the door frame to the left and right edges of the door frame is the same. Therefore, the height of the line of sight above the ground varies depending on the target shooting perspective.
[0077] As an example, in step S1033, this embodiment, when calculating the opening degree of the swing door, categorizes the line-of-sight height from the ground according to its height value. Different categories correspond to different calculation methods to adapt to the viewing angle characteristics at different line-of-sight heights from the ground. Optionally, step S1033 includes:
[0078] Step S331: When the line of sight is 0 meters above the ground, the opening degree of the swing door is the upper opening angle;
[0079] In step S332, when the height of the line of sight from the ground is any value less than the height of the door panel, the ratio of the height of the door panel to the height of the line of sight from the ground is calculated. Based on the ratio information, the upper opening angle and the lower opening angle are weighted and summed to obtain the opening degree of the swing door.
[0080] In step S333, when the line of sight is at the height of the door panel, the opening degree of the swing door is the downward opening angle.
[0081] Here, when the line of sight is 0 meters above the ground, it means that the camera is close to the ground. At this time, the camera cannot capture the lower edge of the door panel and the lower edge of the door frame, and cannot obtain the lower opening angle. However, it can capture the upper edge of the door panel and the upper edge of the door frame, and calculate the upper opening angle. At this time, the upper opening angle is used as the opening degree of the swing door.
[0082] Similarly, when the line of sight is at the same height as the door panel, it means that the camera is on the plane where the upper edge of the door frame is located. At this time, the camera cannot capture the upper edge of the door panel and the upper edge of the door frame, and cannot obtain the upper opening angle. However, it can capture the lower edge of the door panel and the lower edge of the door frame and calculate the lower opening angle. The lower opening angle is then used as the degree of opening of the swing door.
[0083] The above two scenarios represent extreme perspectives. In practical applications, the line-of-sight height from the ground is typically any value less than the door panel height. As a preferred embodiment of the invention, when the line-of-sight height from the ground is any value less than the door panel height, this embodiment calculates the ratio of the door panel height to the line-of-sight height from the ground. Based on this ratio, it sets the weights for the upper and lower opening angles, and then performs a weighted summation to obtain the opening degree of the swing door.
[0084] As a preferred example of the present invention, the following are calculation formulas for the opening degree of the swing door at three different line-of-sight heights from the ground:
[0085]
[0086] In the above formula, h represents the height of the line of sight from the ground, h′ represents the height of the door panel, and α(h) represents the opening degree of the hinged door. When the height of the line of sight from the ground is any value less than the height of the door panel, this embodiment calculates the ratio of the door panel height to the height of the line of sight from the ground. According to the ratio information The weight information of the opening and closing angle is set as follows: The weight information of the opening and closing angle is Then, the weighted sum is calculated according to the weight information to obtain the opening degree of the swing door.
[0087] By calculating the degree of opening and closing using the above method, the robot can accurately detect the degree of door opening and closing at different line-of-sight heights, and it can be applied to robots of various heights.
[0088] In one embodiment, step S101, namely acquiring a swing door sample image and training a swing door key point detection model based on the swing door sample image, includes:
[0089] Step S1011: Obtain an initial image of a swing door, wherein the initial image of the swing door includes a swing door in various open and closed states;
[0090] Step S1012: Mark the key points of the swing door in the initial image of the swing door to obtain a sample image of the swing door;
[0091] Step S1013: The key point detection model is trained using the sample images of the swing door to obtain the key point detection model of the swing door.
[0092] As a preferred example of the present invention, the keypoint detection model includes the YOLOv5-pose model. The swing door sample images should include images of swing doors in various open and closed states, and keypoint information should be pre-added. Then, the swing door sample images are used as training samples and input into the preset keypoint detection model for training, thereby obtaining the swing door keypoint detection model.
[0093] Therefore, by acquiring swing doors with multiple opening and closing states as initial samples, rich data is provided, enabling the model to have better generalization ability and ensuring good accuracy in corner detection. By marking key points of swing doors on the initial sample images, the positions of important feature points of the door, such as the edge positions of the door frame and door panel, can be accurately determined, improving the accuracy of swing door opening degree detection.
[0094] The robot-based method for detecting the opening degree of a swing door provided by this invention can also be applied to double-leaf swing doors. For ease of understanding, Figure 3 This is a schematic diagram of a double-leaf swing door captured by a robot from a target shooting perspective, as provided in an embodiment of the present invention. For the double-leaf swing door, the robot-based swing door opening degree detection method provided in this invention is used to detect the opening degree of each door panel separately, thereby obtaining the opening degree of each door panel.
[0095] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0096] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements the robot-based method for detecting the degree of opening and closing of a swing door as described above.
[0097] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a robot-based method for detecting the opening degree of a swing door.
[0098] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:
[0099] Acquire sample images of swing doors, and train a key point detection model for swing doors based on the sample images of swing doors;
[0100] The image to be detected is acquired, and the key points of the swing door in the image to be detected are obtained by the swing door key point detection model. The image to be detected is acquired by the robot from the target shooting perspective.
[0101] The opening degree of the swing door is calculated based on the key points of the swing door and the target shooting angle. The opening degree represents the observed value of the opening angle of the swing door panel around the rotation axis under the target shooting angle.
[0102] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.
[0103] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0104] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A method for detecting the opening degree of a swing door based on a robot, characterized in that, The method includes: Acquire sample images of a swing door, and train a key point detection model for the swing door based on the sample images. The key point detection model includes the YOLOv5-pose model. The image to be detected is obtained, and the key points of the swing door in the image to be detected are obtained by the swing door key point detection model. The image to be detected is acquired by the robot from the target shooting perspective. The target shooting perspective is the robot's line of sight perpendicular to the plane where the door frame is located and directly facing the center line of the door frame. The opening degree of the swing door is calculated based on the key points of the swing door and the target shooting angle. The opening degree represents the observed value of the opening angle of the swing door panel around the rotation axis under the target shooting angle. The step of calculating the opening degree of the swing door based on the key points of the swing door and the target shooting angle includes: Based on the key points of the swing door, obtain the upper opening angle and the lower opening angle of the swing door respectively; Obtain the height of the swing door panel and the line-of-sight height above the ground corresponding to the target shooting angle of the robot; The opening degree of the hinged door is calculated based on the door panel height, the line-of-sight height from the ground, and the upper and lower opening angles. Specifically, this includes: when the line-of-sight height from the ground is 0, the opening degree of the hinged door is the upper opening angle; when the line-of-sight height from the ground is any value less than the door panel height, the ratio of the door panel height to the line-of-sight height from the ground is calculated, and the upper and lower opening angles are weighted and summed based on the ratio to obtain the opening degree of the hinged door; when the line-of-sight height from the ground is the door panel height, the opening degree of the hinged door is the lower opening angle.
2. The method for detecting the opening degree of a swing door based on a robot as described in claim 1, characterized in that, The step of obtaining the upper and lower opening angles of the swing door based on the key points of the swing door includes: Obtain the key points of the swing door, which include the upper left corner of the door frame, the upper right corner of the door frame, the upper right corner of the door panel, the lower left corner of the door frame, the lower right corner of the door frame, and the lower right corner of the door panel; The upward opening angle of the swing door is calculated based on the upper left corner of the door frame, the upper right corner of the door frame, and the upper right corner of the door panel. The downward opening angle of the swing door is calculated based on the lower left corner of the door frame, the lower right corner of the door frame, and the lower right corner of the door panel.
3. The robot-based method for detecting the opening degree of a swing door as described in claim 2, characterized in that, The formula for calculating the upward opening angle of the swing door is: ； in, Let (x1, y1) represent the opening angle, (x2, y2) represent the coordinates of the upper left corner of the door frame, (x3, y3) represent the coordinates of the upper right corner of the door frame, and (x3, y3) represent the coordinates of the upper right corner of the door panel. The formula for calculating the downward opening angle of the swing door is: ； in, Let (x4, y4) represent the lower opening angle, (x5, y5) represent the coordinates of the lower left corner of the door frame, (x6, y6) represent the coordinates of the lower right corner of the door panel, and (x6, y6) represent the coordinates of the lower right corner of the door frame.
4. The method for detecting the opening degree of a swing door based on a robot as described in claim 1, characterized in that, The step of acquiring sample images of swing doors and training a key point detection model for swing doors based on the sample images includes: Acquire an initial image of a swing door, the initial image of which includes a swing door in various open and closed states; The initial image of the swing door is annotated with key points to obtain a sample image of the swing door; The key point detection model is trained using the sample images of the swing door to obtain the key point detection model for the swing door.
5. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the robot-based method for detecting the degree of opening and closing of a swing door as described in any one of claims 1 to 4.
6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the robot-based method for detecting the degree of opening and closing of a swing door as described in any one of claims 1 to 4.