Method and apparatus for determining distances of warning tape, and electronic device and storage medium

Abstract

Provided in the present application are a method and apparatus for determining the distances of a warning tape, and an electronic device and a storage medium. The method comprises: detecting whether there is a warning tape in both an environment image at a first moment and an environment image at a second moment, wherein a camera apparatus is mounted on a robot; if so, respectively extracting warning tape contours from the environment image at the first moment and the environment image at the second moment, and determining a first warning tape contour and a second warning tape contour; on the basis of feature points in the first warning tape contour and feature points in the second warning tape contour, determining target feature points; and on the basis of the pixel coordinates of the target feature points, the camera apparatus, and wheel odometer data between the environment image at the first moment and the environment image at the second moment, determining the distances between the robot and the warning tape, such that the distances between the warning tape and the robot at different moments are quickly and accurately determined.

Description

Method, device, electronic device and storage medium for determining warning zone distance

[0001] This application claims priority to the Chinese patent application with application number 202311558850.9 filed with the China Patent Office on November 21, 2023, and invention name “A method, device, electronic device and storage medium for determining the distance of a warning zone”, and introduces the entire contents of the above patent application into this application by reference. Technical Field

[0002] The present application relates to the field of robotics, and in particular to a method, device, electronic device, and storage medium for determining a distance from a warning zone. Background Art

[0003] Warning tape is a common feature at inspection sites. When avoiding obstacles within warning tape, robots need to calculate the distance between them. However, due to the shape and installation method of the warning tape, it is difficult to scan and detect the tape using multi-line lidar, making it difficult to obtain distance information. In the mobile robotics industry, depth cameras are generally used to measure the distance to objects such as warning tape. However, due to the limited measurement range of depth cameras and their significant interference in outdoor environments, they are unable to obtain accurate distance information to objects such as warning tape in the environment. Therefore, improving the accuracy of warning tape distance determination has become a technical issue that cannot be underestimated. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method, device, electronic device and storage medium for determining the distance of the warning belt, which can quickly and accurately determine the distance between the warning belt and the robot at different times through the pixel coordinates of the target feature points, the robot's wheel odometer data and the camera device at different times.

[0005] The present application provides a method for determining the distance of a warning zone, the method comprising:

[0006] Acquire an environmental image captured in real time by a camera device, and detect whether a warning belt exists in both the environmental image at a first moment and the environmental image at a second moment; wherein the camera device is mounted on the robot;

[0007] If yes, extracting the outlines of the warning belts in the environment image at the first moment and the environment image at the second moment respectively to determine the first warning belt outline and the second warning belt outline;

[0008] Determining target feature points based on feature points in the first warning zone outline and feature points in the second warning zone outline;

[0009] The distance between the robot and the warning belt is determined based on the pixel coordinates of the target feature point, the camera device, and the wheel odometer data between the environment image at the first moment and the environment image at the second moment.

[0010] In a possible implementation, determining target feature points based on feature points in the first warning zone outline and feature points in the second warning zone outline includes:

[0011] Extracting feature points in the first warning zone outline and feature points in the second warning zone outline;

[0012] Feature points in the first warning zone outline and feature points in the second warning zone outline are matched, and successfully matched target feature points are determined based on the Hamming distance.

[0013] In a possible implementation, determining target feature points based on feature points in the first warning zone outline and feature points in the second warning zone outline includes:

[0014] Extracting feature points in the first warning zone outline and feature points in the second warning zone outline;

[0015] Feature points in the first warning zone outline and feature points in the second warning zone outline are matched, and successfully matched target feature points are determined based on the Hamming distance.

[0016] In a possible implementation, with respect to the first warning zone outline, extracting feature points in the first warning zone outline and feature points in the second warning zone outline includes:

[0017] Determining the coordinates of a circumscribed rectangle of the first warning zone outline based on a pixel coordinate range of the first warning zone outline;

[0018] Determine a feature extraction area of ​​the first warning zone outline based on an image ROI setting rule and the circumscribed rectangle coordinates;

[0019] The feature extraction area is divided into a plurality of sub-areas, feature points in each sub-area are extracted respectively, and the number of feature points extracted from each sub-area is controlled.

[0020] In one possible implementation, determining the distance between the robot and the warning belt based on the target feature point, the camera device, and wheel odometer data between the environment image at the first moment and the environment image at the second moment includes:

[0021] Based on the camera optical center position of the camera device at the first moment, the camera optical center position of the camera device at the second moment, the wheel odometer data between the environmental image at the first moment and the environmental image at the second moment, and the similar triangle relationship formed by the pixel coordinates of the target feature point on the physical imaging plane, the distance between the robot and the warning belt at the first moment and the distance between the robot and the warning belt at the second moment are determined.

[0022] In one possible implementation, for a plurality of target feature points, determining the distance between the robot and the warning tape based on pixel coordinates of the target feature points, the camera device, and wheel odometer data between the environment image at the first moment and the environment image at the second moment further includes:

[0023] For each target feature point, determining a first reference distance between the robot and the warning tape at the target feature point at the first moment, and a second reference distance between the robot and the warning tape at the target feature point at the second moment based on the pixel coordinates of the target feature point, the camera device, and wheel odometer data between the environment image at the first moment and the environment image at the second moment;

[0024] Performing least squares processing on the first reference distances of the plurality of target feature points to determine the distance between the robot and the warning belt at a first moment;

[0025] The second reference distances of the plurality of target feature points are processed by least square method to determine the distance between the robot and the warning belt at two moments.

[0026] In a possible implementation, after extracting the warning zone outlines from the environment image at the first moment and the environment image at the second moment to determine the first warning zone outline and the second warning zone outline, the determination method further includes:

[0027] If the target feature point is not determined in the first warning zone outline and the second warning zone outline, increasing the total number of extracted feature points;

[0028] Alternatively, the midpoint of the intersection line of a perpendicular line to the horizontal center point of the first warning zone outline and the first warning zone outline is determined as the target feature point.

[0029] The embodiment of the present application further provides a device for determining the distance of a warning zone, characterized in that the device comprises:

[0030] a warning tape recognition module, configured to obtain an environmental image captured in real time by a camera device and detect whether a warning tape exists in both the environmental image at a first moment and the environmental image at a second moment; wherein the camera device is mounted on the robot;

[0031] a contour extraction module, configured to, if yes, extract the contours of the warning belt in the environment image at the first moment and the environment image at the second moment respectively, to determine a first warning belt contour and a second warning belt contour;

[0032] A feature point matching module, configured to determine target feature points based on feature points in the first warning zone outline and feature points in the second warning zone outline;

[0033] A warning zone distance calculation module is used to determine the distance between the robot and the warning zone based on the pixel coordinates of the target feature point, the camera device, and the wheel odometer data between the environment image at the first moment and the environment image at the second moment.

[0034] In a possible implementation manner, when the feature point matching module is used to determine the target feature points based on the feature points in the first warning zone outline and the feature points in the second warning zone outline, the feature point matching module is specifically used to:

[0035] Extracting feature points in the first warning zone outline and feature points in the second warning zone outline;

[0036] Feature points in the first warning zone outline and feature points in the second warning zone outline are matched, and successfully matched target feature points are determined based on the Hamming distance.

[0037] An embodiment of the present application also provides an electronic device, comprising: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the method for determining the distance of the warning zone as described above are performed.

[0038] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program executes the steps of the method for determining the distance of the warning zone as described above.

[0039] The embodiments of the present application provide a method, device, electronic device, and storage medium for determining the distance of a warning zone. The method includes: obtaining an environmental image captured in real time by a camera device, detecting whether a warning zone exists in both the environmental image at a first moment and the environmental image at a second moment; wherein the camera device is mounted on a robot; if so, extracting the outline of the warning zone from the environmental image at the first moment and the environmental image at the second moment, respectively, to determine a first warning zone outline and a second warning zone outline; determining a target feature point based on the feature points in the first warning zone outline and the feature points in the second warning zone outline; and determining the distance between the robot and the warning zone based on the pixel coordinates of the target feature point, the camera device, and the wheel odometer data between the environmental image at the first moment and the environmental image at the second moment. Using the pixel coordinates of the target feature point, the robot's wheel odometer data, and the camera device at different moments, the distance between the warning zone and the robot at different moments can be quickly and accurately determined.

[0040] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0042] FIG1 is a flow chart of a method for determining a warning zone distance provided in an embodiment of the present application;

[0043] FIG2 is a schematic diagram of similar triangle relationships constructed by imaging of an imaging device provided in an embodiment of the present application;

[0044] FIG3 is a schematic diagram of a method for determining a distance of a warning zone provided in an embodiment of the present application;

[0045] FIG4 is a schematic diagram of a structure of a device for determining a distance to a warning zone provided in an embodiment of the present application;

[0046] FIG5 is a second structural diagram of a device for determining a distance to a warning zone provided in an embodiment of the present application;

[0047] FIG6 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work falls within the scope of protection of the present application.

[0049] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0050] In order to enable those skilled in the art to use the contents of this application, the following implementation method is provided in combination with the specific application scenario of "determining the distance of the warning zone". For those skilled in the art, the general principles defined here can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application.

[0051] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0052] The following methods, devices, electronic devices or computer-readable storage media of the embodiments of the present application can be applied to any scenario where the distance to the warning zone needs to be determined. The embodiments of the present application are not limited to specific application scenarios. Any solution using the method, device, electronic device and storage medium for determining the distance to the warning zone provided by the embodiments of the present application is within the scope of protection of this application.

[0053] Research has found that warning tape is a common feature at inspection sites. When avoiding obstacles within these tapes, robots need to calculate the distance between them. However, due to the tape's shape and installation method, it's difficult to scan and detect it using multi-line lidar, making it difficult to obtain distance information. In the mobile robotics industry, distances to objects like warning tapes are typically calculated using depth cameras. However, due to the limited range of depth cameras and their significant interference in outdoor environments, accurate distance information to objects like warning tapes cannot be obtained. Therefore, improving the accuracy of distance determination within warning tapes has become a significant technical challenge.

[0054] Based on this, an embodiment of the present application provides a method for determining the distance of a warning belt, which quickly and accurately determines the distance between the warning belt and the robot at different times through the pixel coordinates of the target feature points, the robot's wheel odometer data, and the camera device at different times.

[0055] Please refer to Figure 1, which is a flow chart of a method for determining the distance of a warning zone provided in an embodiment of the present application. As shown in Figure 1, the determination method provided in an embodiment of the present application includes:

[0056] S101: Acquire an environmental image captured in real time by a camera device, and detect whether a warning belt exists in both the environmental image at a first moment and the environmental image at a second moment; wherein the camera device is installed on a robot.

[0057] In this step, the environment image captured by the camera device in real time is obtained, and then it is detected whether the warning belt exists in both the environment image at the first moment and the environment image at the second moment.

[0058] The first moment and the second moment are two moments separated by a preset time interval.

[0059] Here, whether a warning zone exists in the environment image at the first moment is determined by performing preprocessing, contour extraction, contour screening, and warning zone contour judgment on the environment image at the first moment.

[0060] S102: If yes, extract the outlines of the warning zones in the environment image at the first moment and the environment image at the second moment respectively to determine a first warning zone outline and a second warning zone outline.

[0061] In this step, if it exists, the outlines of the warning belt in the environment image at the first moment and the environment image at the second moment are extracted respectively to determine the first warning belt outline and the second warning belt outline.

[0062] S103: Determine target feature points based on the feature points in the first warning zone outline and the feature points in the second warning zone outline.

[0063] In this step, target feature points are determined based on the feature points in the first warning zone outline and the feature points in the second warning zone outline.

[0064] In a possible implementation, determining target feature points based on feature points in the first warning zone outline and feature points in the second warning zone outline includes:

[0065] A: Extracting feature points in the first warning zone outline and feature points in the second warning zone outline.

[0066] Here, feature points in the first warning zone outline and feature points in the second warning zone outline are extracted.

[0067] In a possible implementation, with respect to the first warning zone outline, extracting feature points in the first warning zone outline and feature points in the second warning zone outline includes:

[0068] I: Determine the coordinates of the circumscribed rectangle of the first warning zone outline based on the outline pixel coordinate range of the first warning zone outline.

[0069] Here, the coordinates of the circumscribed rectangle of the first warning zone outline are determined according to the outline pixel coordinate range of the first warning zone outline.

[0070] II: Based on the image ROI setting rules and the circumscribed rectangle coordinates, determine the feature extraction area of ​​the first warning zone outline.

[0071] Here, the feature extraction area of ​​the first warning zone outline is determined according to the image ROI setting rules and the circumscribed rectangle coordinates.

[0072] III: Divide the feature extraction area into multiple sub-areas, extract feature points in each sub-area respectively, and control the number of feature points extracted from each sub-area.

[0073] Here, the feature extraction region is divided into a plurality of sub-regions, feature points in each sub-region are extracted respectively, and the number of feature points extracted from each sub-region is controlled.

[0074] Based on the pixel coordinate range of the identified warning tape outline, a region of interest (ROI) is set for the image so that the warning tape outline is contained within the ROI. Feature extraction is performed within the ROI, narrowing the feature extraction range. Specifically, the coordinates of the warning tape's bounding rectangle are obtained based on the warning tape outline. The vertex coordinates of the warning tape's bounding rectangle are (x1, y1), (x2, y2), (x3, y3), and (x4, y4). The length and width of the set rectangular ROI are determined based on the length and width of the image and the length and width of the warning tape outline. The specific principles for setting the ROI are as follows: the ROI must be able to completely contain the warning tape outline; the ROI width (vertical distance) should not be too small, especially when the warning tape outline width is relatively small; the minimum width can be set to 1 / 5 of the image's vertical width; and the ROI length (horizontal distance) can be set to the image length, with the minimum length being set to 1 / 4 of the image's horizontal length. The feature extraction area is then divided into several sub-areas. Feature extraction is performed on each sub-area separately. The number of feature points to be extracted in each sub-area is controlled to ensure that the extracted feature points are evenly distributed in the image, increasing the probability that the feature points fall within the warning zone outline. ORB feature points are then extracted from each sub-area using the OpenCV library function. A range of extraction numbers can be set for each sub-area to control the number of feature points extracted. If the number of extracted features does not meet the requirements, the pixel difference threshold used in the FAST corner calculation in the ORB feature point extraction function can be adjusted within a certain range to ensure that the set number of feature points is extracted for each sub-area.

[0075] Here, the process of extracting the feature points from the second warning zone outline is consistent with the process of extracting the feature points from the first warning zone outline, and will not be described in detail in this section.

[0076] B: Matching the feature points in the first warning zone outline and the feature points in the second warning zone outline, and determining the successfully matched target feature points based on the Hamming distance.

[0077] Here, the feature points in the first warning zone outline and the feature points in the second warning zone outline are matched, and the successfully matched target feature points are determined based on the Hamming distance.

[0078] Since the number of feature points in the warning zone outline is not too large, brute force matching can be used directly for solving the problem. When using the brute force matching method, the matching result is selected based on the Hamming distance.

[0079] In a possible implementation, after extracting the warning zone outlines from the environment image at the first moment and the environment image at the second moment to determine the first warning zone outline and the second warning zone outline, the determination method further includes:

[0080] If the target feature point is not determined in the first warning zone outline and the second warning zone outline, the total number of extracted feature points is increased; or, the midpoint of the intersection line of the perpendicular line of the horizontal center point of the first warning zone outline and the first warning zone outline is determined as the target feature point.

[0081] Here, if the target feature point is not determined in the first warning zone outline and the second warning zone outline, the total number of extracted feature points is increased; or the midpoint of the intersection line of the vertical line of the horizontal center point of the first warning zone outline and the first warning zone outline is determined as the target feature point.

[0082] Among them, if the target feature points that are successfully matched within the warning zone outline are not successfully obtained, the following methods can be used to handle it. First, the probability of feature points in the warning zone outline is increased by increasing the total number of extracted feature points, thereby increasing the probability of successful matching. For example, the number of extracted feature points is initially set to 500. If no matching point pairs are obtained, the number of feature points can be increased to 1000. If the matching points in the warning zone outline cannot be obtained by adjusting the number of feature points within a certain range, the pixel coordinates of the vertical line of the horizontal center point of the warning zone outline in the two frames of images can be calculated inside the outline. Specifically, the midpoint of the intersection of the vertical line and the outline can be selected as the coordinates of the target feature point.

[0083] S104: Determine the distance between the robot and the warning belt based on the pixel coordinates of the target feature point, the camera device, and the wheel odometer data between the environment image at the first moment and the environment image at the second moment.

[0084] In this step, the distance between the robot and the warning belt is determined based on the pixel coordinates of the target feature point, the camera device, and the wheel odometer data between the environment image at the first moment and the environment image at the second moment.

[0085] Among them, the main function of the wheel odometry is to provide the robot's position data at different times, which is used to calculate the robot's position movement distance at two times. Since the monocular camera is usually installed by connecting it to the robot's rigid body, the movement distance of the camera at two times can be obtained.

[0086] In one possible implementation, determining the distance between the robot and the warning belt based on the target feature point, the camera device, and wheel odometer data between the environment image at the first moment and the environment image at the second moment includes:

[0087] Based on the camera optical center position of the camera device at the first moment, the camera optical center position of the camera device at the second moment, the wheel odometer data between the environmental image at the first moment and the environmental image at the second moment, and the similar triangle relationship formed by the pixel coordinates of the target feature point on the physical imaging plane, the distance between the robot and the warning belt at the first moment and the distance between the robot and the warning belt at the second moment are determined.

[0088] Here, when the determined target feature point is one, the distance between the robot and the warning belt at the first moment and the distance between the robot and the warning belt at the second moment are determined based on the similar triangle relationship formed by the camera optical center position of the camera device at the first moment, the camera optical center position of the camera device at the second moment, the wheel odometer data between the environmental image at the first moment and the environmental image at the second moment, and the pixel coordinates of the target feature point on the physical imaging plane.

[0089] In this scheme, the camera device on the mobile robot is installed horizontally. The environmental data is collected by the camera device. The position information of the robot at different times during the movement can be obtained based on the wheel odometer data on the robot. Similar triangles are constructed through the geometric relationship in the imaging model of the camera device at different times. By solving the triangle, the distance between the warning belt and the robot at different times can be calculated.

[0090] Further, please refer to Figure 2, which is a schematic diagram of the similar triangle relationship constructed by the imaging device provided in the embodiment of the present application. As shown in Figure 2, according to the camera device, a similar triangle relationship is formed for a target feature point on the warning belt in the world coordinate system, the optical center of the camera, and the image point of the target feature point on the warning belt on the physical imaging plane. According to the geometric properties of similar triangles, a set of equations is established to solve and determine the distance between the robot and the warning belt at different times. In Figure 2, O1 and O2 are the positions of the optical center of the camera at two moments, M is the target feature point on the warning belt, B1 and B2 are the projection points of the optical center of the camera on the physical imaging plane at two moments, Y1 and Y2 are the imaging points of M on the physical imaging plane at two moments, A1 and A2 are the vertical projection points of the optical center of the camera on the ground at two moments, and G is the vertical projection point of point M on the ground. Since the camera device is installed horizontally, let the height of the optical center above the ground be h, then the length of the line segment in Figure 2 satisfies the geometric relationship: O1A1=O2A2=NG=h. The focal length of the camera can be obtained by calibrating the camera's internal parameters, set as f, that is, in the above figure: B1O1=B2O2=f, the distance Δd moved by the camera at two times can be obtained by the robot's wheeled odometer. Assuming that the distances from the camera to the warning belt at the two times are d1 and d2 respectively, then: d1=A1G, d2=A2G, Δd=d1-d2. A point on the warning belt at two times and the corresponding imaging point can form a similar triangle relationship according to the camera imaging model, that is, triangle O1B1Y1 is similar to triangle O1NM, and triangle O2B2Y2 is similar to triangle O2NM. According to the geometric properties of similar triangles, the following equation can be obtained: Δd = d1 - d2. The three equations above can be combined to form a system of equations. Δd is obtained from the robot's wheel odometer data, the camera focal length f is obtained through camera intrinsic calibration, and B1Y1 and B2Y2 are derived from the pixel coordinates of the warning zone's feature points and the camera's intrinsic parameters. The unknown quantities are MN, d1, and d2. Solving this system of equations yields the distances d1 and d2 between the camera and the warning zone at the corresponding moment. Based on the camera's mounting position on the robot, the distance between the warning zone and the robot at the corresponding moment can be determined.

[0091] In one possible implementation, for a plurality of target feature points, determining the distance between the robot and the warning tape based on pixel coordinates of the target feature points, the camera device, and wheel odometer data between the environment image at the first moment and the environment image at the second moment further includes:

[0092] i: For each target feature point, based on the pixel coordinates of the target feature point, the camera device, and the wheel odometer data between the environmental image at the first moment and the environmental image at the second moment, determine the first reference distance between the robot and the warning belt at the target feature point at the first moment, and the second reference distance between the robot and the warning belt at the target feature point at the second moment.

[0093] Here, if multiple target feature points are determined, for each target feature point, based on the pixel coordinates of the target feature point, the camera device, and the wheel odometer data between the environmental image at the first moment and the environmental image at the second moment, the first reference distance between the robot and the warning belt at the target feature point at the first moment and the second reference distance between the robot and the warning belt at the target feature point at the second moment are determined.

[0094] ii: performing least squares processing on the first reference distances of the plurality of target feature points to determine the distance between the robot and the warning zone at the first moment; performing least squares processing on the second reference distances of the plurality of target feature points to determine the distance between the robot and the warning zone at the second moment.

[0095] Here, the first reference distances of the multiple target feature points are processed by least squares method to determine the distance between the robot and the warning belt at the first moment; the second reference distances of the multiple target feature points are processed by least squares method to determine the distance between the robot and the warning belt at the second moment.

[0096] Further, please refer to Figure 3, which is a schematic diagram of a method for determining the distance of a warning belt provided in an embodiment of the present application. As shown in Figure 3, the camera device captures environmental images, and subscribes to image data through ROS (Robot Operating System) communication to obtain environmental images at two moments in a specific time interval. Determine whether there is a warning belt in the image. If there is a warning belt, extract the contours of the warning belt in the environmental images at the two moments respectively; if there is no warning belt, continue to subscribe to the image and perform warning belt recognition judgment. When there is a warning belt in the two frames of environmental images, extract ORB feature points from the two frames of environmental images respectively, obtain the ORB feature points in the outline of the warning belt, and match the feature points in the outline of the warning belt of the two frames of images to obtain the matched target feature points. According to the pixel coordinates of the matched target feature points and the wheel odometer data information at the corresponding moment of the image, calculate the distance between the warning belt and the camera, and according to the installation method of the camera device on the robot, obtain the distance between the warning belt and the robot, and publish the calculated distance result through ROS.

[0097] In this solution, the executor for determining the distance to the warning tape is ROS (Robot Operating System). ROS obtains the environmental image, analyzes the environmental image, identifies and extracts the outline information of the warning tape in the image, obtains the corresponding matching feature points on the warning tape in the two frames of image, and then calculates the distance between the warning tape and the robot based on the wheel odometer data.

[0098] An embodiment of the present application provides a method for determining the distance of a warning zone, the method comprising: obtaining an environmental image captured in real time by a camera device, detecting whether a warning zone exists in both the environmental image at a first moment and the environmental image at a second moment; wherein the camera device is mounted on a robot; if so, extracting the outline of the warning zone in the environmental image at the first moment and the environmental image at the second moment, respectively, to determine a first warning zone outline and a second warning zone outline; determining a target feature point based on the feature points in the first warning zone outline and the feature points in the second warning zone outline; and determining the distance between the robot and the warning zone based on the pixel coordinates of the target feature point, the camera device, and the wheel odometer data between the environmental image at the first moment and the environmental image at the second moment. Using the pixel coordinates of the target feature point, the robot's wheel odometer data, and the camera device at different moments, the distance between the warning zone and the robot at different moments can be quickly and accurately determined.

[0099] Please refer to Figures 4 and 5. Figure 4 is a schematic diagram of the structure of a device for determining the distance to a warning zone provided in an embodiment of the present application; Figure 5 is a schematic diagram of the structure of a device for determining the distance to a warning zone provided in an embodiment of the present application. As shown in Figure 4, the determining device 400 includes:

[0100] a warning tape recognition module 410 for acquiring an environmental image captured in real time by a camera device and detecting whether a warning tape exists in both the environmental image at a first moment and the environmental image at a second moment; wherein the camera device is mounted on the robot;

[0101] The contour extraction module 420 is configured to extract the contours of the warning zone in the environment image at the first moment and the environment image at the second moment respectively, to determine a first warning zone contour and a second warning zone contour;

[0102] A feature point matching module 430 is configured to determine target feature points based on the feature points in the first warning zone outline and the feature points in the second warning zone outline;

[0103] The warning zone distance calculation module 440 is used to determine the distance between the robot and the warning zone based on the pixel coordinates of the target feature point, the camera device, and the wheel odometer data between the environment image at the first moment and the environment image at the second moment.

[0104] Furthermore, when the feature point matching module 430 is used to determine the target feature points based on the feature points in the first warning zone outline and the feature points in the second warning zone outline, the feature point matching module 430 is specifically used to:

[0105] Extracting feature points in the first warning zone outline and feature points in the second warning zone outline;

[0106] Feature points in the first warning zone outline and feature points in the second warning zone outline are matched, and successfully matched target feature points are determined based on the Hamming distance.

[0107] Furthermore, when the feature point matching module 430 is used to extract the feature points in the first warning zone outline and the feature points in the second warning zone outline, the feature point matching module 430 is specifically used to:

[0108] Determining the coordinates of a circumscribed rectangle of the first warning zone outline based on a pixel coordinate range of the first warning zone outline;

[0109] Determine a feature extraction area of ​​the first warning zone outline based on an image ROI setting rule and the circumscribed rectangle coordinates;

[0110] The feature extraction area is divided into a plurality of sub-areas, feature points in each sub-area are extracted respectively, and the number of feature points extracted from each sub-area is controlled.

[0111] Furthermore, when the warning zone distance calculation module 440 is used to determine the distance between the robot and the warning zone based on the target feature point, the camera device, and the wheel odometer data between the environment image at the first moment and the environment image at the second moment, the warning zone distance calculation module 440 is specifically configured to:

[0112] Based on the camera optical center position of the camera device at the first moment, the camera optical center position of the camera device at the second moment, the wheel odometer data between the environmental image at the first moment and the environmental image at the second moment, and the similar triangle relationship formed by the pixel coordinates of the target feature point on the physical imaging plane, the distance between the robot and the warning belt at the first moment and the distance between the robot and the warning belt at the second moment are determined.

[0113] Furthermore, when the warning zone distance calculation module 440 is used to determine the distance between the robot and the warning zone for a plurality of target feature points based on the pixel coordinates of the target feature points, the camera device, and the wheel odometer data between the environment image at the first moment and the environment image at the second moment, the warning zone distance calculation module 440 is specifically used to:

[0114] For each target feature point, determining a first reference distance between the robot and the warning tape at the target feature point at the first moment, and a second reference distance between the robot and the warning tape at the target feature point at the second moment based on the pixel coordinates of the target feature point, the camera device, and wheel odometer data between the environment image at the first moment and the environment image at the second moment;

[0115] Performing least squares processing on the first reference distances of the plurality of target feature points to determine the distance between the robot and the warning belt at a first moment;

[0116] The second reference distances of the plurality of target feature points are processed by least square method to determine the distance between the robot and the warning belt at two moments.

[0117] Furthermore, as shown in FIG5 , the apparatus 400 for determining the distance of the warning zone further includes a feature point modification module 450 , which is configured to:

[0118] If the target feature point is not determined in the first warning zone outline and the second warning zone outline, increasing the total number of extracted feature points;

[0119] Alternatively, the midpoint of the intersection line of a perpendicular line to the horizontal center point of the first warning zone outline and the first warning zone outline is determined as the target feature point.

[0120] An embodiment of the present application provides a device for determining the distance of a warning zone, the device comprising: a warning zone recognition module for acquiring an environmental image captured in real time by a camera device and detecting whether a warning zone exists in both the environmental image at a first moment and the environmental image at a second moment; wherein the camera device is mounted on a robot; a contour extraction module for, if so, extracting the contours of the warning zone from the environmental image at the first moment and the environmental image at the second moment, respectively, to determine a first warning zone contour and a second warning zone contour; a feature point matching module for determining a target feature point based on feature points in the first warning zone contour and feature points in the second warning zone contour; and a warning zone distance calculation module for determining the distance between the robot and the warning zone based on the pixel coordinates of the target feature point, the camera device, and wheel odometer data between the environmental image at the first moment and the environmental image at the second moment. Using the pixel coordinates of the target feature point, the robot's wheel odometer data, and the camera device at different moments, the distance between the warning zone and the robot at different moments can be quickly and accurately determined.

[0121] Please refer to Figure 6 , which is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. As shown in Figure 6 , the electronic device 600 includes a processor 610 , a memory 620 , and a bus 630 .

[0122] The memory 620 stores machine-readable instructions executable by the processor 610. When the electronic device 600 is running, the processor 610 communicates with the memory 620 through the bus 630. When the machine-readable instructions are executed by the processor 610, the steps of the method for determining the warning zone distance in the method embodiment shown in Figure 1 above can be executed. The specific implementation method can be found in the method embodiment and will not be repeated here.

[0123] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for determining the distance of the warning zone in the method embodiment shown in FIG1 above can be executed. The specific implementation method can be found in the method embodiment and will not be repeated here.

[0124] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0125] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. There may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interface, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0126] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0127] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0128] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0129] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-mentioned embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. These modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for determining the distance of a warning zone, characterized in that: The determination method comprises: Acquire an environment image captured by a camera device in real time, and detect whether a warning belt exists in both the environment image at the first moment and the environment image at the second moment; wherein the camera device is installed on the robot; If yes, respectively extract the outlines of the warning belt in the environment image at the first moment and the environment image at the second moment to determine the first warning belt outline and the second warning belt outline; Determining target feature points based on feature points in the first warning zone outline and feature points in the second warning zone outline; Based on the pixel coordinates of the target feature point, the camera device, and the wheel odometer data between the environment image at the first moment and the environment image at the second moment, the distance between the robot and the warning belt is determined.

2. The determination method according to claim 1, characterized in that: The determining of target feature points based on the feature points in the first warning zone outline and the feature points in the second warning zone outline includes: Extracting feature points in the first warning zone outline and feature points in the second warning zone outline; The feature points in the first warning zone outline and the feature points in the second warning zone outline are matched, and target feature points that are successfully matched are determined based on the Hamming distance.

3. The determination method according to claim 2, characterized in that: With respect to the first warning zone outline, extracting feature points in the first warning zone outline and feature points in the second warning zone outline includes: Determining the coordinates of the circumscribed rectangle of the first warning belt outline based on the outline pixel coordinate range of the first warning belt outline; Determine a feature extraction area of ​​the first warning zone contour based on the image ROI setting rule and the circumscribed rectangle coordinates; The feature extraction area is divided into a plurality of sub-areas, feature points in each sub-area are extracted respectively, and the number of feature points extracted from each sub-area is controlled.

4. The determination method according to claim 1, characterized in that: The determining of the distance between the robot and the warning belt based on the wheel odometer data between the camera device and the environment image at the first moment and the environment image at the second moment based on the target feature point includes: Based on the camera optical center position of the camera device at the first moment, the camera optical center position of the camera device at the second moment, the wheel odometer data between the environment image at the first moment and the environment image at the second moment, and the similar triangle relationship formed by the pixel coordinates of the target feature point on the physical imaging plane, the distance between the robot and the warning belt at the first moment and the distance between the robot and the warning belt at the second moment are determined.

5. The determination method according to claim 1, characterized in that: For the plurality of target feature points, determining the distance between the robot and the warning belt based on the pixel coordinates of the target feature points, the camera device, and the wheel odometer data between the environment image at the first moment and the environment image at the second moment, further comprising: For each of the target feature points, based on the pixel coordinates of the target feature point, the camera device, and the wheel odometer data between the environment image at the first moment and the environment image at the second moment, determine a first reference distance between the robot and the warning belt at the target feature point at the first moment, and a second reference distance between the robot and the warning belt at the target feature point at the second moment; Performing least squares processing on the first reference distances of the plurality of target feature points to determine the distance between the robot and the warning belt at a first moment; The second reference distances of the plurality of target feature points are processed by least square method to determine the distance between the robot and the warning belt at two moments.

6. The determination method according to claim 1, characterized in that: After respectively extracting the warning zone contours in the environment image at the first moment and the environment image at the second moment to determine the first warning zone contour and the second warning zone contour, the determination method further includes: If the target feature point is not determined in the first warning zone outline and the second warning zone outline, increasing the total number of extracted feature points; Alternatively, the midpoint of the intersection line of a vertical line of the horizontal center point of the first warning belt outline and the first warning belt outline is determined as the target feature point.

7. A device for determining the distance of a warning belt, characterized in that: The determining device comprises: A warning belt recognition module is used to obtain the environment image collected by the camera device in real time, and detect whether the warning belt exists in both the environment image at the first moment and the environment image at the second moment; wherein the camera device is installed on the robot; A contour extraction module, for respectively extracting the contours of the warning belt in the environment image at the first moment and the environment image at the second moment, to determine the first warning belt contour and the second warning belt contour; A feature point matching module, used to determine target feature points based on feature points in the first warning zone outline and feature points in the second warning zone outline; The warning zone distance calculation module is used to determine the distance between the robot and the warning zone based on the pixel coordinates of the target feature point, the camera device, and the wheel odometer data between the environment image at the first moment and the environment image at the second moment.

8. The determination device according to claim 7, characterized in that: When the feature point matching module is used to determine the target feature points based on the feature points in the first warning zone outline and the feature points in the second warning zone outline, the feature point matching module is specifically used to: Extracting feature points in the first warning zone outline and feature points in the second warning zone outline; The feature points in the first warning zone outline and the feature points in the second warning zone outline are matched, and target feature points that are successfully matched are determined based on the Hamming distance.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the method for determining the distance of the warning zone as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for determining the distance of the warning zone according to any one of claims 1 to 6 are executed.

Images