Automatic driving method and device of patrol robot

Abstract

The invention is suitable for the technical field of image processing, and provides an automatic driving method and device for a patrol robot, and the method comprises the steps: collecting real-time image data in a process that the patrol robot advances based on a preset path; extracting a plurality of real-time feature object areas in the real-time image data and image positions corresponding to the plurality of real-time feature object areas; correcting the current environment coordinate system according to the feature object areas corresponding to the plurality of pre-stored video frames, the feature positions corresponding to the feature object areas, the plurality of real-time feature object areas and the image positions corresponding to the plurality of real-time feature object areas to obtain a target environment coordinate system; and controlling the patrol robot to execute a patrol task based on the target environment coordinate system and a preset path. According to the method, the positioning precision and the automatic driving performance of the robot in a complex environment can be remarkably improved, and the problems of positioning error accumulation and path deviation in the prior art are solved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to an automatic driving method and device of a patrol robot. BACKGROUND

[0002] With the continuous progress of automation technology, patrol robots as an important intelligent device are widely used in security and inspection fields. One of the core functions is automatic driving, which can perform patrol tasks according to the predetermined path. The traditional automatic driving system of the patrol robot usually relies on built-in sensors and positioning algorithms to realize path planning and navigation. However, in actual application, due to the complexity and uncertainty of the environment, the existing automatic driving technology faces some challenges, especially in precise positioning and path tracking.

[0003] Currently, automatic driving technology relies on sensor data such as lidar, camera and GPS devices, combined with SLAM (Simultaneous Localization and Mapping) algorithm or other positioning methods for path planning and positioning. However, many existing solutions still have problems such as low positioning accuracy, path error accumulation and the influence of environmental changes on positioning. These problems are particularly evident when the patrol robot performs tasks, especially in complex environments or in the presence of obstructions, dynamic changes, etc. The existing system is prone to positioning deviation, which affects the task execution efficiency and safety of the patrol robot. SUMMARY

[0004] Therefore, the embodiments of the present application provide an automatic driving method and device of a patrol robot to solve the technical problem that the existing system is prone to positioning deviation, which affects the task execution efficiency and safety of the patrol robot.

[0005] The first aspect of the embodiments of the present application provides an automatic driving method of a patrol robot, which comprises: acquiring a plurality of pre-stored video frames collected by the patrol robot on a preset path, a plurality of feature object regions corresponding to the plurality of pre-stored video frames respectively, and feature positions corresponding to the feature object regions; wherein the feature positions refer to position data of the feature object regions in the image coordinate system in the pre-stored video frames; the feature object regions refer to image regions corresponding to markers placed in the actual environment; collecting real-time image data during the patrol robot traveling based on the preset path; extracting a plurality of real-time feature object regions in the real-time image data and image positions corresponding to the plurality of real-time feature object regions respectively; Correcting the current environment coordinate system based on the characteristic object regions corresponding to the plurality of pre-stored video frames, the characteristic positions corresponding to the characteristic object regions, the plurality of real-time characteristic object regions, and the image positions corresponding to the plurality of real-time characteristic object regions to obtain a target environment coordinate system; wherein the current environment coordinate system refers to the environment coordinate system used for the current positioning of the robot; Based on the target environment coordinate system and the preset path, the patrol robot is controlled to perform a patrol mission.

[0006] Furthermore, the step of correcting the current environment coordinate system according to the characteristic object areas corresponding to the plurality of pre-stored video frames, the characteristic positions corresponding to the characteristic object areas, the plurality of real-time characteristic object areas, and the image positions corresponding to the plurality of real-time characteristic object areas to obtain the corrected environment coordinate system includes: Extracting target pre-stored video frames corresponding to the multiple real-time feature object regions from the multiple pre-stored video frames according to the feature object regions corresponding to the multiple pre-stored video frames; respectively calculating the matching degrees between the feature positions corresponding to the plurality of feature object regions in the target pre-stored video frame and the image positions of the plurality of real-time feature object regions; When the matching degree is greater than a first threshold, extracting the preset environmental position coordinates corresponding to the target pre-stored video frame corresponding to the matching degree; wherein the preset environmental position coordinates refer to the position coordinates of the target pre-stored video frame collected by the patrol robot in a standard environmental coordinate system; Obtaining the real-time environmental position coordinates of the patrol robot; wherein the real-time environmental position coordinates refer to the position coordinates of the patrol robot collecting the real-time image data in the current environmental coordinate system; When the difference between the real-time environment position coordinates and the preset environment position coordinates is greater than a second threshold, the current environment coordinate system is adjusted according to the difference to obtain the corrected environment coordinate system.

[0007] Furthermore, the step of respectively calculating the matching degrees between the feature positions corresponding to the plurality of feature object regions in the target pre-stored video frame and the image positions of the plurality of real-time feature object regions includes: Respectively extracting the first number of pixels in the feature object area and the second number of pixels in the real-time feature object area corresponding to the same feature object; The matching degree is calculated according to the first number of pixels, the second number of pixels, the feature position, and the image position.

[0008] Furthermore, the step of calculating the matching degree according to the first number of pixels, the second number of pixels, the feature position, and the image position includes: calculate a first difference value between the first number of pixel points and the second number of pixel points corresponding to the same feature object; calculate a second difference value between the feature position and the image position corresponding to the same feature object; If any one of the first difference values corresponding to the plurality of feature objects is greater than a third threshold value or any one of the second difference values corresponding to the plurality of feature objects is greater than a fourth threshold value, determine that the matching degree is 0; If none of the first difference values is greater than the third threshold value or none of the second difference values is greater than the fourth threshold value, divide the first difference value by the first number of pixel points corresponding to the first difference value to obtain a difference proportion; take the average value corresponding to the plurality of difference proportions as a first difference value; multiply each of the second difference values by a preset difference coefficient to obtain an initial difference value; take the average value corresponding to the plurality of initial difference values as a second difference value; perform weighted summation on the first difference data and the second difference value to obtain the matching degree.

[0009] Further, the step of extracting a plurality of real-time feature object regions in the real-time image data and image positions corresponding to the plurality of real-time feature object regions respectively comprises: perform binaryzation processing on the real-time image data to obtain a binaryzation image; perform contour detection on the binaryzation image to obtain a plurality of closed contour regions; obtain a plurality of preset feature arrays corresponding to the plurality of feature object regions respectively; extract a target feature array corresponding to each of the plurality of closed contour regions; wherein the target feature array is used to describe the distribution relationship between a plurality of sector sub-regions in the closed contour region; calculate a first similarity between the preset feature array corresponding to each of the plurality of feature object regions and the target feature array respectively; when the first similarity is greater than a third threshold value, take the closed contour region corresponding to the first similarity as the real-time feature object region; take the image coordinates of the first center of the real-time feature object region as the image position.

[0010] Further, the step of obtaining a plurality of preset feature arrays corresponding to the plurality of feature object regions respectively comprises: extract the center of each of the closed contour regions; extracting a plurality of sector sub-regions in eight preset angle ranges based on the center in the closed contour region; wherein the eight preset angle ranges include 0-45 degrees, 45-90 degrees, 90-135 degrees, 135-180 degrees, 180-225 degrees, 225-270 degrees, 270-315 degrees, and 315-360 degrees; respectively counting the number of pixel points in the eight sector sub-regions; dividing the number of pixel points corresponding to each sector sub-region by the maximum number of pixel points to obtain a first feature value; arranging the eight first feature values in the order of the eight sector sub-regions to obtain a first feature array; extracting a midpoint on a pair of edges of the sector sub-region; wherein the pair of edges refers to edges opposite to each other in the sector sub-region; respectively calculating a first distance between the center and the midpoint corresponding to each of the eight sector sub-regions; dividing the first distance corresponding to each sector sub-region by the maximum first distance to obtain a second feature value; arranging the eight second feature values in the order of the eight sector sub-regions to obtain a second feature array; combining the first feature array and the second feature array to obtain a target feature array.

[0011] Further, when the first similarity is greater than a third threshold value, the step of taking the closed contour region corresponding to the first similarity as the real-time feature object region comprises: obtaining a plurality of preset pixel value distribution intervals when the first similarity is greater than a third threshold value; respectively counting the first pixel number in the closed contour region corresponding to the first similarity in the plurality of preset pixel value distribution intervals; respectively counting the second pixel number in the feature object region corresponding to the first similarity in the plurality of preset pixel value distribution intervals; normalizing the plurality of first pixel numbers to obtain a plurality of first values; normalizing the plurality of second pixel numbers to obtain a plurality of second values; arranging the first value corresponding to each of the plurality of preset pixel value distribution intervals in the order of the plurality of preset pixel value distribution intervals to obtain a third feature array; arranging the second value corresponding to each of the plurality of preset pixel value distribution intervals in the order of the plurality of preset pixel value distribution intervals to obtain a fourth feature array; calculating a second similarity between the third feature array and the fourth feature array; When the second similarity is greater than a fourth threshold value, a closed contour region corresponding to the second similarity is taken as the real-time feature object region.

[0012] The second aspect of the embodiment of the present application provides an automatic driving device of a patrol robot, which comprises: An acquisition unit is configured to acquire a plurality of pre-stored video frames collected by the patrol robot on a preset path, a feature object region corresponding to each of the pre-stored video frames, and a feature position corresponding to the feature object region; wherein the feature position refers to position data of the feature object region in the pre-stored video frames in an image coordinate system; and the feature object region refers to an image region corresponding to a marker placed in an actual environment; An acquisition unit is configured to acquire real-time image data collected by the patrol robot during travel based on the preset path; An extraction unit is configured to extract a plurality of real-time feature object regions in the real-time image data and an image position corresponding to each of the real-time feature object regions; A correction unit is configured to correct a current environment coordinate system according to the feature object region corresponding to each of the pre-stored video frames, the feature position corresponding to the feature object region, the plurality of real-time feature object regions, and the image position corresponding to each of the real-time feature object regions, to obtain a target environment coordinate system; wherein the current environment coordinate system refers to an environment coordinate system currently positioned by the robot; A control unit is configured to control the patrol robot to perform a patrol task based on the target environment coordinate system and the preset path.

[0013] The third aspect of the embodiment of the present application provides a terminal device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the automatic driving method of the patrol robot of the first aspect when executing the computer program.

[0014] The fourth aspect of the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps in the automatic driving method of the patrol robot of the first aspect when executed by a processor.

[0015] Compared with the prior art, the embodiment of the present application has the beneficial effects that: by comparing the feature object regions in the pre-stored video frames and the real-time image data during the patrol robot's travel, the current environment coordinate system is corrected. By precisely matching multiple real-time feature object regions with the landmark regions in the pre-stored video frames, the positioning accuracy of the patrol robot can be greatly improved. In a dynamically changing environment, this positioning correction can update the coordinate system in real time, reducing the positioning deviation caused by environmental changes and error accumulation. By using the landmarks placed in the actual environment as feature object regions, this method can adapt to various complex environmental conditions. Whether in a frequently changing environment or under the influence of factors such as occlusion and light changes, the patrol robot can autonomously correct its positioning through precise feature object recognition and image matching technology, ensuring that the robot can successfully complete the patrol task in the actual environment. Traditional autonomous driving systems often face the problem of path error accumulation and positioning drift, especially without external positioning assistance. By combining the feature positions of landmarks in pre-stored video frames with real-time image data for coordinate system correction, the present application effectively avoids the accumulation of path errors and positioning drift. Each correction adjusts the coordinate system based on the latest environmental data, keeping the patrol robot in an accurate positioning state at all times. By continuously updating and correcting the target environment coordinate system, the patrol robot can accurately follow the preset path. The accuracy of positioning directly affects the efficiency of task execution, and the technical solution of the present application can provide stable and reliable path tracking capability during task execution, thereby effectively improving the completion degree and efficiency of the patrol task and ensuring that the robot can efficiently complete the patrol work. In summary, the patrol robot autonomous driving method of the present application can significantly improve the positioning accuracy and autonomous driving performance of the robot in complex environments, overcoming the positioning error accumulation and path deviation problems in the prior art, enhancing the task execution ability and environmental adaptability of the robot, and providing reliable technical support for the wide application of patrol robots. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or related technical descriptions. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0017] Figure 1 A schematic flow chart of an automatic driving method of a patrol robot provided by the present application is shown; Figure 2 A schematic diagram of an automatic driving device of a patrol robot provided by an embodiment of the present application is shown; Figure 3 A schematic diagram of a terminal device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0018] In the following description, specific details such as particular system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

[0019] The embodiments of the present invention provide an automatic driving method and device for a patrol robot to solve the technical problem that existing systems are prone to positioning deviations, thereby affecting the task execution efficiency and safety of the patrol robot.

[0020] First, the present invention provides an automatic driving method for a patrol robot. Figure 1 , Figure 1 FIG1 shows a schematic flow chart of an automatic driving method of a patrol robot provided by the present invention. Figure 1 As shown, the automatic driving method of the patrol robot may include the following steps: Step 101: Acquire multiple pre-stored video frames captured by a patrol robot on a preset path, characteristic object regions corresponding to each of the multiple pre-stored video frames, and characteristic positions corresponding to the characteristic object regions; wherein the characteristic positions refer to position data of the characteristic object regions in the pre-stored video frames in an image coordinate system; and the characteristic object regions refer to image regions corresponding to markers placed in an actual environment; Multiple pre-stored video frames are used to construct a visual benchmark template database. These pre-stored video frames record the images and coordinate positions of "recognizable landmarks" distributed along the preset path in their normal state, serving as a reference standard for subsequent comparison and correction.

[0021] Feature object regions refer to the areas of landmarks identified in an image, including but not limited to QR codes, reflective stickers, and patterns. Feature positions refer to the locations of these regions in the image coordinate system.

[0022] Step 102: collecting real-time image data while the patrol robot is moving along the preset path; As the patrol robot travels along a pre-set path, it collects real-time image data that changes over time, reflecting the robot's situation in different locations and environments.

[0023] From the real-time image data, the robot extracts a plurality of real-time feature object regions. The real-time feature object region represents the region of the feature object in the current environment in the image.

[0024] Each real-time feature object region corresponds to an image position, i.e., the position of the feature object region in the current image coordinate system.

[0025] Step 103: Extracting a plurality of real-time feature object regions in the real-time image data and the image positions corresponding to the plurality of real-time feature object regions respectively; Specifically, step 103 specifically includes steps 1031 to 1037: Step 1031: performing binaryzation processing on the real-time image data to obtain a binaryzation image; The purpose of performing binaryzation processing on the real-time image data is to convert the pixel values in the image into two basic states (such as black and white, or 0 and 1) to facilitate subsequent processing. In this way, the information in the image can be simplified, and the key part (such as the feature object) in the image can be highlighted.

[0026] Using a thresholding method, the part of the image with a pixel value greater than a certain preset threshold is set to white (value 1), and the part below the threshold is set to black (value 0).

[0027] Step 1032: performing contour detection on the binaryzation image to obtain a plurality of closed contour regions; Performing contour detection on the binaryzation image is used to identify the edges and regions of different objects in the image. Through contour detection, closed object regions in the image can be extracted, which are candidate regions of the target feature object.

[0028] The contour detection method includes but is not limited to Canny edge detection or Hough transform, etc., which can identify and extract the closed edge region in the image.

[0029] Step 1033: obtaining a plurality of preset feature arrays corresponding to the plurality of feature object regions respectively; Obtaining a plurality of preset feature arrays corresponding to the plurality of feature object regions. The preset feature array is used to describe the distribution relationship between a plurality of sector sub-regions in the feature object region. The calculation logic of the preset feature array is consistent with the target feature array.

[0030] Step 1034: extracting a plurality of target feature arrays corresponding to the plurality of closed contour regions respectively; wherein the target feature array is used to describe the distribution relationship between a plurality of sector sub-regions in the closed contour region; Extract the target feature array corresponding to each closed contour region. This feature array is used to describe the distribution relationship between multiple sub-regions (such as fan-shaped regions) in the closed contour region, helping to determine whether the region is a feature object region.

[0031] By dividing and analyzing the closed contour region, the distribution features of the sub-regions inside the region are extracted.

[0032] Specifically, step 1034 specifically includes steps B1 to B10: Step B1: Extract the center of each closed contour region; Get the geometric center of the closed contour region. The center point is the average value of the positions of all pixel points in the region, representing the center of gravity of the region. The center of the region can be determined by calculating the average value of the coordinates of all pixels in the contour region.

[0033] Step B2: Based on the center in the closed contour region, extract eight fan-shaped sub-regions in the preset angle range; wherein the eight preset angle ranges include 0 to 45 degrees, 45 degrees to 90 degrees, 90 degrees to 135 degrees, 135 degrees to 180 degrees, 180 degrees to 225 degrees, 225 degrees to 270 degrees, 270 degrees to 315 degrees, and 315 degrees to 360 degrees; According to the center of the closed contour region, the region is divided into eight fan-shaped sub-regions. These fan-shaped regions are divided according to the preset angle range, from 0° to 360°, every 45° as an angle segment. These fan-shaped regions can be divided by a polar coordinate system, limiting the range of each fan-shaped region to the angle range of the center point, and further analyzing the pixels in the region.

[0034] It can be understood that the fan-shaped sub-region is surrounded by the edge of the closed contour region corresponding to the two radii and the preset angle range.

[0035] Step B3: Count the number of pixel points in each of the eight fan-shaped sub-regions; Count the number of pixel points in each fan-shaped region, which represents the density or filling degree of the region.

[0036] Step B4: Divide the number of pixel points corresponding to each fan-shaped sub-region by the maximum number of pixel points to obtain a first feature value; Calculate the relative density of each fan-shaped sub-region by normalizing the number of pixel points in the region with the maximum number of pixel points to obtain a standardized feature value.

[0037] Step B5: According to the order of the eight fan-shaped sub-regions, arrange the eight first feature values to obtain a first feature array; The first characteristic values of the eight sector regions are arranged in a preset order to obtain a one-dimensional characteristic array. The array contains the relative pixel density information of each sector sub-region in the region.

[0038] The first characteristic values of each sector sub-region are arranged in order to form an array, facilitating subsequent comparison and analysis.

[0039] Step B6: extracting the midpoint of the opposite side of the sector sub-region; wherein the opposite side refers to the edge opposite the sector sub-region angle; For each sector region, the midpoint of the opposite side is extracted. Here, the "opposite side" refers to the edge opposite the sector region angle (the edge of the closed contour region corresponding to the angle). The center point of the opposite side of the sector region is calculated through geometric analysis, which can usually be determined by calculating the average coordinates of each pixel on the edge.

[0040] Step B7: calculating the first distance between the center and the corresponding midpoint of each of the eight sector sub-regions, respectively; The distance between the midpoint of each sector sub-region and the center of the closed contour region is calculated to describe the geometric distribution of the sector region.

[0041] Step B8: dividing the first distance corresponding to each sector sub-region by the maximum first distance to obtain a second characteristic value; The characteristic of the relative position of each sector sub-region is calculated, and the distance value of each sector sub-region is normalized to make the distance between different regions comparable.

[0042] Step B9: arranging the eight second characteristic values in the order of the eight sector sub-regions to obtain a second characteristic array; The second characteristic values of the eight sector regions are arranged in order to form a one-dimensional characteristic array. The array contains the geometric distribution information of each sector sub-region relative to the center point.

[0043] Step B10: combining the first characteristic array and the second characteristic array to obtain a target characteristic array.

[0044] The first characteristic array (pixel density characteristic) and the second characteristic array (geometric distribution characteristic) are combined to obtain a complete target characteristic array. The array contains the density and position characteristics of each sector sub-region, which can more comprehensively describe the characteristics of the closed contour region.

[0045] In the corresponding embodiments of steps B1 to B10, the feature array describing the density of the region is obtained by dividing each region into a plurality of fan-shaped sub-regions and counting the number of pixel points in each sub-region. Then, the feature array describing the geometric distribution of the region is obtained by calculating the distance between the center of the region and the midpoint of each fan-shaped sub-region. Finally, the two feature arrays are combined into a complete target feature array, which is used for subsequent matching and analysis. This method ensures a fine-grained description of the feature object, which helps to improve the object recognition and positioning accuracy in the automatic driving process of the patrol robot.

[0046] Step 1035: Calculate the first similarity between the preset feature array corresponding to each of the plurality of feature object regions and the target feature array, respectively; By comparing the preset feature array of each feature object region with the target feature array of the region, the similarity between them is calculated. This similarity is used to determine whether the current closed contour region meets the standard of the target feature object region.

[0047] The similarity between the two feature arrays can be calculated using similarity measurement methods such as Euclidean distance, cosine similarity, Jaccard index, etc. When the similarity value is high, it indicates that the two regions may belong to the same type of object.

[0048] Step 1036: When the first similarity is greater than a third threshold, the closed contour region corresponding to the first similarity is taken as the real-time feature object region; According to the comparison of the calculated first similarity value and the preset third threshold, it is determined whether the closed contour region can be considered as a real-time feature object region.

[0049] If the similarity is greater than the set third threshold, it means that the matching degree between the current closed contour region and the target feature object region is high, and it can be considered as an effective feature object region.

[0050] The region is marked as a "real-time feature object region", which provides a basis for subsequent positioning and matching.

[0051] Specifically, step 1036 specifically includes steps C1 to C9: Step C1: When the first similarity is greater than a third threshold, a plurality of preset pixel value distribution intervals are obtained; Ensure that only when the first similarity reaches a predetermined threshold, subsequent processing is continued. This threshold judgment determines whether a specific closed contour region is further analyzed.

[0052] According to the third threshold value set by the system, the closed contour regions with sufficient similarity are screened out, so as to further process the pixel information of the regions. The preset pixel value distribution interval represents the pixel value range to be analyzed by the system, which is usually different intervals of image pixel values, such as brightness or color range.

[0053] Step C2: respectively counting the first pixel numbers in the closed contour regions corresponding to the first similarity and belonging to the multiple preset pixel value distribution intervals; The pixel values in each closed contour region are counted, and the number of pixels in each pixel value distribution interval is recorded.

[0054] All pixels in the closed contour region are traversed, and it is judged which pixel value distribution interval each pixel belongs to, and the number of pixels in each interval is counted.

[0055] Step C3: respectively counting the second pixel numbers in the feature object regions corresponding to the first similarity and belonging to the multiple preset pixel value distribution intervals; In the feature object region, the number of pixels in each preset pixel value distribution interval is counted. This step is used to compare the pixel distribution difference between the feature object region and the closed contour region.

[0056] All pixels in the feature object region are traversed, and the number of pixels in each pixel value distribution interval is counted.

[0057] Step C4: normalizing the multiple first pixel numbers to obtain multiple first values; The counted pixel numbers are normalized to ensure that the pixel numbers of different regions can be compared on the same scale.

[0058] The normalization processing is to divide each pixel number by the maximum pixel number, or to adjust the number to a unified range, such as [0, 1], by other standardization methods.

[0059] Step C5: normalizing the multiple second pixel numbers to obtain multiple second values; The counted pixel numbers are normalized to ensure that the pixel distribution of the feature object region and the closed contour region can be compared fairly.

[0060] Step C6: arranging the first values corresponding to the multiple preset pixel value distribution intervals in the order of the multiple preset pixel value distribution intervals to obtain a third feature array; The first values corresponding to each preset pixel value distribution interval are arranged in the order of the intervals to form a feature array. This feature array describes the normalization of the pixel value distribution in the closed contour region.

[0061] Step C7: Arrange the second values corresponding to each of the plurality of preset pixel value distribution intervals in order to obtain a fourth feature array. Generate a new array to describe the normalized distribution of pixel values in the feature object region.

[0062] Step C8: Calculate the second similarity between the third feature array and the fourth feature array. By calculating the similarity between the two feature arrays, it is evaluated whether the pixel distribution of the closed contour region and the feature object region is similar.

[0063] The similarity measurement method includes but is not limited to cosine similarity, Euclidean distance or Manhattan distance, etc. to calculate the similarity between the two arrays.

[0064] Step C9: When the second similarity is greater than a fourth threshold, the closed contour region corresponding to the second similarity is taken as the real-time feature object region.

[0065] If the similarity of the two feature arrays exceeds a set threshold (i.e. the fourth threshold), the closed contour region is considered as a real-time feature object region, which meets the specific feature.

[0066] According to the calculation result of the second similarity, it is judged whether it exceeds the predetermined threshold. If the condition is met, the region is marked as a real-time feature object region for subsequent automatic driving or patrol tasks.

[0067] In the embodiments corresponding to steps C1 to C9, by statistical processing, normalization processing and similarity calculation in a plurality of preset pixel value distribution intervals, the system can accurately determine which regions meet the specific feature, and select them as real-time feature object regions when the similarity threshold condition is met. This method enhances the perception and understanding ability of the patrol robot automatic driving system to the environment.

[0068] Step 1037: Take the image coordinates of the first center of the real-time feature object region as the image position.

[0069] The first center coordinates of the identified real-time feature object region are taken as the image position of the feature object region. The image position usually refers to the position coordinates of the object region in the image, which can be used for further processing or matching. The image coordinates are obtained by calculating the geometric center of the region (i.e. the average position of all pixels in the region).

[0070] In the embodiments corresponding to steps 1031 to 1037, candidate regions are extracted from the image through binarization and contour detection, then described and compared using feature arrays, and finally judged based on similarity to determine whether the region meets the standard of the feature object and determine the specific location of the feature object in the image. This process ensures that the patrol robot can accurately identify the key feature object region during autonomous driving and provide reliable basis for subsequent positioning and task execution.

[0071] Step 104: Correct the current environment coordinate system based on the feature object regions of the plurality of pre-stored video frames, the feature positions corresponding to the feature object regions, the plurality of real-time feature object regions, and the image positions corresponding to the plurality of real-time feature object regions, to obtain a target environment coordinate system; wherein the current environment coordinate system refers to the environment coordinate system adopted by the robot for current positioning; According to the feature object regions and their position data in the pre-stored video frames, combined with the real-time feature object regions and their image positions extracted from the real-time image data, the system performs coordinate system correction. The current environment coordinate system refers to the current position coordinate system obtained by the robot based on its sensors or positioning system, while the target environment coordinate system is a more accurate coordinate system reflecting the actual environment after correction.

[0072] Specifically, step 104 specifically includes steps 1041 to 1045: Step 1041: Extract the target pre-stored video frames corresponding to the plurality of real-time feature object regions from the plurality of pre-stored video frames based on the feature object regions corresponding to the plurality of pre-stored video frames; According to the corresponding feature object regions in the plurality of pre-stored video frames, extract the target pre-stored video frames corresponding to the plurality of real-time feature object regions from these video frames. The goal here is to identify historical video frames that match the feature objects captured in the current real-time image. By matching these historical data, the robot can maintain consistent positioning under different time and environmental conditions.

[0073] Step 1042: Calculate the matching degree between the feature positions corresponding to the plurality of feature object regions in the target pre-stored video frames and the image positions of the plurality of real-time feature object regions, respectively; Compare the image positions between the plurality of feature object regions in the extracted target pre-stored video frames and the real-time feature object regions, and calculate their matching degree. Matching degree is a measure of the alignment between images, meaning the degree of agreement between features in the current image and marked features in historical video frames.

[0074] Specifically, step 1042 specifically includes steps 10421 to 10422: Step 10421: Extract the first pixel point number of the feature object region corresponding to the same feature object and the second pixel point number of the real-time feature object region, respectively; For each feature object region in the target pre-stored video frame, the number of pixel points in the image region corresponding to the feature object region is extracted. This refers to the number of pixels occupied by the marked feature object region in the image.

[0075] Similarly, for the feature object region extracted from the real-time image data, the number of pixel points in the corresponding region is extracted. This process is the same as the extraction method of the first pixel point number.

[0076] Step 10422: Calculate the matching degree according to the first pixel point number, the second pixel point number, the feature position and the image position.

[0077] The first pixel point number and the second pixel point number provide a quantitative basis for indicating the coverage or size of the feature object region in the image. If the pixel point numbers of two regions are similar, it may mean that the positions or sizes of the two regions in the image are similar, thus having a higher matching degree.

[0078] By comparing the actual position of the feature object and the detected position in the image, the matching degree is evaluated. The smaller the error in position, the higher the matching degree.

[0079] The final result of the matching degree is obtained by considering the pixel point number, position data and the similarity between them. The specific matching degree calculation method is: The calculation of the matching degree is usually a numerical quantification process, and the result is a value between 0 and 1, where 1 represents perfect matching and 0 represents complete mismatch.

[0080] In the embodiments corresponding to steps 10421 to 1042, by extracting the pixel point number of the feature object region and the image position data, the matching degree is calculated by combining these information, so as to evaluate the similarity of the feature object region in the target pre-stored video frame and the real-time feature object region. By comparing these data, the patrol robot can accurately judge the matching degree between the real-time image data and the pre-stored video frame, thereby providing a basis for subsequent positioning and coordinate system correction.

[0081] Specifically, step 10422 specifically includes steps A1 to A8: Step A1: Calculate the first difference between the first pixel point number and the second pixel point number corresponding to the same feature object; The first difference refers to the difference between the number of first pixel points in the corresponding feature object region in the target pre-stored video frame and the number of second pixel points in the real-time image on the same feature object. This difference is used to measure the difference between the number of pixels in the two regions.

[0082] If the difference in the number of pixel points of the two feature regions is large, it may indicate that their scales or sizes in the image are different, which may result in a low matching degree.

[0083] Step A2: calculating a second difference between the feature position and the image position of the same feature object; The second difference is calculated between the feature position of the same feature object in the target pre-stored video frame and the image position in the real-time image. This difference is used to measure the spatial position deviation of the feature object in different images.

[0084] If the deviation of the two positions is large, it indicates that the position of the feature object may have changed, thereby affecting the matching degree.

[0085] Step A3: if any of the first differences corresponding to the plurality of feature objects is greater than a third threshold value or any of the second differences corresponding to the plurality of feature objects is greater than a fourth threshold value, determining that the matching degree is 0; If any of the first differences of the plurality of feature objects is greater than a pre-set third threshold value, or any of the second differences is greater than a pre-set fourth threshold value, it is determined that the matching degree is 0. This indicates that if there is a significant difference in the number of pixel points or a position deviation, the matching degree is too low and does not meet the requirements.

[0086] When the first difference or the second difference is greater than the threshold value, it indicates that the matching degree of the feature object is low and it may not be used for effective matching, so it is determined that the matching is invalid and the matching degree is 0.

[0087] Step A4: if none of the first differences is greater than the third threshold value or none of the second differences is greater than the fourth threshold value, dividing the first difference by the number of first pixel points corresponding to the first difference to obtain a difference ratio; If all the first differences of the feature objects are not greater than the third threshold value, or all the second differences are not greater than the fourth threshold value, the difference ratio is further calculated.

[0088] The difference ratio is obtained by dividing each first difference by the corresponding number of first pixel points. This ratio reflects the proportion of the difference in the number of pixel points relative to the size of the feature region, and a smaller difference ratio indicates a higher matching degree.

[0089] Step A5: taking the average value of the plurality of difference ratios as the first difference value; Step A6: multiply each of the second difference values by a preset difference coefficient to obtain an initial difference value of each feature object; Each second difference value is multiplied by a preset difference coefficient to obtain an initial difference value of each feature object. The difference coefficient can be adjusted according to experience or experimental data to control the influence of position difference on matching degree. Illustratively, the difference coefficient can be set to 0.68.

[0090] Step A7: taking the average value of the plurality of initial difference values as a second difference value; Step A8: weighted sum of the first difference data and the second difference value to obtain the matching degree.

[0091] Finally, the first difference value and the second difference value are weighted and summed. Through weighting, the system can adjust the contribution of different differences to the final matching degree according to their importance. The weighting coefficients can be defined by experiment or preset. Illustratively, the weighting coefficients are 0.6 and 0.4, and a higher weight (0.6) is assigned to the closed contour region (feature 1), meaning that this feature is considered more important to the final judgment. In contrast, a lower weight (0.4) is assigned to the feature object region (feature 2), indicating that this feature has less influence on the final judgment.

[0092] In the embodiments corresponding to steps A1 to A8, through multi-level difference evaluation (pixel point number difference, position difference) and weighted sum, the differences between different feature object regions are considered comprehensively, and a final matching degree is obtained. The steps finely adjust the calculation of matching degree by using threshold judgment, difference proportion, initial difference value calculation, etc., to ensure that only when the differences between feature objects are small, effective matching will be produced. This method can effectively improve the positioning accuracy of the patrol robot and make it more reliable in practical application.

[0093] Step 1043: when the matching degree is greater than a first threshold, extracting a preset environmental position coordinate corresponding to the target pre-stored video frame corresponding to the matching degree; wherein the preset environmental position coordinate refers to the position coordinate of the target pre-stored video frame collected by the patrol robot in the standard environmental coordinate system; When the matching degree is greater than a set first threshold, it means that the matching degree between the real-time image data and the target pre-stored video frame is high. At this time, the corresponding preset environmental position coordinate is extracted from the target pre-stored video frame, and the preset environmental position coordinate is the coordinate data of the video frame collected by the patrol robot in the standard environmental coordinate system.

[0094] These preset environmental position coordinates are calibrated based on a standard environmental coordinate system (such as a known global coordinate system), which helps to provide a reference framework for the positioning of the robot.

[0095] Step 1044: Obtain the real-time environmental position coordinate of the patrol robot; wherein the real-time environmental position coordinate refers to the position coordinate of the patrol robot collecting the real-time image data in the current environmental coordinate system; The real-time environmental position coordinate is obtained when the current robot collects real-time image data. The real-time environmental position coordinate refers to the position of the robot in the current environmental coordinate system adopted by the patrol robot. This coordinate system may have some deviation from the standard environmental coordinate system.

[0096] Step 1045: When the difference between the real-time environmental position coordinate and the preset environmental position coordinate is greater than a second threshold value, adjusting the current environmental coordinate system according to the difference to obtain the corrected environmental coordinate system.

[0097] After obtaining the real-time environmental position coordinate and the preset environmental position coordinate, the system calculates the difference between the two coordinates. If the difference is greater than the second threshold value, it means that the current environmental coordinate system has a large deviation and cannot accurately reflect the actual position of the robot.

[0098] In order to correct the positioning error, the current environmental coordinate system needs to be adjusted according to the difference, so as to obtain the corrected environmental coordinate system (i.e. the corrected environmental coordinate system). In this way, the patrol robot can correct its own position in real time and ensure high accuracy when performing tasks. According to the difference, the origin of the current environmental coordinate system is translated to align with the standard environmental coordinate system.

[0099] In the embodiments corresponding to steps 1041 to 1045, it is described in detail how to correct the current environmental coordinate system by comparing video frames and real-time images during the automatic driving of the robot. This method is based on feature matching of multiple images, accurately calculates and adjusts the position of the robot, solves the problem of robot positioning deviation, and thus improves the accuracy and safety of task execution.

[0100] Step 105: Based on the target environmental coordinate system and the preset path, control the patrol robot to perform a patrol task.

[0101] Once the target environmental coordinate system is determined, the patrol robot can perform a patrol task based on this accurate coordinate system and the preset path.

[0102] Through the corrected environmental coordinate system, the robot can more accurately know its position, avoiding task execution errors or safety problems caused by positioning deviation. The target environmental coordinate system provides an accurate mapping between the robot and the environment, allowing it to patrol stably along the preset path.

[0103] In the embodiments corresponding to steps 101 to 105, the current environment coordinate system is corrected by comparing the feature object regions in the pre-stored video frames and the real-time image data during the patrol robot's travel. By precisely matching multiple real-time feature object regions with the landmark regions in the pre-stored video frames, the positioning accuracy of the patrol robot can be greatly improved. In a dynamically changing environment, this positioning correction can update the coordinate system in real time, reducing the positioning deviation caused by environmental changes and error accumulation. By using the landmarks placed in the actual environment as feature object regions, this method can adapt to various complex environmental conditions. Whether in a frequently changing environment or under the influence of factors such as occlusion and light changes, the patrol robot can autonomously correct its positioning through precise feature object recognition and image matching technology, ensuring that the robot can successfully complete the patrol task in the actual environment. Traditional autonomous driving systems often face problems of path error accumulation and positioning drift, especially without external positioning assistance. By combining the feature positions of landmarks in pre-stored video frames with real-time image data for coordinate system correction, the present invention effectively avoids the accumulation of path errors and positioning drift. Each correction adjusts the coordinate system based on the latest environmental data, maintaining the patrol robot's accurate positioning at all times. By continuously updating and correcting the target environment coordinate system, the patrol robot can accurately follow the pre-set path. The accuracy of positioning directly affects the efficiency of task execution, and the technical solution of the present invention can provide stable and reliable path tracking capability during task execution, thereby effectively improving the completion degree and efficiency of the patrol task, ensuring that the robot can efficiently complete the patrol work. In summary, the patrol robot autonomous driving method of the present invention can significantly improve the positioning accuracy and autonomous driving performance of the robot in complex environments, overcoming the positioning error accumulation and path deviation problems in the prior art, enhancing the robot's task execution ability and environmental adaptability, and providing reliable technical support for the widespread application of patrol robots.

[0104] As Figure 2 The present invention provides an automatic driving device for a patrol robot, please see Figure 2 , Figure 2 A schematic diagram of an automatic driving device for a patrol robot provided by the present invention is shown, as Figure 2 An automatic driving device for a patrol robot provided by the present invention includes: An acquisition unit 21 is used to acquire a plurality of pre-stored video frames collected by a patrol robot on a pre-set path, a plurality of pre-stored video frames each corresponding to a feature object region, and a feature position corresponding to the feature object region; wherein the feature position refers to the position data of the feature object region in the image coordinate system in the pre-stored video frames; the feature object region refers to the image region corresponding to the landmark placed in the actual environment; The collection unit 22 is configured to collect real-time image data during the patrol robot moving based on the preset path; The extraction unit 23 is configured to extract a plurality of real-time feature object regions in the real-time image data and respective image positions of the plurality of real-time feature object regions; The correction unit 24 is configured to correct a current environment coordinate system according to the respective feature object regions of the plurality of pre-stored video frames, the respective feature positions of the feature object regions, the plurality of real-time feature object regions and the respective image positions of the plurality of real-time feature object regions, to obtain a target environment coordinate system; wherein the current environment coordinate system refers to an environment coordinate system adopted by the robot for current positioning; The control unit 25 is configured to control the patrol robot to perform a patrol task based on the target environment coordinate system and the preset path.

[0105] The present invention provides an autonomous driving device for a patrol robot. This device calibrates the current environmental coordinate system by comparing characteristic object regions in pre-stored video frames with real-time image data while the patrol robot is in motion. By accurately matching multiple real-time characteristic object regions with marker regions in pre-stored video frames, the patrol robot's positioning accuracy can be significantly improved. In dynamically changing environments, this positioning correction enables real-time coordinate system updates, reducing positioning deviations caused by environmental changes and error accumulation. By using markers placed in the actual environment as characteristic object regions, this method is adaptable to various complex environmental conditions. Whether in scenes with frequent environmental changes or under the influence of factors such as occlusion and varying lighting, the patrol robot can autonomously correct its positioning through precise characteristic object recognition and image matching technology, ensuring that the robot can successfully complete its patrol mission in the real environment. Conventional autonomous driving systems often face problems with path error accumulation and positioning drift, especially without external positioning assistance. By combining the characteristic positions of markers in pre-stored video frames with real-time image data to perform coordinate system correction, the present invention effectively avoids the problems of path error and positioning drift accumulation. Each calibration will adjust the coordinate system according to the latest environmental data, so that the patrol robot is always in an accurate positioning state. By continuously updating and correcting the target environment coordinate system, the patrol robot can accurately follow the preset path. The accuracy of positioning directly affects the efficiency of task execution. The technical solution of the present invention can provide stable and reliable path tracking capabilities during task execution, thereby effectively improving the completion and efficiency of patrol tasks, and ensuring that the robot can complete patrol work efficiently. In summary, the patrol robot automatic driving method of the present invention can significantly improve the positioning accuracy and automatic driving performance of the robot in complex environments, overcome the problems of positioning error accumulation and path deviation existing in the prior art, enhance the robot's task execution capability and environmental adaptability, and provide reliable technical guarantees for the widespread application of patrol robots.

[0106] Figure 3 FIG. 1 is a schematic diagram of a terminal device provided by an embodiment of the present invention. Figure 3 As shown, a terminal device 3 of this embodiment includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as an autonomous driving program for a patrol robot. When the processor 30 executes the computer program 32, the steps of each of the above-mentioned embodiments of the autonomous driving method for a patrol robot are implemented, such as Figure 1 Alternatively, when the processor 30 executes the computer program 32, the functions of each unit in the above-mentioned device embodiments are realized, for example, Figure 2 Function of the unit shown.

[0107] By way of example, the computer program 32 can be segmented into one or more units stored in the memory 31 and executed by the processor 30 to accomplish the present application. The one or more units can be a series of computer program instruction segments capable of accomplishing a specific function, which are used to describe the execution process of the computer program 32 in the terminal device 3. For example, the computer program 32 can be segmented into units with specific functions as follows: An acquisition unit is configured to acquire a plurality of pre-stored video frames collected by a patrol robot on a preset path, a plurality of feature object regions corresponding to the pre-stored video frames respectively, and feature positions corresponding to the feature object regions. The feature positions refer to position data of the feature object regions in the image coordinate system in the pre-stored video frames. The feature object regions refer to image regions corresponding to markers placed in an actual environment. An acquisition unit is configured to acquire real-time image data collected by the patrol robot during travel based on the preset path. An extraction unit is configured to extract a plurality of real-time feature object regions in the real-time image data and image positions corresponding to the real-time feature object regions respectively. A correction unit is configured to correct a current environment coordinate system according to the plurality of feature object regions corresponding to the pre-stored video frames respectively, the feature positions corresponding to the feature object regions, the plurality of real-time feature object regions, and the image positions corresponding to the real-time feature object regions respectively, to obtain a target environment coordinate system. The current environment coordinate system refers to an environment coordinate system adopted by the robot for current positioning. A control unit is configured to control the patrol robot to perform a patrol task based on the target environment coordinate system and the preset path.

[0108] The terminal device includes but is not limited to the processor 30 and the memory 31. Those skilled in the art can understand that, Figure 3 The terminal device 3 is only an example and does not constitute a limitation on the terminal device 3, which can include more or fewer components than shown, or combine certain components, or different components, for example, the terminal device can also include an input / output device, a network access device, a bus, etc.

[0109] The processor 30 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0110] The memory 31 can be an internal storage unit of the terminal device 3, for example, a hard disk or a memory of the terminal device 3. The memory 31 can also be an external storage device of the terminal device 3, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 31 can include both the internal storage unit and the external storage device of the terminal device 3. The memory 31 is used to store the computer program and other programs and data required by the roaming control device. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0111] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0112] It should be noted that the information interaction, execution process, etc. between the above devices / units, since based on the same concept as the method embodiments of the present application, the specific functions and the technical effects brought by it can be referred to the method embodiments part, and will not be repeated here.

[0113] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be realized in the form of hardware or software functional unit. In addition, the specific name of each functional unit and module is only for easy distinction, and does not limit the protection scope of the present application. The specific working process of the unit and module in the above system can refer to the corresponding process in the foregoing method embodiment, which will not be described here.

[0114] The embodiment of the present application further provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps in each method embodiment.

[0115] The embodiment of the present application provides a computer program product, when the computer program product is run on a mobile terminal, so that the mobile terminal executes to realize the steps in each method embodiment.

[0116] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the present application realizes all or part of the processes in the above-mentioned embodiment methods, which can be completed by a computer program instructing related hardware. The computer program can be stored in a computer readable storage medium, and the computer program can realize the steps in each method embodiment when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form, etc. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the photographing device / terminal equipment, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, such as U disk, mobile hard disk, magnetic disk or optical disk, etc.

[0117] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0118] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on specific application and design constraints of the technical solutions. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0119] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely schematic. The division of the modules or units is merely a logical function division. There can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the display or discussion of the coupling or direct coupling or communication connection between the units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0120] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, which can be located in one place or distributed on a plurality of network units.

[0121] It should be understood that when used in the specification and the appended claims of the present application, the term "comprising" indicates the presence of described features, integers, steps, operations, elements, and / or components, but does not exclude one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0122] It should also be understood that the term "and / or" used in the specification and the appended claims of the present application means any combination of one or more of the associated listed items and all possible combinations thereof, and includes these combinations.

[0123] As used in the specification and the appended claims of the present application, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to monitoring" depending on the context. Similarly, the phrase "if it is determined" or "if it is monitored [that a described condition or event] can be interpreted depending on the context as meaning "upon determining" or "in response to determining" or "upon monitoring [that a described condition or event]" or "in response to monitoring [that a described condition or event]".

[0124] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0125] References to "one embodiment" or "some embodiments" in the present specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present invention. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0126] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A patrol robot automatic driving method, characterized in that: The automatic driving method of the patrol robot includes: Acquire multiple pre-stored video frames captured by the patrol robot on a preset path, characteristic object regions corresponding to each of the multiple pre-stored video frames, and characteristic positions corresponding to the characteristic object regions; wherein the characteristic positions refer to position data of the characteristic object regions in the pre-stored video frames in an image coordinate system; and the characteristic object regions refer to image regions corresponding to markers placed in an actual environment; Collecting real-time image data while the patrol robot is moving along the preset path; Extracting a plurality of real-time feature object regions from the real-time image data and image positions corresponding to the plurality of real-time feature object regions; Correcting the current environment coordinate system based on the characteristic object regions corresponding to the plurality of pre-stored video frames, the characteristic positions corresponding to the characteristic object regions, the plurality of real-time characteristic object regions, and the image positions corresponding to the plurality of real-time characteristic object regions to obtain a target environment coordinate system; wherein the current environment coordinate system refers to the environment coordinate system used for the current positioning of the robot; Based on the target environment coordinate system and the preset path, the patrol robot is controlled to perform a patrol mission.

2. The automatic driving method of the patrol robot according to claim 1, characterized in that: The step of correcting the current environment coordinate system according to the characteristic object areas corresponding to the plurality of pre-stored video frames, the characteristic positions corresponding to the characteristic object areas, the plurality of real-time characteristic object areas, and the image positions corresponding to the plurality of real-time characteristic object areas to obtain the corrected environment coordinate system comprises: Extracting target pre-stored video frames corresponding to the multiple real-time feature object regions from the multiple pre-stored video frames according to the feature object regions corresponding to the multiple pre-stored video frames; respectively calculating the matching degrees between the feature positions corresponding to the plurality of feature object regions in the target pre-stored video frame and the image positions of the plurality of real-time feature object regions; When the matching degree is greater than a first threshold, extracting the preset environmental position coordinates corresponding to the target pre-stored video frame corresponding to the matching degree; wherein the preset environmental position coordinates refer to the position coordinates of the target pre-stored video frame collected by the patrol robot in a standard environmental coordinate system; Obtaining the real-time environmental position coordinates of the patrol robot; wherein the real-time environmental position coordinates refer to the position coordinates of the patrol robot collecting the real-time image data in the current environmental coordinate system; When the difference between the real-time environment position coordinates and the preset environment position coordinates is greater than a second threshold, the current environment coordinate system is adjusted according to the difference to obtain the corrected environment coordinate system.

3. The automatic driving method of the patrol robot according to claim 2, characterized in that: The step of respectively calculating the matching degrees between the feature positions corresponding to the multiple feature object regions in the target pre-stored video frame and the image positions of the multiple real-time feature object regions comprises: Respectively extracting the first number of pixels in the feature object area and the second number of pixels in the real-time feature object area corresponding to the same feature object; The matching degree is calculated according to the first number of pixels, the second number of pixels, the feature position, and the image position.

4. The automatic driving method of the patrol robot according to claim 3, characterized in that: The step of calculating the matching degree according to the first number of pixels, the second number of pixels, the feature position, and the image position includes: Calculate a first difference between the number of first pixels and the number of second pixels corresponding to the same feature object; Calculating a second difference between a feature position and an image position corresponding to the same feature object; If any one of the first differences corresponding to the plurality of feature objects is greater than a third threshold or any one of the second differences corresponding to the plurality of feature objects is greater than a fourth threshold, then determining that the matching degree is 0; If the plurality of first differences are not greater than the third threshold or the plurality of second differences are not greater than the fourth threshold, dividing the first difference by the number of first pixels corresponding to the first difference to obtain a difference ratio; The average value corresponding to the multiple difference proportions is taken as the first difference value; multiplying each of the second difference values ​​by a preset difference coefficient to obtain an initial difference value; taking an average value corresponding to the plurality of initial difference values ​​as a second difference value; The first difference data and the second difference value are weightedly summed to obtain the matching degree.

5. The automatic driving method of the patrol robot according to claim 1, characterized in that: The step of extracting a plurality of real-time feature object regions and image positions corresponding to the plurality of real-time feature object regions in the real-time image data comprises: Binarizing the real-time image data to obtain a binary image; Performing contour detection on the binary image to obtain a plurality of closed contour areas; Obtaining preset feature arrays corresponding to multiple feature object areas; Extracting target feature arrays corresponding to the multiple closed contour regions; wherein the target feature arrays are used to describe the distribution relationship between the multiple fan-shaped sub-regions in the closed contour region; respectively calculating first similarities between a preset feature array corresponding to each of the plurality of feature object regions and the target feature array; When the first similarity is greater than a third threshold, taking the closed contour area corresponding to the first similarity as the real-time feature object area; The image coordinates of the first center of the real-time feature object area are used as the image position.

6. The automatic driving method of the patrol robot according to claim 5, characterized in that: The step of obtaining the preset feature arrays corresponding to the plurality of feature object regions comprises: Extracting the center of each closed contour area; Extracting sector-shaped sub-regions within eight preset angle ranges based on the center of the closed contour area; wherein the eight preset angle ranges include 0 to 45 degrees, 45 to 90 degrees, 90 to 135 degrees, 135 to 180 degrees, 180 to 225 degrees, 225 to 270 degrees, 270 to 315 degrees, and 315 to 360 degrees; Counting the number of pixels in each of the eight sector-shaped sub-areas; Dividing the number of pixels corresponding to each of the sector-shaped sub-regions by the maximum number of pixels to obtain a first characteristic value; Arranging the eight first characteristic values ​​in the order of the eight sector-shaped sub-regions to obtain a first characteristic array; Extracting the midpoint of the opposite side of the sector-shaped sub-region; wherein the opposite side refers to the edge of the sector-shaped sub-region opposite to the included angle; Calculating first distances between the center and corresponding midpoints of the eight sector-shaped sub-regions respectively; Dividing the first distance corresponding to each of the sector-shaped sub-regions by the maximum first distance to obtain a second characteristic value; Arranging the eight second characteristic values ​​in the order of the eight sector-shaped sub-regions to obtain a second characteristic array; The first feature array and the second feature array are combined to obtain a target feature array.

7. The automatic driving method of the patrol robot according to claim 5, characterized in that: When the first similarity is greater than a third threshold, the step of using the closed contour area corresponding to the first similarity as the real-time feature object area includes: When the first similarity is greater than a third threshold, obtaining a plurality of preset pixel value distribution intervals; respectively obtaining by counting the number of first pixels in the closed contour area corresponding to the first similarity and being in the plurality of preset pixel value distribution intervals; respectively obtaining by counting the number of second pixels in the characteristic object region corresponding to the first similarity and being in the plurality of preset pixel value distribution intervals; Normalizing the plurality of first pixel quantities to obtain a plurality of first values; Normalizing the plurality of second pixel quantities to obtain a plurality of second values; Arranging the first values ​​corresponding to the plurality of preset pixel value distribution intervals in the order of the plurality of preset pixel value distribution intervals to obtain a third feature array; Arranging the second values ​​corresponding to the plurality of preset pixel value distribution intervals in the order of the plurality of preset pixel value distribution intervals to obtain a fourth feature array; Calculating a second similarity between the third feature array and the fourth feature array; When the second similarity is greater than a fourth threshold, the closed contour area corresponding to the second similarity is used as the real-time feature object area.

8. An automatic driving device for a patrol robot, characterized in that: The automatic driving device of the patrol robot includes: an acquisition unit, configured to acquire a plurality of pre-stored video frames captured by the patrol robot on a preset path, characteristic object regions corresponding to each of the plurality of pre-stored video frames, and characteristic positions corresponding to the characteristic object regions; wherein the characteristic positions refer to position data of the characteristic object regions in the pre-stored video frames in an image coordinate system; and the characteristic object regions refer to image regions corresponding to markers placed in an actual environment; A collection unit, configured to collect real-time image data while the patrol robot is moving along the preset path; An extraction unit, configured to extract a plurality of real-time feature object regions and image positions corresponding to the plurality of real-time feature object regions from the real-time image data; a correction unit, configured to correct a current environment coordinate system based on the characteristic object regions corresponding to the plurality of pre-stored video frames, characteristic positions corresponding to the characteristic object regions, the plurality of real-time characteristic object regions, and the image positions corresponding to the plurality of real-time characteristic object regions, to obtain a target environment coordinate system; wherein the current environment coordinate system refers to the environment coordinate system used for the current positioning of the robot; A control unit is used to control the patrol robot to perform a patrol mission based on the target environment coordinate system and the preset path.

9. A terminal device, characterized in that: The terminal device includes: a memory, a processor, and an autonomous driving program for a patrol robot stored in the memory and executable on the processor, wherein the autonomous driving program for the patrol robot is configured to implement the steps in the autonomous driving method for a patrol robot as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the automatic driving method of the patrol robot as claimed in any one of claims 1 to 7 are implemented.

Images