A method, system and medium for human tracking for mobile robots based on visual matching

Abstract

Embodiments of the present application provide a mobile robot human tracking method and system based on visual matching, and a medium. The method comprises: collecting a region video in real time, pre-processing the region video to obtain a plurality of single-frame images; performing background compensation on the plurality of single-frame images and performing cross-frame difference processing to obtain difference images; setting search conditions to traverse the difference images to obtain difference image characteristic values; comparing the difference image characteristic values with preset characteristic threshold values to obtain characteristic deviation rates; determining whether the characteristic deviation rates are greater than or equal to a preset deviation rate threshold value; if yes, eliminating image pixel points corresponding to the difference image characteristic values; and if no, storing the corresponding difference image characteristic values to a data set, inputting the data set into a preset position prediction model, outputting human position information, inputting the human position information into a preset following model to output following parameters, and dynamically moving according to the following parameters.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of human body tracking of mobile robots, in particular, to a human body tracking method and system of mobile robots based on visual matching and a medium. BACKGROUND

[0002] A robot is a machine device that automatically performs work. It can accept human command, run a pre-programmed program, or act according to the principles and guidelines formulated by artificial intelligence technology. Its task is to assist or replace human work, such as production, construction, or dangerous work. At present, most robots are not autonomous enough and often need human assistance to achieve human body following.

[0003] The existing mobile robots cannot output following parameters through a following model, dynamically move according to the following parameters, dynamically establish a following path by analyzing the positional relationship between the human body, obstacles and the robot in the area, and cannot realize intelligent tracking of the mobile robot.

[0004] In view of the above problems, an effective technical solution is urgently needed. SUMMARY

[0005] The purpose of the embodiments of the present application is to provide a human body tracking method and system of mobile robots based on visual matching and a medium, which can dynamically establish a following path by analyzing the positional relationship between the human body, obstacles and the robot in the area, and adjust the robot movement parameters in real time to realize intelligent tracking.

[0006] The embodiments of the present application also provide a human body tracking method of mobile robots based on visual matching, comprising:

[0007] Real-time acquisition of area video, pre-processing of the area video to obtain a plurality of single-frame images;

[0008] Background compensation is performed on the plurality of single-frame images, and cross-frame difference processing is performed to obtain difference images;

[0009] Set search conditions to traverse the difference images to obtain difference image feature values;

[0010] The difference image feature values are compared with preset feature threshold values to obtain a feature deviation rate;

[0011] It is judged whether the feature deviation rate is greater than or equal to a preset deviation rate threshold;

[0012] If it is greater than or equal to, the image pixel point corresponding to the difference image feature value is eliminated;

[0013] If less, the corresponding differential image feature value is stored in a data set, the data set is input into a preset position prediction model, and human body position information is output;

[0014] According to the human body position information, a following model is input to output a following parameter, and dynamic movement is performed according to the following parameter.

[0015] Optionally, in the mobile robot human body tracking method based on visual matching provided in the embodiments of the present application, the area video is collected in real time, and the area video is preprocessed to obtain a plurality of single-frame images; the method comprises:

[0016] The area video is collected, and frame segmentation is performed on the video to obtain a plurality of single-frame images;

[0017] The gray value of each pixel point of the single-frame image is obtained, and the gray value of the pixel point is subjected to difference calculation with a preset gray threshold value to obtain a gray difference value;

[0018] If the gray difference value is greater than or equal to a first threshold value and less than a second threshold value, it is determined that the corresponding pixel point is a background area, and is recorded in a background data set;

[0019] If the gray difference value is greater than or equal to the second threshold value, it is determined that the corresponding pixel point is a human body area, and is recorded in a target data set;

[0020] The human body contour is calculated according to the target data set, and human body representation information is generated.

[0021] Optionally, in the mobile robot human body tracking method based on visual matching provided in the embodiments of the present application, the human body contour is calculated according to the target data set, and the human body representation information is generated, which comprises:

[0022] The target data set is obtained, and the distribution density of the pixel points in the target data set is calculated;

[0023] A plurality of dense trajectories are extracted according to the distribution density of the pixel points to obtain a trajectory set;

[0024] The human body movement change data is calculated according to the target data set, and if the human body movement change data is greater than a preset movement threshold value, a distribution weight coefficient is generated;

[0025] The plurality of dense trajectories in the trajectory set are subjected to weight distribution according to the distribution weight coefficient;

[0026] The human body representation information is obtained according to the dense trajectories after weight distribution;

[0027] The human body representation information comprises one or a combination of two or more of human body movement speed information, human body movement direction information, human body movement acceleration information, and human body movement turning information.

[0028] Optionally, in the method for tracking human body of mobile robot based on visual matching, the image is subjected to background compensation and cross-frame difference processing to obtain a difference image, which comprises:

[0029] The two time-series of images are inputted and projected in horizontal and vertical directions to obtain two one-dimensional graphs;

[0030] The projection data of the two images are obtained according to the one-dimensional graphs and subjected to normalization processing to obtain one-dimensional waveform graphs;

[0031] The background compensation data are obtained by performing operation according to the one-dimensional waveform graphs;

[0032] The difference image is obtained by calculating the image difference result according to the background compensation data.

[0033] Optionally, in the method for tracking human body of mobile robot based on visual matching, if the difference value is less than the threshold value, the corresponding difference image feature value is stored in a data set, the data set is inputted into a preset position prediction model, and human body position information is outputted, which further comprises:

[0034] The obstacle distribution information is determined according to the background data set;

[0035] The human body position information and the robot position information are obtained, and a human body coordinate system and a robot coordinate system are respectively established;

[0036] The human body position coordinates are established through the human body coordinate system, and the robot position coordinates are established according to the robot coordinate system;

[0037] The human body position coordinates and the robot position coordinates are subjected to coordinate conversion to generate the human body position coordinates and the robot position coordinates in a world coordinate system;

[0038] The following path is established according to the human body position coordinates and the robot position coordinates in the world coordinate system;

[0039] The obstacle coordinate information is established, and the coordinate difference value is calculated according to the obstacle coordinate information and the following path;

[0040] When the coordinate difference value is greater than a preset threshold value, the following path is updated.

[0041] Optionally, in the method for tracking human body of mobile robot based on visual matching, the human body position information is inputted into a preset following model to output following parameters, and the dynamic movement is performed according to the following parameters, which comprises:

[0042] The distance between the human body position and the robot position is calculated according to the following path to obtain a following distance;

[0043] If the following distance is greater than the first threshold value and less than the second threshold value, a first compensation parameter is generated, and the robot movement parameter is corrected according to the first compensation parameter;

[0044] If the following distance is greater than the second threshold value, a second compensation parameter is generated, and the robot movement parameter is corrected according to the second compensation parameter;

[0045] The robot movement parameter includes movement speed, movement acceleration, movement deceleration, and movement displacement.

[0046] In a second aspect, the embodiments of the present application provide a mobile robot human tracking system based on visual matching, which comprises a memory and a processor, the memory comprising a program of a mobile robot human tracking method based on visual matching, and the program of the mobile robot human tracking method based on visual matching is executed by the processor to realize the following steps:

[0047] Real-time acquisition of area video, pre-processing of the area video, and obtaining a plurality of single frame images;

[0048] Background compensation and cross-frame difference processing of the plurality of single frame images to obtain difference images;

[0049] Setting search conditions to traverse the difference images to obtain difference image feature values;

[0050] Comparing the difference image feature values with preset feature threshold values to obtain feature deviation rates;

[0051] Judging whether the feature deviation rate is greater than or equal to a preset deviation rate threshold value;

[0052] If yes, the image pixel point corresponding to the difference image feature value is removed;

[0053] If no, the corresponding difference image feature value is stored in a data set, the data set is input into a preset position prediction model, and human position information is output;

[0054] According to the human position information, a following model is input to output a following parameter, and dynamic movement is performed according to the following parameter.

[0055] Optionally, in the mobile robot human tracking system based on visual matching, the real-time acquisition of area video, the pre-processing of the area video, and the obtaining of a plurality of single frame images comprise:

[0056] Acquisition of area video, frame division of the video, and obtaining of a plurality of single frame images;

[0057] The gray value of each pixel point of the single frame image is acquired, and the gray value of the pixel point is subtracted from a preset gray threshold to obtain a gray difference value;

[0058] If the gray difference value is greater than or equal to the first threshold and less than the second threshold, it is determined that the corresponding pixel point is a background region, and is recorded to a background data set;

[0059] If the gray difference value is greater than or equal to the second threshold, it is determined that the corresponding pixel point is a human body region, and is recorded to a target data set;

[0060] A human body contour is calculated according to the target data set, and human body representation information is generated.

[0061] Optionally, in the mobile robot human body tracking system based on visual matching provided in the embodiments of the present application, the image is subjected to background compensation and cross-frame difference processing to obtain a difference image, which comprises:

[0062] Two time-series images are inputted, and are projected in horizontal and vertical directions to obtain two one-dimensional curve graphs;

[0063] Projection data of the two images are acquired according to the one-dimensional curve graphs, and are subjected to normalization processing to obtain one-dimensional waveform graphs;

[0064] Background compensation data are obtained by performing operation according to the one-dimensional waveform graphs;

[0065] The image difference result is calculated according to the background compensation data to obtain the difference image.

[0066] In a third aspect, the embodiments of the present application further provide a computer readable storage medium, wherein the computer readable storage medium comprises a mobile robot human body tracking method program based on visual matching, and the mobile robot human body tracking method program based on visual matching is executed by a processor to implement the steps of the mobile robot human body tracking method based on visual matching as described in any one of the above.

[0067] From the above, the embodiment of the application provides a mobile robot human tracking method, system and medium based on visual matching, video in a region is collected in real time, the video in the region is preprocessed, and a plurality of single-frame images are obtained; background compensation is performed on the plurality of single-frame images, and cross-frame difference processing is performed, to obtain difference images; search conditions are set to traverse the difference images, to obtain difference image feature values; the difference image feature values are compared with preset feature threshold values, to obtain feature deviation rates; whether the feature deviation rates are greater than or equal to a preset deviation rate threshold value is judged; if yes, image pixel points corresponding to the difference image feature values are removed; and if no, the corresponding difference image feature values are stored in a data set, the data set is input into a preset position prediction model, and human position information is output; the following model is input according to the human position information, to output following parameters, and dynamic movement is performed according to the following parameters; a following path is dynamically established by analyzing the positional relationship among the human, the obstacle and the robot in the region, and the movement parameters of the robot are adjusted in real time, to realize intelligent tracking.

[0068] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0069] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation to the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0070] Figure 1 The flowchart of the mobile robot human tracking method based on visual matching provided by the embodiment of the present application;

[0071] Figure 2 The flowchart of generating human representation information according to the target data set of the mobile robot human tracking method based on visual matching provided by the embodiment of the present application;

[0072] Figure 3 The flowchart of generating human representation information according to the assigned weight coefficient of the mobile robot human tracking method based on visual matching provided by the embodiment of the present application;

[0073] Figure 4 The following path updating flowchart of the mobile robot human tracking method based on visual matching provided by the embodiment of the present application;

[0074] Figure 5 A structure schematic diagram of a mobile robot human tracking system based on visual matching provided by an embodiment of the present application. DETAILED DESCRIPTION

[0075] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0076] It should be noted that similar reference numerals and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it need not be further defined and explained in subsequent drawings. Meanwhile, in the description of the present application, the terms "first", "second", and the like are only used to distinguish description, and cannot be understood as indicating or implying relative importance.

[0077] Please refer to Figure 1 , Figure 1 is a flowchart of a mobile robot human tracking method based on visual matching in some embodiments of the present application. The mobile robot human tracking method based on visual matching is used in a terminal device, and the mobile robot human tracking method based on visual matching comprises the following steps:

[0078] S101, real-time acquisition of a region video, pre-processing of the region video, and obtaining a plurality of single-frame images;

[0079] S102, background compensation of the plurality of single-frame images, and cross-frame difference processing, and obtaining difference images;

[0080] S103, setting of search conditions, traversal of the difference images, and obtaining of difference image feature values;

[0081] S104, comparison of the difference image feature values with preset feature threshold values, and obtaining of feature deviation rates;

[0082] S105, judgment of whether the feature deviation rate is greater than or equal to a preset deviation rate threshold value;

[0083] If yes, the image pixel points corresponding to the difference image feature values are eliminated;

[0084] If less, the corresponding differential image feature value is stored to a data set, the data set is input to a preset position prediction model, and human position information is output;

[0085] S106, a following parameter is output according to the human position information input to a preset following model, and dynamic movement is performed according to the following parameter.

[0086] Please refer to Figure 2 , Figure 2 is a flowchart for generating human representation information according to a target data set in a mobile robot human tracking method based on visual matching in some embodiments of the present application. According to an embodiment of the present application, regional video is collected in real time, and pre-processing is performed on the regional video to obtain a plurality of single-frame images; including:

[0087] S201, collecting regional video, frame cutting is performed on the video, and a plurality of single-frame images are obtained;

[0088] S202, obtaining the gray value of each pixel point of the single-frame image, performing difference calculation on the gray value of the pixel point and a preset gray threshold value, and obtaining a gray difference value;

[0089] S203, if the gray difference value is greater than or equal to a first threshold value and less than a second threshold value, it is determined that the corresponding pixel point is a background region, and is recorded to a background data set;

[0090] S204, if the gray difference value is greater than or equal to the second threshold value, it is determined that the corresponding pixel point is a human body region, and is recorded to a target data set;

[0091] S205, calculating a human contour according to the target data set, and generating human representation information.

[0092] Please refer to Figure 3 , Figure 3 is a flowchart for generating human representation information according to an assigned weight coefficient in a mobile robot human tracking method based on visual matching in some embodiments of the present application. According to an embodiment of the present application, a human contour is calculated according to a target data set, and human representation information is generated, including:

[0093] S301, obtaining a target data set, and calculating the distribution density of pixel points in the target data set;

[0094] S302, extracting a plurality of dense trajectories according to the distribution density of the pixel points to obtain a trajectory set;

[0095] S303, calculating human movement change data according to the target data set, and if the human movement change data is greater than a preset movement threshold value, an assigned weight coefficient is generated;

[0096] S304, performing weight assignment on a plurality of dense trajectories in the trajectory set according to the assigned weight coefficient;

[0097] S305, obtaining human body representation information according to the dense trajectory after weight distribution;

[0098] The human body representation information includes one or a combination of two or more of human body moving speed information, human body moving direction information, human body moving acceleration information and human body turning information.

[0099] It should be noted that the center of mass of the human body or the joint position of the human body, the turning information of the human body can be calculated through the human body contour, and the human body state change information is represented through the center of mass of the human body, the joint position of the human body and the turning information of the human body, so as to provide a basis for effective following of the robot and improve the following accuracy.

[0100] According to the embodiment of the present application, the image is background compensated and cross-frame difference processing is performed to obtain a difference image, which includes:

[0101] The two time series of images are input, and are projected in the horizontal and vertical directions to obtain two one-dimensional curve graphs;

[0102] The projection data of the two images are obtained according to the one-dimensional curve graph, and normalization processing is performed to obtain a one-dimensional waveform graph;

[0103] The background compensation data is obtained by performing operation according to the one-dimensional waveform graph;

[0104] The image difference result is calculated according to the background compensation data to obtain the difference image.

[0105] Please refer to Figure 4 , Figure 4 is a follow-up path updating flowchart of a mobile robot human body tracking method based on visual matching in some embodiments of the present application. According to the embodiment of the present application, if < 0, the corresponding difference image feature value is stored in a data set, the data set is input into a preset position prediction model, and human body position information is output, and the method further includes:

[0106] S401, judging obstacle distribution information according to background data set;

[0107] S402, obtaining human body position information and robot position information, and respectively establishing a human body coordinate system and a robot coordinate system;

[0108] S403, establishing human body position coordinates through the human body coordinate system, and establishing robot position coordinates according to the robot coordinate system;

[0109] S404, performing coordinate conversion on the human body position coordinates and the robot position coordinates to generate human body position coordinates and robot position coordinates in a world coordinate system;

[0110] S405, establishing a following path according to the human body position coordinates in the world coordinate system and the robot position coordinates;

[0111] S406, establishing obstacle coordinate information, and performing coordinate difference calculation according to the obstacle coordinate information and the following path; when the coordinate difference is greater than a preset threshold, the following path is updated.

[0112] According to the embodiment of the present application, the following model is input into the preset following model to output the following parameters, and the dynamic movement is performed according to the following parameters, including:

[0113] The distance between the human body position and the robot position is calculated according to the following path, and the following distance is obtained;

[0114] If the following distance is greater than the first threshold and less than the second threshold, the first compensation parameter is generated, and the robot movement parameter is corrected according to the first compensation parameter;

[0115] If the following distance is greater than the second threshold, the second compensation parameter is generated, and the robot movement parameter is corrected according to the second compensation parameter;

[0116] The robot movement parameter includes movement speed, movement acceleration, movement deceleration and movement displacement; the first threshold is less than the second threshold.

[0117] It should be noted that setting the following distance can ensure that the moving robot is always within a safe range and will not collide with the human body while achieving intelligent following, thereby improving the following accuracy.

[0118] According to the embodiment of the present application, a plurality of single-frame images are background compensated and cross-frame difference processing is performed to obtain a difference image.

[0119] A cross-frame interval is set, a collection time window is set according to the cross-frame interval, and images are collected in different time sequences;

[0120] The images collected in different time sequences are compared to obtain pixel difference values;

[0121] It is judged whether the pixel difference value is within a preset pixel threshold, if yes, the cross-frame interval data is retained;

[0122] If not, the cross-frame interval is adjusted, and the collection time window is updated again.

[0123] It should be noted that by setting different collection time windows, invalid data in the image can be removed, data that best represents the image parameters can be retained, and the image processing effect can be improved.

[0124] Please refer to Figure 5 , Figure 5is a structural schematic diagram of a visual matching-based mobile robot human tracking system in some embodiments of the present application. In a second aspect, the embodiments of the present application provide a visual matching-based mobile robot human tracking system 5, which comprises a memory 51 and a processor 52, the memory 51 comprising a program of a visual matching-based mobile robot human tracking method, the program of the visual matching-based mobile robot human tracking method being executed by the processor to implement the following steps:

[0125] Real-time acquisition of area video, pre-processing of the area video to obtain a plurality of single-frame images;

[0126] Background compensation of the plurality of single-frame images and cross-frame difference processing to obtain difference images;

[0127] Setting search conditions to traverse the difference images to obtain difference image feature values;

[0128] Comparing the difference image feature values with preset feature threshold values to obtain feature deviation rates;

[0129] Judging whether the feature deviation rates are greater than or equal to a preset deviation rate threshold value;

[0130] If greater than or equal to, eliminating the image pixel points corresponding to the difference image feature values;

[0131] If less than, storing the corresponding difference image feature values to a data set, inputting the data set into a preset position prediction model, and outputting human position information;

[0132] Inputting the human position information into a preset following model to output following parameters, and dynamically moving according to the following parameters.

[0133] According to the embodiments of the present application, real-time acquisition of area video, pre-processing of the area video to obtain a plurality of single-frame images, comprising:

[0134] Acquisition of area video, frame division of the video to obtain a plurality of single-frame images;

[0135] Obtaining the gray value of each pixel point of the single-frame image, and performing difference calculation on the gray value of the pixel point and a preset gray threshold value to obtain a gray difference value;

[0136] If the gray difference value is greater than or equal to a first threshold value and less than a second threshold value, it is determined that the corresponding pixel point is a background area, and is recorded to a background data set;

[0137] If the gray difference value is greater than or equal to the second threshold value, it is determined that the corresponding pixel point is a human body area, and is recorded to a target data set;

[0138] Calculating the human body contour according to the target data set, and generating human body representation information.

[0139] According to the embodiment of the present application, the human body contour is calculated according to the target data set, and the human body representation information is generated, including:

[0140] The target data set is acquired, and the pixel point distribution density in the target data set is calculated;

[0141] According to the pixel point distribution density, a plurality of dense tracks are extracted to obtain a track set;

[0142] The human body movement change data is calculated according to the target data set, and if the human body movement change data is greater than a preset movement threshold, a distribution weight coefficient is generated;

[0143] According to the distribution weight coefficient, the plurality of dense tracks in the track set are distributed with weights;

[0144] According to the dense tracks after weight distribution, the human body representation information is obtained;

[0145] The human body representation information includes one or a combination of two or more of human body movement speed information, human body movement direction information, human body movement acceleration information, and human body turning information in the process of human body movement.

[0146] It should be noted that the human body centroid or human body joint position, human body turning information can be calculated by the human body contour, and the human body state change information is represented by the human body centroid, human body joint position and human body turning information, which provides a basis for effective following of the robot and improves the following accuracy.

[0147] According to the embodiment of the present application, the image is background compensated and cross-frame difference processing is performed to obtain a difference image, including:

[0148] Two time series of images are input, and are projected in horizontal and vertical directions to obtain two one-dimensional curve graphs;

[0149] According to the one-dimensional curve graph, the projection data of the two images is obtained, and normalized processing is performed to obtain a one-dimensional waveform graph;

[0150] The background compensation data is obtained by operating the one-dimensional waveform graph;

[0151] The difference image is obtained by calculating the image difference result according to the background compensation data.

[0152] According to the embodiment of the present application, if the difference image feature value is less than the threshold, the corresponding difference image feature value is stored in the data set, the data set is input into a preset position prediction model, and the human body position information is output, and further including:

[0153] Obtain the obstacle distribution information according to the background data set;

[0154] Obtain human position information and robot position information, and establish human coordinate system and robot coordinate system respectively;

[0155] Establish human position coordinates through the human coordinate system, and establish robot position coordinates according to the robot coordinate system;

[0156] Perform coordinate conversion on the human position coordinates and the robot position coordinates, to generate human position coordinates and robot position coordinates under a world coordinate system;

[0157] Establish a following path according to the human position coordinates and the robot position coordinates under the world coordinate system;

[0158] Establish obstacle coordinate information, and perform coordinate difference calculation on the obstacle coordinate information and the following path;

[0159] When the coordinate difference is greater than a preset threshold, the following path is updated.

[0160] According to the embodiment of the present application, a preset following model is input according to human position information, and a following parameter is output, and dynamic movement is performed according to the following parameter, including:

[0161] Calculate the distance between the human position and the robot position according to the following path, to obtain a following distance;

[0162] If the following distance is greater than a first threshold and less than a second threshold, a first compensation parameter is generated, and the robot movement parameter is corrected according to the first compensation parameter;

[0163] If the following distance is greater than the second threshold, a second compensation parameter is generated, and the robot movement parameter is corrected according to the second compensation parameter;

[0164] The robot movement parameter includes movement speed, movement acceleration, movement deceleration, and movement displacement; the first threshold is less than the second threshold.

[0165] It should be noted that setting the following distance can ensure that the moving robot is always within a safe range, and the intelligent following is realized without colliding with the human body, and the following precision is improved.

[0166] According to the embodiment of the present application, a plurality of single-frame images are subjected to background compensation, and cross-frame difference processing is performed, to obtain a difference image;

[0167] A cross-frame interval is set, a collection time window is set according to the cross-frame interval, and images are collected in different time sequences;

[0168] The images collected in different time sequences are compared, to obtain pixel difference values;

[0169] determining whether the pixel difference value is within a preset pixel threshold, if yes, retaining the cross-frame interval data;

[0170] if no, adjusting the cross-frame interval, and updating the acquisition time window again.

[0171] It should be noted that by setting different acquisition time windows, invalid data in the image can be eliminated, data that best expresses the image parameters can be retained, and the processing effect of the image can be improved.

[0172] The third aspect of the present application provides a computer readable storage medium, the readable storage medium comprising a mobile robot human body tracking method based on visual matching program, when the mobile robot human body tracking method based on visual matching program is executed by a processor, the steps of the mobile robot human body tracking method based on visual matching of any one of the above are realized.

[0173] The mobile robot human body tracking method, system and medium based on visual matching disclosed in the present application, by real-time acquisition of regional video, pre-processing the regional video to obtain a plurality of single frame images, background compensation is performed on the plurality of single frame images, and cross-frame difference processing is performed to obtain difference images, search conditions are set to traverse the difference images to obtain difference image characteristic values, the difference image characteristic values are compared with a preset characteristic threshold to obtain a characteristic deviation rate, whether the characteristic deviation rate is greater than or equal to a preset deviation rate threshold is judged, if yes, the image pixel points corresponding to the difference image characteristic values are eliminated, if no, the corresponding difference image characteristic values are stored in a data set, the data set is input into a preset position prediction model, and human body position information is output, the position prediction model is input into a preset following model according to the human body position information to output following parameters, and dynamic movement is performed according to the following parameters, the following path is dynamically established by analyzing the positional relationship among the human body, the obstacle and the robot in the region, and the intelligent tracking is realized by real-time adjustment of the robot movement parameters.

[0174] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative, for example, the division of units is only a logical function division, and actual implementation can have another division mode, such as: a plurality of units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed components can be through some interface, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0175] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0176] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0177] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0178] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A method of human tracking for a mobile robot based on visual matching, characterized by, The method comprises the following steps: real-time acquisition of area video, pre-processing of the area video, and obtaining a plurality of single-frame images; background compensation of the plurality of single-frame images, and cross-frame difference processing to obtain difference images; setting search conditions to traverse the difference images to obtain difference image feature values; comparing the difference image feature values with preset feature threshold values to obtain feature deviation rates; judging whether the feature deviation rates are greater than or equal to a preset deviation rate threshold value; if yes, eliminating the image pixel points corresponding to the difference image feature values; if no, storing the corresponding difference image feature values into a data set, inputting the data set into a preset position prediction model, and outputting human body position information; inputting the human body position information into a preset following model to output following parameters, and dynamically moving according to the following parameters; The method comprises the following steps: acquiring area video, frame division of the video, and obtaining a plurality of single-frame images; obtaining the gray value of each pixel point of the single-frame image, and performing difference calculation on the gray value of the pixel point and a preset gray threshold value to obtain a gray difference value; if the gray difference value is greater than or equal to a first threshold value and less than a second threshold value, the corresponding pixel point is determined to be a background area, and is recorded into a background data set; if the gray difference value is greater than or equal to the second threshold value, the corresponding pixel point is determined to be a human body area, and is recorded into a target data set; calculating the human body contour according to the target data set, and generating human body representation information; The method comprises the following steps: obtaining the target data set, and calculating the pixel point distribution density in the target data set; extracting a plurality of dense trajectories according to the pixel point distribution density to obtain a trajectory set; calculating human body movement change data according to the target data set, and generating an allocation weight coefficient if the human body movement change data is greater than a preset movement threshold value; weight allocation is performed on the plurality of dense trajectories in the trajectory set according to the allocation weight coefficient; obtaining human body representation information according to the dense trajectories after weight allocation; The human body representation information comprises one or a combination of two or more of human body movement speed information, human body movement direction information, human body movement acceleration information, and human body movement turning information. The method comprises the following steps: inputting two continuous time sequence images, projecting in the horizontal and vertical directions to obtain two one-dimensional curve graphs; obtaining projection data of the two images according to the one-dimensional curve graphs, and performing normalization processing to obtain a one-dimensional waveform graph; performing operation according to the one-dimensional waveform graph to obtain background compensation data; calculating image difference results according to the background compensation data to obtain difference images.

2. The vision matching-based mobile robot human tracking method according to claim 1, characterized by, The method further comprises the following steps: judging obstacle distribution information according to the background data set; obtaining human body position information and robot position information, and establishing a human body coordinate system and a robot coordinate system respectively; The human body position coordinates are established by a human body coordinate system, and robot position coordinates are established according to a robot coordinate system; The human body position coordinates and the robot position coordinates are converted to generate human body position coordinates and robot position coordinates in a world coordinate system; The human body position coordinates and the robot position coordinates in the world coordinate system are used to establish a following path; Obstacle coordinate information is established, and coordinate difference values are calculated according to the obstacle coordinate information and the following path; When the coordinate difference value is greater than a preset threshold value, the following path is updated.

3. The vision matching-based mobile robot human tracking method according to claim 2, characterized by, The following parameter is output by inputting the human body position information into a preset following model, and dynamic movement is performed according to the following parameter, including: The distance between the human body position and the robot position is calculated according to the following path to obtain a following distance; If the following distance is greater than a first threshold value and less than a second threshold value, a first compensation parameter is generated, and the robot movement parameter is corrected according to the first compensation parameter; If the following distance is greater than the second threshold value, a second compensation parameter is generated, and the robot movement parameter is corrected according to the second compensation parameter; The robot movement parameter includes movement speed, movement acceleration, movement deceleration, and movement displacement; and the first threshold value is less than the second threshold value.

4. A vision matching based mobile robot human tracking system for implementing the steps of the vision matching based mobile robot human tracking method according to any one of claims 1 to 3, characterized by The system includes a memory and a processor, the memory includes a program of a mobile robot human body tracking method based on visual matching, and the program of the mobile robot human body tracking method based on visual matching is executed by the processor to realize the following steps: Real-time acquisition of area video, pre-processing of the area video, and obtaining a plurality of single frame images; Background compensation is performed on the plurality of single frame images, and cross-frame difference processing is performed to obtain difference images; Search conditions are set to traverse the difference images to obtain difference image feature values; The difference image feature values are compared with preset feature threshold values to obtain feature deviation rates; It is judged whether the feature deviation rate is greater than or equal to a preset deviation rate threshold value; If it is greater than or equal to, the image pixel point corresponding to the difference image feature value is removed; If it is less than, the corresponding difference image feature value is stored in a data set, the data set is input into a preset position prediction model, and human body position information is output; The following parameter is output by inputting the human body position information into a preset following model, and dynamic movement is performed according to the following parameter.

5. A computer readable storage medium, characterized in that, The computer readable storage medium includes a program of a mobile robot human body tracking method based on visual matching, and the program of the mobile robot human body tracking method based on visual matching is executed by the processor to realize the steps of the mobile robot human body tracking method based on visual matching in any one of claims 1 to 3.

Images