Navigation method and system of household intelligent robot

Abstract

The invention relates to the technical field of robot navigation, and discloses a navigation method and system for a household intelligent robot, and the system comprises a visual collection module which processes a plurality of visual images to obtain a visual path image, and a visual judgment module which analyzes the visual path image and the visual path image collected last time to obtain a visual path image; a path correction point is generated for the advancing path based on an analysis result, a visual analysis module performs clustering sequence division on all path pixel points according to each path pixel point and a clustering center point when a suspected path point is recognized, a visual processing module compares a visual path image with a visual path image collected last time, and a path correction point is generated for the advancing path according to the comparison result. Determining a friction index of the advancing path according to a comparison result and a friction path model, judging whether the advancing path is abnormal or not based on the comprehensive navigation index, and determining whether to re-plan the advancing path or not according to a judgment result. The obstacle of the advancing path is avoided by sensing the ground change, and the navigation reliability is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of robot navigation, and in particular, to a navigation method and system for a household intelligent robot. Background Art

[0002] In daily life scenarios, household intelligent robots undertake various tasks such as cleaning and patrolling based on the pre-planned travel routes. However, during their movement, it is inevitable to encounter various emergencies, such as accidentally splashing water by humans, aging and bursting of household pipes, and tipping of containers, which cause liquid leakage and then form liquid accumulation on the ground. Currently, for the navigation and obstacle avoidance of robots, it mainly focuses on static obstacles such as tables, chairs, and cabinets, as well as dynamic obstacles such as walking people and running pets, lacking the perception of the ground conditions. When facing the liquid accumulation on the ground, it is difficult for the robot to identify it, resulting in the inability to effectively judge the travel path, thus affecting the stability and reliability of robot navigation.

[0003] Therefore, it is necessary to design a navigation method and system for a household intelligent robot to solve the problems existing in the current technology. Summary of the Invention

[0004] In view of this, the present invention proposes a navigation method and system for a household intelligent robot, aiming to solve the problem that the current navigation and obstacle avoidance lack the perception of the ground conditions, and when facing the liquid accumulation on the ground, it is difficult for the robot to identify it, resulting in the inability to effectively judge the travel path, thus affecting the stability and reliability of robot navigation.

[0005] On the one hand, the present invention proposes a navigation system for a household intelligent robot, including:

[0006] A visual acquisition module, configured to determine a travel path, collect images of the travel path based on a plurality of visual devices, determine a plurality of visual images, and process the plurality of visual images to obtain a visual path image;

[0007] A visual judgment module, configured to analyze the visual path image and the visual path image collected last time, and generate path correction points for the travel path based on the analysis result, where the path correction points include conforming path points, away-from-path points, and suspected path points;

[0008] A visual analysis module, configured to when the suspected path point is recognized, obtain all path pixel points of the visual path image, determine the clustering center point of the visual path image, divide all path pixel points into clustering sequences according to each path pixel point and the clustering center point, and determine a first navigation factor and a second navigation factor of the travel path according to the division result;

[0009] The visual processing module is configured to determine the navigation index of the travel path according to the first navigation factor and the second navigation factor, compare the visual path image with the previously captured visual path image, determine the friction index of the travel path according to the comparison result and the friction path model, determine the comprehensive navigation index of the travel path according to the navigation index and the friction index, judge whether there is an abnormality in the travel path based on the comprehensive navigation index, and determine whether to re-plan the travel path according to the judgment result.

[0010] Further, when processing a plurality of visual images to obtain a visual path image, it includes:

[0011] The visual acquisition module performs denoising processing on a plurality of visual images, extracts and matches the feature points of the denoised plurality of visual images by using a preset image algorithm, and determines the interconnection parameters between the plurality of visual images;

[0012] The preset image algorithm includes a first image algorithm and a second image algorithm. Detect the corner points of the denoised plurality of visual images according to the first image algorithm and use them as feature points;

[0013] Generate a binary descriptor for each feature point according to the second image algorithm, obtain the Hamming distance between each binary descriptor and all binary descriptors, and determine the interconnection parameters between the plurality of visual images based on the Hamming distance. The interconnection parameters include relative position and rotation relationship;

[0014] Register a plurality of visual images according to the interconnection parameters, merge the registered plurality of visual images based on a multi-band fusion algorithm to determine a merged image, and perform smooth stitching and color balance on the merged image to obtain the visual path image.

[0015] Further, when analyzing the visual path image and the previously captured visual path image and generating a path correction point for the travel path based on the analysis result, it includes:

[0016] The visual judgment module performs coincidence matching on the visual path image and the previously captured visual path image, and generates the path correction point for the travel path according to the coincidence matching result;

[0017] When the visual path image coincides with the previously captured visual path image, generate the conforming path point for the travel path and continue to move forward according to the travel path;

[0018] When the visual path image does not coincide with the previously captured visual path image, determine the away path point and the suspected path point according to the degree of coincidence;

[0019] When there is local overlap between the visual path image and the previously captured visual path image, suspected path points are generated for the travel path.

[0020] When there is no local overlap between the visual path image and the previously captured visual path image, away path points are generated for the travel path, and the travel path is re - determined.

[0021] Further, when dividing all path pixel points into clustering sequences according to each path pixel point and the clustering center point, and determining the first navigation factor and the second navigation factor of the travel path according to the division result, it includes:

[0022] The visual analysis module determines the clustering distance between each path pixel point and the clustering center point, and preset a first preset clustering distance and a second preset clustering distance, where the first preset clustering distance is less than the second preset clustering distance.

[0023] The clustering sequence includes a path first sequence, a path second sequence, and a path third sequence.

[0024] When the clustering distance is less than or equal to the first preset clustering distance, the path pixel point is divided into the path first sequence.

[0025] When the clustering distance is greater than the first preset clustering distance and less than or equal to the second preset clustering distance, the path pixel point is divided into the path second sequence.

[0026] When the clustering distance is greater than the second preset clustering distance, the path pixel point is divided into the path third sequence.

[0027] The first navigation factor and the second navigation factor of the travel path are determined according to the path first sequence and the path second sequence.

[0028] Further, when determining the first navigation factor and the second navigation factor of the travel path according to the path first sequence and the path second sequence, it includes:

[0029] The visual analysis module normalizes the path first sequence, determines the path pixel value of each path pixel point, calculates the first standard deviation of the normalized path first sequence, and extracts the first maximum path pixel value and the first minimum path pixel value of the normalized path first sequence.

[0030] The visual analysis module normalizes the second path sequence, determines the path pixel values of each path pixel point, calculates the second standard deviation of the second path sequence after normalization, and extracts the second maximum path pixel value and the second minimum path pixel value of the second path sequence after normalization;

[0031] Determine the first navigation factor and the second navigation factor of the traveling path according to the first standard deviation, the first maximum path pixel value, the first minimum path pixel value, the second standard deviation, the second maximum path pixel value, the second minimum path pixel value, the first path sequence after normalization, and the second path sequence after normalization.

[0032] Further, the navigation system of the household intelligent robot includes:

[0033] The visual analysis module determines the first navigation factor and the second navigation factor of the traveling path according to the following formula:

[0034]

[0035] Where G represents the first navigation factor, U represents the first standard deviation, n represents the number of path pixel values in the first path sequence after normalization, Pi represents the i-th path pixel value in the first path sequence after normalization, Pmin represents the first minimum path pixel value, Pmax represents the first maximum path pixel value, E represents the second navigation factor, T represents the second standard deviation, m represents the number of path pixel values in the second path sequence after normalization, Qj represents the j-th path pixel value in the second path sequence after normalization, Qmin represents the second minimum path pixel value, and Qmax represents the second maximum path pixel value.

[0036] Further, when comparing the visual path image with the previously acquired visual path image and determining the friction index of the traveling path according to the comparison result and the friction path model, it includes:

[0037] The visual processing module deletes the areas in the visual path image that have local overlap with the previously acquired visual path image, and determines the target visual path image according to the deletion result;

[0038] The visual processing module obtains a historical visual image dataset and divides the historical visual image dataset into a training set and a test set;

[0039] Obtain a pre-selected neural network model, iteratively train the neural network model according to the training set, test the iteratively trained neural network model according to the test set, determine the R-squared value of the training set and the R-squared value of the test set after each iterative training, and determine the friction path model according to the relationship between the R-squared value of the training set and the R-squared value of the test set after each iterative training;

[0040] Substitute the target visual path image into the friction path model to determine the friction index of the travel path.

[0041] Further, when determining the friction path model according to the relationship between the R-squared value of the training set and the R-squared value of the test set after each iterative training, it includes:

[0042] The visual processing module sets an R-squared threshold;

[0043] If the R-squared value of the training set and the R-squared value of the test set of the neural network model after the current iterative training do not reach the R-squared threshold, and the R-squared value of the training set is greater than the R-squared value of the test set, then add a regularization term to the neural network model after the current iterative training and continue the iterative training;

[0044] If the R-squared value of the training set and the R-squared value of the test set of the neural network model after the current iterative training do not reach the R-squared threshold, and the R-squared value of the training set is less than the R-squared value of the test set, then reduce the amplitude of the change of the neural network model after the current iterative training in the gradient direction and continue the iterative training;

[0045] If the R-squared value of the training set and the R-squared value of the test set of the neural network model after the current iterative training do not reach the R-squared threshold, and the R-squared value of the training set is equal to the R-squared value of the test set, then use grid search to adjust the hyperparameters of the neural network model after the current iterative training and continue the iterative training;

[0046] If the R-squared value of the training set and the R-squared value of the test set of the neural network model after the current iterative training both reach the R-squared threshold, stop the iterative training and use the neural network model after the current iterative training as the friction path model.

[0047] Further, when determining the comprehensive navigation index of the travel path according to the navigation index and the friction index, judging whether there is an abnormality in the travel path based on the comprehensive navigation index, and determining whether to re-plan the travel path according to the judgment result, it includes:

[0048] The comprehensive navigation index is the product value of the navigation index and the friction index, and the visual processing module sets a comprehensive navigation index threshold;

[0049] When the comprehensive navigation index is greater than or equal to the comprehensive navigation index threshold, it is determined that the travel path is abnormal, and the travel path is re-planned;

[0050] When the comprehensive navigation index is less than the comprehensive navigation index threshold, it is determined that the travel path is normal, and the robot continues to move forward according to the travel path.

[0051] Compared with the prior art, the beneficial effects of the present invention are as follows: By analyzing the visual path image and the visual path image collected last time, path correction points are generated, which can perceive the ground changes of the travel path in real time, so as to adjust the travel direction and travel path in time, avoid the error of path planning, improve the accuracy of robot navigation, divide the path pixel points of the visual path image into clustering sequences, and determine the first navigation factor and the second navigation factor, comprehensively consider the local characteristics of the travel path, provide detailed and accurate ground information, and improve the stability and reliability of navigation. The comprehensive navigation index is determined according to the navigation index and the friction index. When the robot is navigating, it can well cope with the complex and changeable ground conditions in the home environment. When the ground changes cause the travel path to be abnormal, corresponding judgments can be made, so that the robot maintains good navigation performance and environmental adaptability.

[0052] On the other hand, the present application also provides a navigation method for a home intelligent robot, which is applied to the navigation system of the above-mentioned home intelligent robot, and includes:

[0053] Determine the travel path, collect images of the travel path based on a plurality of visual devices, determine a plurality of visual images, and process the plurality of visual images to obtain a visual path image;

[0054] Analyze the visual path image and the visual path image collected last time, and generate path correction points for the travel path based on the analysis results. The path correction points include conforming path points, away path points, and suspected path points;

[0055] When the suspected path point is recognized, obtain all the path pixel points of the visual path image, determine the clustering center point of the visual path image, divide all the path pixel points into clustering sequences according to each path pixel point and the clustering center point, and determine the first navigation factor and the second navigation factor of the travel path according to the division result;

[0056] Determine the navigation index of the travel path according to the first navigation factor and the second navigation factor, compare the visual path image with the previously acquired visual path image, determine the friction index of the travel path according to the comparison result and the friction path model, determine the comprehensive navigation index of the travel path according to the navigation index and the friction index, judge whether there is an abnormality in the travel path based on the comprehensive navigation index, and determine whether to re-plan the travel path according to the judgment result.

[0057] It can be understood that the above-mentioned navigation method and system of a household intelligent robot have the same beneficial effects, which will not be elaborated here. Brief Description of the Drawings

[0058] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered as limiting the present invention. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0059] Figure 1 It is a functional block diagram of a navigation system of a household intelligent robot provided by an embodiment of the present invention;

[0060] Figure 2 It is a flowchart of a navigation method of a household intelligent robot provided by an embodiment of the present invention. Detailed Embodiments

[0061] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art. It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the drawings and in combination with the embodiments.

[0062] In some embodiments of the present application, refer to Figure 1 As shown, a navigation system of a household intelligent robot includes:

[0063] A visual acquisition module, configured to determine a travel path, acquire images of the travel path based on a plurality of visual devices, determine a plurality of visual images, and process the plurality of visual images to obtain a visual path image;

[0064] The visual judgment module is configured to analyze the visual path image and the previously acquired visual path image, generate path correction points for the traveling path based on the analysis result, and the path correction points include compliant path points, away-from-path points, and suspected path points;

[0065] The visual analysis module is configured to, when a suspected path point is recognized, obtain all path pixel points of the visual path image, determine the clustering center point of the visual path image, perform clustering sequence division on all path pixel points according to each path pixel point and the clustering center point, and determine the first navigation factor and the second navigation factor of the traveling path according to the division result;

[0066] The visual processing module is configured to determine the navigation index of the traveling path according to the first navigation factor and the second navigation factor, compare the visual path image with the previously acquired visual path image, determine the friction index of the traveling path according to the comparison result and the friction path model, determine the comprehensive navigation index of the traveling path according to the navigation index and the friction index, judge whether there is an abnormality in the traveling path based on the comprehensive navigation index, and determine whether to re-plan the traveling path according to the judgment result.

[0067] Specifically, the household intelligent robot is equipped with several visual devices, preferably 5 visual devices, to ensure comprehensive collection of information about the surrounding environment. The visual acquisition module captures images of the travel path, thereby obtaining several visual images (the number of which is determined by the visual devices, here it is 5). The path information of the visual images is extracted through image processing technology, and finally a visual path image is formed, providing a reliable data basis for the subsequent visual judgment module. The visual judgment module is responsible for comparing the currently obtained visual path image with the previously acquired visual path image. By comparing and identifying the image features of the visual path image, corresponding path correction points (conforming path points, away-from-path points, and suspected path points) are generated on the travel path to determine whether there are certain changes in the travel path. The changes include situations such as temporary watering and liquid splashing. When a conforming path point is recognized, it means that the robot can continue to move forward on the travel path. When a suspected path point is recognized, further analysis is required later. When an away-from-path point is recognized, it indicates that there is a large area of water accumulation or other situations on the current travel path, so the travel path needs to be re-determined. If there is no previously acquired visual path image (when the household intelligent robot is just powered on), the visual path image of the initial travel path determined by the robot according to the path algorithm is saved as a reference standard for the next judgment. The dynamic adjustment of the robot's travel path based on the path correction points ensures the reliability of the robot's travel. When the visual judgment module recognizes a suspected path point, the visual analysis module obtains all the path pixel points and calculates the clustering center point of the visual path image through the clustering algorithm (K-means). Based on the distance from each path pixel point to the clustering center point, all the path pixel points are divided into clustering sequences, and the first navigation factor and the second navigation factor are calculated according to the division results. The path information of the travel path is refined through the clustering algorithm to ensure that the robot can recognize complex ground conditions, thereby improving the navigation accuracy. The visual processing module calculates the navigation index based on the first navigation factor and the second navigation factor to measure the feasibility of the current path. The navigation index is the product value of the first navigation factor and the second navigation factor. At the same time, the friction index of the travel path is determined based on the friction path model, comprehensively considering the influence of the ground material (such as wooden floor, carpet, etc.) on the robot's travel. The comprehensive navigation index is calculated by combining the navigation index and the friction index, and then the abnormality of the travel path is judged, improving the stability and intelligence of the robot's navigation.

[0068] It can be understood that through visual acquisition, image analysis, and calculation of the comprehensive navigation index, the robot can accurately identify the dynamic changes of the travel path and make dynamic adjustments according to the characteristic information of the travel path, avoiding navigation errors caused by increased ground complexity and ensuring the reliability of the robot's stable travel in different ground environments.

[0069] In some embodiments of the present application, when processing a plurality of visual images to obtain a visual path image, it includes: the visual acquisition module denoises the plurality of visual images, extracts and matches the feature points of the denoised plurality of visual images using a preset image algorithm, determines the interconnection parameters between the plurality of visual images. The preset image algorithm includes a first image algorithm and a second image algorithm. Detect the corner points of the denoised plurality of visual images according to the first image algorithm and use them as feature points. Generate a binary descriptor for each feature point according to the second image algorithm, and obtain the Hamming distance between each binary descriptor and all binary descriptors. Based on the Hamming distance, determine the interconnection parameters between the plurality of visual images. The interconnection parameters include relative position and rotation relationship. Register the plurality of visual images according to the interconnection parameters, merge the registered plurality of visual images based on the multi-band fusion algorithm, determine the merged image, and perform smooth stitching and color balance on the merged image to obtain the visual path image.

[0070] Specifically, first denoise a plurality of visual images. Since noise will interfere with the subsequent extraction and matching of feature points and reduce the accuracy of the results, through denoising, the accuracy and stability of subsequent feature point extraction from images are improved. The preset image algorithm is the ORB algorithm. The first image algorithm represents the FAST algorithm, and the second image algorithm represents the BRIEF algorithm. The ORB algorithm integrates the FAST algorithm and the BRIEF algorithm. The first image algorithm finds corner points by comparing the gray values of the central pixel and the surrounding pixels. For example, taking a certain pixel as the center and taking 16 surrounding pixel points, if there are consecutive N pixel points whose gray values are higher or lower than the gray value of the central pixel, then determine that pixel point as a corner point. The second image algorithm selects several pairs of pixel points in the neighborhood of the feature point to compare the magnitude of their gray values, arranges the comparison results in order to form a binary string, which is used as the binary descriptor of the feature point. Calculate the Hamming distance between each binary descriptor and all binary descriptors. The Hamming distance is obtained by performing an exclusive OR operation on two binary strings and counting the number of "1"s in the result. The process of calculating the Hamming distance is lengthy and mature, and will not be described in detail here.

[0071] It can be understood that based on the Hamming distance, the RANSAC algorithm can be used to determine the interconnection parameters of multiple visual images. According to the interconnection parameters, affine transformation, perspective transformation, etc. are used to register multiple visual images to eliminate factors such as translation, rotation, and deformation caused by different shooting angles between visual images, so as to achieve precise alignment between images. And merge the registered multiple visual images according to the multi-band fusion algorithm to improve the overall effect of the merged image. Perform smooth stitching and color balance on the merged image to ensure color consistency and brightness balance, and improve the image quality of the visual path image.

[0072] In some embodiments of the present application, when analyzing the visual path image and the previously captured visual path image and generating path correction points for the traveling path based on the analysis results, it includes: The visual judgment module performs coincidence matching on the visual path image and the previously captured visual path image, and generates path correction points for the traveling path according to the coincidence matching result. When the visual path image coincides with the previously captured visual path image, path compliance points are generated for the traveling path, and the robot continues to move forward along the traveling path. When the visual path image does not coincide with the previously captured visual path image, the away path points and suspected path points are determined according to the degree of coincidence. When there is partial coincidence between the visual path image and the previously captured visual path image, suspected path points are generated for the traveling path. When there is no partial coincidence between the visual path image and the previously captured visual path image, away path points are generated for the traveling path, and the traveling path is re - determined.

[0073] Specifically, by performing coincidence matching between the visual path image and the previously captured visual path image, the changes existing in the traveling path are accurately identified, and path correction points are generated according to the coincidence situation to dynamically adjust the traveling path of the robot, improving the reliability and intelligence level of the robot navigation. When the visual path image coincides with the previously captured visual path image, it indicates that the ground environment of the traveling path is the same as that of the previous traveling path, so path compliance points are generated and there is no need to adjust, and the robot can continue to move forward along the traveling path. When the visual path image does not coincide with the previously captured visual path image, further analysis is required to determine whether there is partial coincidence. When there is partial coincidence, it means that some changes have occurred in the ground environment of the traveling path, but there are still some ground environments that are the same as those of the previous traveling path, so further analysis is needed to generate suspected path points. When there is no partial coincidence, it indicates that the ground environment of the traveling path is completely different from that of the previous traveling path, so away path points are generated and the traveling path is re - determined to avoid the changes in the ground environment, ensuring the safety of the robot navigation. According to the obtained visual path image, the situation of the traveling path is automatically judged and corresponding decisions are made, improving the autonomy and intelligence level of the robot navigation.

[0074] In some embodiments of the present application, when performing clustering sequence division on all path pixel points according to each path pixel point and the clustering center point, and determining the first navigation factor and the second navigation factor of the traveling path based on the division result, it includes: the visual analysis module determines the clustering distance between each path pixel point and the clustering center point, a first preset clustering distance and a second preset clustering distance are preset in advance, the first preset clustering distance is less than the second preset clustering distance, the clustering sequence includes a first path sequence, a second path sequence, and a third path sequence. When the clustering distance is less than or equal to the first preset clustering distance, the path pixel point is divided into the first path sequence; when the clustering distance is greater than the first preset clustering distance and less than or equal to the second preset clustering distance, the path pixel point is divided into the second path sequence; when the clustering distance is greater than the second preset clustering distance, the path pixel point is divided into the third path sequence. The first navigation factor and the second navigation factor of the traveling path are determined according to the first path sequence and the second path sequence.

[0075] Specifically, for the determination of the clustering center point, one of the K-means algorithm, hierarchical clustering algorithm, and density clustering algorithm is used, which can be selected according to actual needs without specific limitation. Moreover, the method for determining the clustering distance is long and mature, so it will not be described in detail here. The first preset clustering distance and the second preset clustering distance are flexibly set according to the determined clustering distance, which is not specifically limited in this embodiment. By dynamically dividing the path pixel points into the first path sequence, the second path sequence, and the third path sequence, it lays a foundation for subsequent determination of the first navigation factor and the second navigation factor of the traveling path. Since there is partial overlap between the visual path image and the previously acquired visual path image, and when the clustering distance of the path pixel points is greater than the second preset clustering distance, it indicates that these path pixel points are relatively scattered, and the presented ground environment is divergent. For example, some scattered water droplets far from the ground water accumulation. During the actual navigation of the robot, these divergent situations are not necessary to be considered. Therefore, only the first path sequence and the second path sequence are used to determine the first navigation factor and the second navigation factor of the traveling path.

[0076] In some embodiments of the present application, when determining the first navigation factor and the second navigation factor of the travel path according to the first path sequence and the second path sequence, it includes: the visual analysis module normalizes the first path sequence, determines the path pixel value of each path pixel point, calculates the first standard deviation of the first path sequence after normalization, extracts the first maximum path pixel value and the first minimum path pixel value of the first path sequence after normalization, the visual analysis module normalizes the second path sequence, determines the path pixel value of each path pixel point, calculates the second standard deviation of the second path sequence after normalization, extracts the second maximum path pixel value and the second minimum path pixel value of the second path sequence after normalization, and determines the first navigation factor and the second navigation factor of the travel path according to the first standard deviation, the first maximum path pixel value, the first minimum path pixel value, the second standard deviation, the second maximum path pixel value, the second minimum path pixel value, the first path sequence after normalization, and the second path sequence after normalization.

[0077] In some embodiments of the present application, the navigation system of the household intelligent robot includes: the visual analysis module determines the first navigation factor and the second navigation factor of the travel path according to the following formula:

[0078]

[0079] Wherein, G represents the first navigation factor, U represents the first standard deviation, n represents the number of path pixel values in the first path sequence after normalization, Pi represents the i-th path pixel value in the first path sequence after normalization, Pmin represents the first minimum path pixel value, Pmax represents the first maximum path pixel value, E represents the second navigation factor, T represents the second standard deviation, m represents the number of path pixel values in the second path sequence after normalization, Qj represents the j-th path pixel value in the second path sequence after normalization, Qmin represents the second minimum path pixel value, and Qmax represents the second maximum path pixel value.

[0080] Specifically, normalizing the first path sequence and the second path sequence converts numerical values of different magnitudes to a unified scale, eliminating the influence of the dimension and value range of the numerical values themselves, making different path sequences comparable, providing a basis for calculating the first navigation factor and the second navigation factor, and the navigation index is the product value of the first navigation factor and the second navigation factor.

[0081] In some embodiments of the present application, when comparing the visual path image with the previously captured visual path image and determining the friction index of the travel path according to the comparison result and the friction path model, the following steps are included: The visual processing module deletes the regions in the visual path image that have partial overlap with the previously captured visual path image, determines the target visual path image according to the deletion result. The visual processing module obtains the historical visual image dataset, divides the historical visual image dataset into a training set and a test set, obtains a pre-selected neural network model, iteratively trains the neural network model according to the training set, tests the iteratively trained neural network model according to the test set, determines the R-squared value of the training set and the R-squared value of the test set after each iterative training, determines the friction path model according to the relationship between the R-squared value of the training set and the R-squared value of the test set after each iterative training, and substitutes the target visual path image into the friction path model to determine the friction index of the travel path.

[0082] Specifically, the regions in the visual path image that have partial overlap with the previously captured visual path image are deleted to remove the duplicate information in the visual path image, and only the difference information from the previously captured visual path image is retained. The difference information represents the ground differences of the travel path. For example, if there is ground water accumulation that was not present in the previously captured visual path image, it is separated and processed separately, avoiding the waste of computing resources and the extension of processing time caused by redundant data. The historical visual image dataset includes various liquid spraying situations that occurred in the travel path, as well as the friction coefficients corresponding to the robot. For example, the friction coefficient between the wheels (synthetic rubber) of the robot and the waterlogged road surface. The historical visual image dataset is divided into a training set and a test set, and 60% - 70% of the data is used as the training set, and the rest is used as the test set. Ensure that both the training set and the test set contain a variety of data. The training set is used to train the neural network model, while the test set is used to test the performance of the trained model. Using these data to train and test the neural network model improves the generalization ability and accuracy of the model. A neural network model is pre-selected as the initial model. The model contains multiple layers, different types of neurons, activation functions and other parameters, and can capture the complex relationships in the data. The data in the training set is used to iteratively train the neural network model. During each iteration, the model attempts to learn the patterns and relationships in the data to improve its prediction or classification ability.

[0083] It can be understood that the R-squared value of the training set refers to the R-squared value of the model on the training set after each iterative training, while the R-squared value of the test set refers to the R-squared value of the model on the test set after each iterative training. During each iteration, the stability of the model is measured by comparing the relationship between the R-squared value of the training set and the R-squared value of the test set, ensuring that the model has good fitting ability and generalization performance. It avoids the problems of inaccuracy or overfitting caused by blindly using the model, enabling the finally determined friction path model to accurately output the friction index of the travel path, where the friction index refers to the friction coefficient between the robot and the ground of the travel path. Through the training and testing of the model, the intelligence and automation of the system are improved, laying a data foundation for subsequent determination of the comprehensive navigation index.

[0084] In some embodiments of the present application, when determining the friction path model according to the relationship between the R-squared value of the training set and the R-squared value of the test set after each iterative training, it includes: the visual processing module sets the R-squared threshold. If the R-squared value of the training set and the R-squared value of the test set of the neural network model after the current iterative training do not reach the R-squared threshold, and the R-squared value of the training set is greater than the R-squared value of the test set, then a regularization term is added to the neural network model after the current iterative training, and iterative training continues. If the R-squared value of the training set and the R-squared value of the test set of the neural network model after the current iterative training do not reach the R-squared threshold, and the R-squared value of the training set is less than the R-squared value of the test set, then the amplitude of the change of the neural network model after the current iterative training in the gradient direction is reduced, and iterative training continues. If the R-squared value of the training set and the R-squared value of the test set of the neural network model after the current iterative training do not reach the R-squared threshold, and the R-squared value of the training set is equal to the R-squared value of the test set, then grid search is used to adjust the hyperparameters of the neural network model after the current iterative training, and iterative training continues. If the R-squared value of the training set and the R-squared value of the test set of the neural network model after the current iterative training both reach the R-squared threshold, then iterative training is stopped, and the neural network model after the current iterative training is used as the friction path model.

[0085] Specifically, the R-squared threshold is preferably 0.85. When the R-squared value of the training set is greater than that of the test set and neither reaches the R-squared threshold, it indicates that the model over-learns the specific patterns in the training set data and cannot generalize to new data. Then, a regularization term is added. By restricting the complexity of the model, the regularization term prevents the model from overfitting the patterns and details in the training set data, thereby enhancing the generalization ability of the model. As a result, when facing new unseen data, the model can accurately output the friction index. When the R-squared value of the training set is less than that of the test set and neither reaches the R-squared threshold, it indicates that the fitting effect of the model on the training set data is poor, possibly due to uneven data distribution in the training set and other factors, resulting in the model not learning the internal laws of the data during the iterative training process. Then, the amplitude of the change in the neural network model after the current iterative training in the gradient direction is reduced, and the iterative training continues to avoid the model aggressively updating parameters during the iterative training process and missing the optimal solution, so that the model converges stably to a suitable parameter configuration to improve the performance of the model on the training set and the overall data. When the R-squared value of the training set is equal to that of the test set and neither reaches the R-squared threshold, grid search is used to adjust the hyperparameters of the neural network model after the current iterative training. Grid search traverses the value space of the hyperparameters to find a combination of hyperparameters that enables the model to stably fit the data, thereby improving the performance and accuracy of the model.

[0086] It can be understood that only when both the R-squared values of the training set and the test set reach the R-squared threshold, the iterative training is stopped and it is determined as the friction path model. If one reaches the R-squared threshold while the other does not, for example, when the R-squared value of the training set is greater than that of the test set and the R-squared value of the training set reaches the R-squared threshold, the regularization term is also added. Therefore, when there is a situation where one reaches, it is consistent with the above processing method. This ensures that the model has good fitting degree and generalization ability on both the training set and the test set, improves the reliability and accuracy of the model, and by continuously optimizing the parameters of the neural network model, it can accurately output the friction index of the target visual path image, enhancing the stability and reliability of robot navigation.

[0087] In some embodiments of the present application, when determining the comprehensive navigation index of the travel path according to the navigation index and the friction index, judging whether the travel path is abnormal based on the comprehensive navigation index, and determining whether to re-plan the travel path according to the judgment result, it includes: the comprehensive navigation index is the product value of the navigation index and the friction index, and the visual processing module sets a comprehensive navigation index threshold. When the comprehensive navigation index is greater than or equal to the comprehensive navigation index threshold, it is determined that the travel path is abnormal and the travel path is re-planned. When the comprehensive navigation index is less than the comprehensive navigation index threshold, it is determined that the travel path is not abnormal and the robot continues to move forward according to the travel path.

[0088] Specifically, by comprehensively considering the navigation index and the friction index, the ground changes ahead are perceived, a comprehensive navigation index threshold is set and judgments are made based on this threshold. When the comprehensive navigation index is greater than or equal to the comprehensive navigation index threshold, the path is determined to be abnormal and re-planned, while when it is less than the comprehensive navigation index threshold, it is determined to be normal and the robot continues to move forward, ensuring the safety and stability of the navigation.

[0089] In summary, the beneficial effects of the present invention are as follows: By analyzing the visual path image and the visual path image collected last time to generate path correction points, the ground changes of the traveling path can be perceived in real time, so as to timely adjust the traveling direction and the traveling path, avoid the error of path planning, improve the accuracy of robot navigation, divide the path pixel points of the visual path image into clustering sequences, and determine the first navigation factor and the second navigation factor, comprehensively considering the local characteristics of the traveling path, providing detailed and accurate ground information, and enhancing the stability and reliability of navigation. Determine the comprehensive navigation index according to the navigation index and the friction index. When the robot is navigating, it can well cope with the complex and changeable ground conditions in the home environment. When the ground changes cause the traveling path to be abnormal, corresponding judgments can be made, enabling the robot to maintain good navigation performance and environmental adaptability.

[0090] In another preferred manner based on the above embodiments, refer to Figure 2 As shown, this embodiment provides a navigation method for a home intelligent robot, which is applied to the navigation system of the above-mentioned home intelligent robot, and includes:

[0091] S100: Determine the traveling path, collect images of the traveling path based on a plurality of visual devices, determine a plurality of visual images, and process the plurality of visual images to obtain a visual path image;

[0092] S200: Analyze the visual path image and the visual path image collected last time, generate path correction points for the traveling path based on the analysis results, and the path correction points include conforming path points, away path points, and suspected path points;

[0093] S300: When a suspected path point is recognized, obtain all the path pixel points of the visual path image, determine the clustering center point of the visual path image, divide all the path pixel points into clustering sequences according to each path pixel point and the clustering center point, and determine the first navigation factor and the second navigation factor of the traveling path according to the division result;

[0094] S400: Determine the navigation index of the travel path based on the first navigation factor and the second navigation factor. Compare the visual path image with the previously captured visual path image. Determine the friction index of the travel path according to the comparison result and the friction path model. Determine the comprehensive navigation index of the travel path based on the navigation index and the friction index. Judge whether there is an abnormality in the travel path based on the comprehensive navigation index, and determine whether to re-plan the travel path according to the judgment result.

[0095] Specifically, by analyzing the visual path image and the previously captured visual path image, path correction points are generated, which can perceive the ground changes of the travel path in real time, so as to adjust the travel direction and travel path in a timely manner, avoid the errors of path planning, improve the accuracy of robot navigation. Divide the path pixel points of the visual path image into clustering sequences, and determine the first navigation factor and the second navigation factor, which comprehensively consider the local features of the travel path, provide detailed and accurate ground information, and improve the stability and reliability of navigation. Determine the comprehensive navigation index according to the navigation index and the friction index. When the robot is navigating, it can well handle the complex and changeable ground conditions in the home environment. When the ground changes cause abnormalities in the travel path, corresponding judgments can be made, so that the robot can maintain good navigation performance and environmental adaptability.

[0096] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0097] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for realizing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0098] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction means that implements the functions specified in one or more of the processes Figure 1 one or more of the processes and / or Figure 1 boxes or functions specified in a plurality of boxes.

[0099] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one or more of the processes Figure 1 one or more of the processes and / or Figure 1 boxes or functions specified in a plurality of boxes.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications or equivalent substitutions can still be made to the specific embodiments of the present invention, and any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered by the protection scope of the claims of the present invention.

Claims

1. A navigation system for a home intelligent robot, characterized in that, Including: A visual acquisition module, configured to determine a travel path, perform image acquisition on the travel path based on a plurality of visual devices, determine a plurality of visual images, process the plurality of visual images, and obtain a visual path image; A visual judgment module, configured to analyze the visual path image and the visual path image acquired last time, generate path correction points for the travel path based on the analysis result, and the path correction points include compliant path points, away-from-path points, and suspected path points; A visual analysis module, configured to, when a suspected path point is recognized, acquire all path pixel points of the visual path image, determine a clustering center point of the visual path image, perform clustering sequence division on all path pixel points according to each path pixel point and the clustering center point, and determine a first navigation factor and a second navigation factor of the travel path according to the division result; A visual processing module, configured to determine a navigation index of the travel path according to the first navigation factor and the second navigation factor, compare the visual path image with the visual path image acquired last time, determine a friction index of the travel path according to the comparison result and a friction path model, determine a comprehensive navigation index of the travel path according to the navigation index and the friction index, judge whether there is an abnormality in the travel path based on the comprehensive navigation index, and determine whether to re-plan the travel path according to the judgment result.

2. The navigation system of the home intelligent robot according to claim 1, wherein, When processing a plurality of visual images to obtain a visual path image, it includes: The visual acquisition module performs denoising processing on a plurality of visual images, extracts and matches feature points of the denoised plurality of visual images by using a preset image algorithm, and determines interconnection parameters between the plurality of visual images; The preset image algorithm includes a first image algorithm and a second image algorithm. Detect corner points of the denoised plurality of visual images according to the first image algorithm and use them as feature points; Generate a binary descriptor for each feature point according to the second image algorithm, obtain the Hamming distance between each binary descriptor and all binary descriptors, and determine the interconnection parameters between the plurality of visual images based on the Hamming distance. The interconnection parameters include relative position and rotation relationship; According to the interconnection parameters, register the plurality of visual images, merge the registered plurality of visual images based on a multi-band fusion algorithm to determine a merged image, and perform smooth stitching and color balance on the merged image to obtain the visual path image.

3. The navigation system of the home intelligent robot according to claim 2, characterized in that, When analyzing the visual path image and the visual path image acquired last time and generating path correction points for the travel path based on the analysis result, it includes: The visual judgment module performs coincidence matching on the visual path image and the visual path image acquired last time, and generates the path correction points for the travel path according to the coincidence matching result; When the visual path image coincides with the visual path image acquired last time, generate the compliant path points for the travel path and continue to move forward according to the travel path; When the visual path image does not coincide with the previously acquired visual path image, determine the away path point and the suspected path point according to the degree of coincidence; When there is partial coincidence between the visual path image and the previously acquired visual path image, generate suspected path points for the traveling path; When there is no partial coincidence between the visual path image and the previously acquired visual path image, generate away path points for the traveling path and re-determine the traveling path.

4. The navigation system of the home intelligent robot according to claim 3, characterized in that, When dividing all path pixel points into clustering sequences according to each path pixel point and the clustering center point, and determining the first navigation factor and the second navigation factor of the traveling path according to the division result, it includes: The visual analysis module determines the clustering distance between each path pixel point and the clustering center point, and preset a first preset clustering distance and a second preset clustering distance, where the first preset clustering distance is less than the second preset clustering distance; The clustering sequence includes a path first sequence, a path second sequence, and a path third sequence; When the clustering distance is less than or equal to the first preset clustering distance, divide the path pixel point into the path first sequence; When the clustering distance is greater than the first preset clustering distance and less than or equal to the second preset clustering distance, divide the path pixel point into the path second sequence; When the clustering distance is greater than the second preset clustering distance, divide the path pixel point into the path third sequence; Determine the first navigation factor and the second navigation factor of the traveling path according to the path first sequence and the path second sequence.

5. The navigation system of the home intelligent robot according to claim 4, characterized in that, When determining the first navigation factor and the second navigation factor of the traveling path according to the path first sequence and the path second sequence, it includes: The visual analysis module normalizes the path first sequence, determines the path pixel value of each path pixel point, calculates the first standard deviation of the path first sequence after normalization, and extracts the first maximum path pixel value and the first minimum path pixel value of the path first sequence after normalization; The visual analysis module normalizes the path second sequence, determines the path pixel value of each path pixel point, calculates the second standard deviation of the path second sequence after normalization, and extracts the second maximum path pixel value and the second minimum path pixel value of the path second sequence after normalization; Determine the first navigation factor and the second navigation factor of the traveling path according to the first standard deviation, the first maximum path pixel value, the first minimum path pixel value, the second standard deviation, the second maximum path pixel value, the second minimum path pixel value, the path first sequence after normalization, and the path second sequence after normalization.

6. The navigation system of the home intelligent robot according to claim 5, characterized in that, It includes: The visual analysis module determines the first navigation factor and the second navigation factor of the traveling path according to the following formula: Among them, G represents the first navigation factor, U represents the first standard deviation, n represents the number of path pixel values in the first path sequence after normalization, Pi represents the i-th path pixel value in the first path sequence after normalization, Pmin represents the first minimum path pixel value, Pmax represents the first maximum path pixel value, E represents the second navigation factor, T represents the second standard deviation, m represents the number of path pixel values in the second path sequence after normalization, Qj represents the j-th path pixel value in the second path sequence after normalization, Qmin represents the second minimum path pixel value, and Qmax represents the second maximum path pixel value.

7. The navigation system of the household intelligent robot according to claim 6, characterized in that When comparing the visual path image with the previously acquired visual path image and determining the friction index of the traveling path according to the comparison result and the friction path model, it includes: The visual processing module deletes the region in the visual path image that has local overlap with the previously acquired visual path image, and determines the target visual path image according to the deletion result; The visual processing module obtains the historical visual image dataset and divides the historical visual image dataset into a training set and a test set; Obtain a pre-selected neural network model, perform iterative training on the neural network model according to the training set, test the iteratively trained neural network model according to the test set, and determine the R-squared value of the training set and the R-squared value of the test set after each iterative training. According to the relationship between the R-squared value of the training set and the R-squared value of the test set after each iterative training, determine the friction path model; Substitute the target visual path image into the friction path model to determine the friction index of the traveling path.

8. The navigation system of the home intelligent robot according to claim 7, characterized in that, When determining the friction path model according to the relationship between the R-squared value of the training set and the R-squared value of the test set after each iterative training, it includes: The visual processing module sets the R-squared threshold; If the R-squared value of the training set and the R-squared value of the test set of the neural network model after the current iterative training do not reach the R-squared threshold, and the R-squared value of the training set is greater than the R-squared value of the test set, then add a regularization term to the neural network model after the current iterative training and continue iterative training; If the R-squared value of the training set and the R-squared value of the test set of the neural network model after the current iterative training do not reach the R-squared threshold, and the R-squared value of the training set is less than the R-squared value of the test set, then reduce the change amplitude of the neural network model after the current iterative training in the gradient direction and continue iterative training; If the R-squared value of the training set and the R-squared value of the test set of the neural network model after the current iterative training do not reach the R-squared threshold, and the R-squared value of the training set is equal to the R-squared value of the test set, then use grid search to adjust the hyperparameters of the neural network model after the current iterative training and continue iterative training; If the R-squared value of the training set and the R-squared value of the test set of the neural network model after the current iterative training both reach the R-squared threshold, stop iterative training and use the neural network model after the current iterative training as the friction path model.

9. The navigation system of the household intelligent robot according to claim 8, characterized in that, When determining the comprehensive navigation index of the travel path based on the navigation index and the friction index, and judging whether there is an abnormality in the travel path based on the comprehensive navigation index, and determining whether to re-plan the travel path according to the judgment result, it includes: The comprehensive navigation index is the product value of the navigation index and the friction index, and the visual processing module sets a comprehensive navigation index threshold; When the comprehensive navigation index is greater than or equal to the comprehensive navigation index threshold, it is determined that there is an abnormality in the travel path, and the travel path is re-planned; When the comprehensive navigation index is less than the comprehensive navigation index threshold, it is determined that there is no abnormality in the travel path, and continue to move forward according to the travel path.

10. A navigation method for a home intelligent robot, applied to the navigation system of the home intelligent robot according to any one of claims 1-9, characterized in that, It includes: Determine the travel path, collect images of the travel path based on a number of visual devices, determine a number of visual images, and process the number of visual images to obtain a visual path image; Analyze the visual path image and the visual path image collected last time, and generate a path correction point for the travel path based on the analysis result. The path correction point includes a conforming path point, a far-from-path point, and a suspected path point; When the suspected path point is recognized, obtain all path pixel points of the visual path image, and determine the clustering center point of the visual path image. Divide all path pixel points into clustering sequences according to each path pixel point and the clustering center point, and determine the first navigation factor and the second navigation factor of the travel path according to the division result; Determine the navigation index of the travel path according to the first navigation factor and the second navigation factor, compare the visual path image with the visual path image collected last time, determine the friction index of the travel path according to the comparison result and the friction path model, determine the comprehensive navigation index of the travel path according to the navigation index and the friction index, judge whether there is an abnormality in the travel path based on the comprehensive navigation index, and determine whether to re-plan the travel path according to the judgment result.

Images