An underconstrained environment detection method, device, and computer-readable storage medium

Abstract

The present application discloses an underconstrained environment detection method, apparatus and computer-readable storage medium, which relates to the field of intelligent navigation. By collecting the point cloud data of each frame of the image during the laser positioning process; extracting the straight lines formed by the point cloud data; determining the parallelism between the extracted straight lines; and determining the underconstrained situation of the current environment according to the parallelism. It can be seen from this that the above solution extracts the straight lines formed by the point cloud data in the image and determines the parallelism of the straight lines; since there is a corresponding relationship between the straight line parallelism and the environmental constraint, the greater the straight line parallelism, the worse the environmental constraint at this time, so the detection of the underconstrained environment is realized through the straight line parallelism. The influence of the point cloud matching data on the mapping under the underconstrained environment is avoided, and the mapping quality is improved.

Description

Technical Field

[0001] This application relates to the field of intelligent navigation, and particularly to a method and device for detecting under-constrained environments and a computer-readable storage medium. Background Art

[0002] Simultaneous Localization And Mapping (SLAM) and Visual SLAM have received increasing attention and are gradually being applied in various fields, such as in sweeping robots, Automated Guided Vehicles (AGVs), and autonomous driving vehicles.

[0003] However, during the mapping process, in some under-constrained environments where the environments are too similar and lack features, due to the environment around the robot, the existing constraints cannot determine the predicted pose within a small range, resulting in an increase in the pose estimation error. For example, Figure 1 As shown, when mapping in a long corridor environment, there are only two parallel long straight lines in the map. When the robot uses radar to match the laser point cloud with the map, there are good constraints only in the direction perpendicular to the wall, while in the direction parallel to the wall, the matching effect at each position is the same. Therefore, the predicted pose will have a large error. At the same time, the results of these incorrect matches will also be used as constraints for backend optimization, further affecting other correct constraints and reducing the mapping quality. Summary of the Invention

[0004] The purpose of this application is to provide a method and device for detecting under-constrained environments and a computer-readable storage medium, which can detect the under-constrained situation in the mapping environment and thus improve the mapping quality.

[0005] To solve the above technical problems, this application provides a method for detecting under-constrained environments, including:

[0006] Collecting the point cloud data of each frame of the image during the laser positioning process;

[0007] Extracting the straight lines formed by the point cloud data;

[0008] Determining the parallelism between the extracted straight lines;

[0009] Determining the under-constrained situation of the current environment according to the parallelism.

[0010] Preferably, the determining the under-constrained situation of the current environment according to the parallelism includes:

[0011] Obtaining the information entropy corresponding to the parallelism of each straight line;

[0012] Determine the underconstrained situation of the current environment according to the information entropy.

[0013] Preferably, the extraction of the straight lines formed by the point cloud data includes:

[0014] Respectively fit the point cloud data in the current frame image with the remaining point cloud data in the current frame image to generate various sub-line segments;

[0015] Fit the point cloud data located on the extension lines of each of the seed line segments with the point cloud data located on the same seed line segment to generate each of the straight lines;

[0016] Merge the straight lines with a parallelism greater than a first threshold or the coincident straight lines among the straight lines as the straight lines extracted from the point cloud data.

[0017] Preferably, the respectively fitting the point cloud data in the current frame image with the remaining point cloud data in the current frame image includes:

[0018] Respectively perform least squares fitting on the point cloud data in the current frame image with a preset number of the remaining point cloud data in the current frame image;

[0019] Preferably, the obtaining of the information entropy corresponding to the parallelism of each of the straight lines includes:

[0020] Respectively obtain the included angle between the normal vector of each of the straight lines and the x-axis as the inclination angle of each of the straight lines;

[0021] Obtain the distribution histogram of the inclination angles in a preset number of angular intervals; wherein, the sizes of the angular intervals are equal and continuous, and the angular range composed of the angular intervals is from 0 degrees to 180 degrees;

[0022] Determine the probability of each of the inclination angles in each of the angular intervals according to the distribution histogram;

[0023] Determine the information entropy of each of the straight lines according to the probability of each of the inclination angles in each of the angular intervals.

[0024] Preferably, the determining of the underconstrained situation of the current environment according to the information entropy includes:

[0025] If the information entropy is not greater than a second threshold, confirm that the straight lines are parallel and the current environment is underconstrained;

[0026] If the information entropy is not less than a third threshold, confirm that the directions of the straight lines are dispersed and the current environment meets the constraint requirements;

[0027] If the information entropy is greater than the second threshold and less than the third threshold, and there is a pair of target angle intervals with a distance not less than the fourth threshold and corresponding probabilities not less than the fifth threshold, it is confirmed that the line directions are dispersed and the current environment meets the constraint requirements; otherwise, it is confirmed that the lines are parallel and the current environment is under-constrained.

[0028] Wherein, the target angle interval is the angle interval including the inclination angle, and the second threshold is less than the third threshold.

[0029] Preferably, after merging the lines with a parallelism greater than the first threshold or the overlapping lines among the lines as the lines extracted from the point cloud data, the method further includes:

[0030] Deleting the lines whose lengths do not meet the preset length.

[0031] To solve the above technical problems, the present application further provides an under-constrained environment detection device, including:

[0032] An acquisition module, configured to acquire the point cloud data of each frame of the image during the laser positioning process;

[0033] An extraction module, configured to extract the lines formed by the point cloud data;

[0034] A first determination module, configured to determine the parallelism between the extracted lines;

[0035] A second determination module, configured to determine the under-constrained situation of the current environment according to the parallelism.

[0036] To solve the above technical problems, the present application further provides another under-constrained environment detection device, including:

[0037] A memory, configured to store a computer program;

[0038] A processor, configured to implement the steps of the above under-constrained environment detection method when executing the computer program.

[0039] To solve the above technical problems, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above under-constrained environment detection method are implemented.

[0040] The underconstrained environment detection method provided by this application collects the point cloud data of each frame of the image during the laser positioning process; extracts the straight lines formed by the point cloud data; determines the parallelism between the extracted straight lines; and determines the underconstrained situation of the current environment according to the parallelism. It can be seen from this that the above solution extracts the straight lines formed by the point cloud data in the image and determines the parallelism of the straight lines. Since there is a corresponding relationship between the straight line parallelism and the environmental constraints, the greater the straight line parallelism, the worse the current environmental constraints. Therefore, the detection of the underconstrained environment is realized through the straight line parallelism. This avoids the influence of the point cloud matching data in the underconstrained environment on mapping and improves the mapping quality.

[0041] In addition, the embodiment of this application also provides an underconstrained environment detection device and a computer-readable storage medium, and the effects are the same as above. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of this application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0043] Figure 1 It is a schematic diagram of long corridor environment mapping provided by this application;

[0044] Figure 2 It is a flowchart of an underconstrained environment detection method provided by an embodiment of this application;

[0045] Figure 3 It is a flowchart of the underconstrained environment detection method in the application scenario provided by an embodiment of this application;

[0046] Figure 4 It is a schematic structural diagram of an underconstrained environment detection device provided by an embodiment of this application;

[0047] Figure 5 It is a schematic structural diagram of another underconstrained environment detection device provided by an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] Next, the technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only some embodiments of this application, rather than all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of this application.

[0049] The core of this application is to provide an underconstrained environment detection method, device and computer-readable storage medium.

[0050] To enable those skilled in the art to better understand the solution of this application, the following further detailed description of this application will be given in conjunction with the accompanying drawings and specific embodiments.

[0051] Figure 2 It is a flowchart of an underconstrained environment detection method provided for an embodiment of this application. As Figure 2 shown, the underconstrained environment detection method includes:

[0052] S10: Collect the point cloud data of each frame of the image during the laser positioning process.

[0053] During the laser positioning process of the robot, the robot scans the surrounding environment through a lidar and collects the point cloud data in each frame of the scanned image (point cloud data). Point cloud data refers to a set of vectors in a three-dimensional coordinate system, and its scanned data is recorded in the form of points, containing information about the surrounding environment. Each point contains three-dimensional coordinates and may contain color information (RGB) or reflection intensity information (Intensity). The color information is usually obtained by a camera to acquire a color image, and then the color information of the corresponding pixel at the corresponding position is assigned to the corresponding point in the point cloud. The acquisition of the reflection intensity information is to receive the echo intensity of the laser collected by the receiving device, and this intensity information is related to the surface material, roughness, incident angle direction, transmitted energy, and laser wavelength of the target.

[0054] S11: Extract the straight lines formed by the point cloud data.

[0055] It can be understood that since the point cloud data is distributed on each frame of the image and the scanned data is recorded in the form of points, multiple straight lines can be generated based on the point cloud data. According to the properties of straight lines, a straight line can be determined by at least two point cloud data, and any point cloud data can form a straight line with the remaining point cloud data in the current frame of the image. In specific implementation, straight lines containing a preset number of point cloud data can also be extracted. In this embodiment, the specific process of extracting the straight lines formed by the point cloud data is not limited and depends on the specific implementation situation.

[0056] S12: Determine the parallelism between the extracted straight lines.

[0057] Parallelism refers to the degree of parallelism between two straight lines and is used to evaluate the parallel state between straight lines. When obtaining the parallelism between two straight lines, one of the straight lines is the evaluation reference. During the laser positioning process, there may be a situation where the environment around the robot is too similar and lacks features. For example, in Figure 1In the long corridor environment, the constraint conditions in the direction parallel to the wall are not sufficient to determine the predicted pose of the robot within a small range. In this environment, for each straight line formed by the point cloud data collected, the parallelism of the straight lines will be higher than that of the straight lines formed by the point cloud data in other constraint environments, and the greater the parallelism of the straight lines, the worse the environmental constraints at this time. Therefore, the parallelism of the straight lines formed by the point cloud data can be used as a detection basis for the under-constrained situation of the environment.

[0058] S13: Determine the under-constrained situation of the current environment according to the parallelism.

[0059] As can be seen from S12, there is a corresponding relationship between the straight line parallelism and the environmental constraints. The greater the straight line parallelism, the worse the environmental constraints at this time. Therefore, the under-constrained situation of the current environment can be determined according to the parallelism. Specifically, the under-constrained situation of the current environment can be determined by comparing the relationship between the obtained straight line parallelism and the threshold; the information entropy corresponding to the straight line parallelism can also be obtained. The information entropy is used to characterize the uncertainty of the straight line parallel event, so as to determine the under-constrained situation of the current environment according to the relationship between the information entropy and the threshold. In this embodiment, the specific process of determining the under-constrained situation of the current environment according to the parallelism is not limited and depends on the specific implementation situation.

[0060] In this embodiment, by collecting the point cloud data of each frame of the image during the laser positioning process; extracting the straight lines formed by the point cloud data; determining the parallelism between the extracted straight lines; determining the under-constrained situation of the current environment according to the parallelism. It can be seen from this that the above solution extracts the straight lines formed by the point cloud data in the image and determines the parallelism of the straight lines; since there is a corresponding relationship between the straight line parallelism and the environmental constraints, the greater the straight line parallelism, the worse the environmental constraints at this time, so the detection of the under-constrained environment is realized through the straight line parallelism. The influence of the point cloud matching data on the mapping in the under-constrained environment is avoided, and the mapping quality is improved.

[0061] Based on the above embodiment:

[0062] As a preferred embodiment, determining the under-constrained situation of the current environment according to the parallelism includes:

[0063] Obtain the information entropy corresponding to the parallelism of each straight line;

[0064] Determine the under-constrained situation of the current environment according to the information entropy.

[0065] Information entropy is used to describe the uncertainty of the occurrence of various possible events in the information source. Uncertainty is positively correlated with information entropy. For the parallelism between lines, the greater the parallelism between two lines, the closer the two lines are to being parallel, and the worse the environmental constraints at this time. For the information entropy representing the parallel relationship between two lines, the smaller the uncertainty of the parallelism of the two lines, the smaller the information entropy. Therefore, as a preferred embodiment, the under-constrained situation of the current environment is determined according to the parallelism. Specifically, the information entropy corresponding to the parallelism of each line can be obtained, and the under-constrained situation of the current environment is determined according to the information entropy. In this embodiment, there is no limitation on the method of obtaining the information entropy, and there is no limitation on the specific process of determining the under-constrained situation of the current environment according to the information entropy, which depends on the specific implementation situation.

[0066] In this embodiment, the information entropy corresponding to the parallelism of each line is obtained, and the under-constrained situation of the current environment is determined according to the information entropy, realizing the detection of the under-constrained situation of the environment.

[0067] Based on the above embodiment:

[0068] As a preferred embodiment, the extraction of the lines composed of point cloud data includes:

[0069] The point cloud data in the current frame image is respectively fitted with the remaining point cloud data in the current frame image to generate various sub-line segments;

[0070] The point cloud data located on the extension lines of various sub-line segments is fitted with the point cloud data located on the same sub-line segment to generate each line;

[0071] The lines with a parallelism greater than the first threshold or the overlapping lines among the lines are merged as the lines extracted from the point cloud data.

[0072] As a preferred embodiment, to extract the lines composed of point cloud data, it is first necessary to preliminarily extract each line. Specifically, the point cloud data in the current frame image is respectively fitted with the remaining point cloud data in the current frame image to generate various sub-line segments.

[0073] It should be noted that in the specific implementation, each point cloud data in the current frame image is fitted with the remaining point cloud data in the current frame image, and each point cloud data forms a seed line segment with the remaining point cloud data. In this embodiment, there is no limitation on the specific number of the remaining point cloud data in the fitting process, which depends on the specific implementation situation.

[0074] Since the seed line segments are just straight lines preliminarily extracted from the point cloud data, and there will be some point cloud data in all the point cloud data that lies on the extension lines of the seed line segments. Therefore, in order to generate straight lines with complete lengths, the point cloud data on the extension lines of various seed line segments is further fitted with the point cloud data on the same seed line segment, thereby generating each straight line. To determine the point cloud data on the extension lines of various seed line segments, specifically, it can be determined by the distance between the point cloud data outside the seed line segment and the extension line of the seed line segment. When the distance is 0, it is considered that the point cloud data lies on the extension line of the seed line segment; it can also be judged by the coordinates of the point cloud data and the equation of the extension line of the seed line segment, depending on the specific implementation situation.

[0075] After obtaining each straight line, there will be some parallel or coincident straight lines among them. To reduce the computational amount, the straight lines with a parallelism greater than the first threshold or the coincident straight lines among each straight line are merged as the straight lines extracted from the point cloud data. In this embodiment, there is no limitation on the first threshold, which depends on the specific implementation situation.

[0076] It should be noted that in this embodiment, there is no limitation on the specific fitting method of the point cloud data in the process of the seed line segment and straight line generation. It can be fitted by the least squares method, and can also be fitted by the method of approximating discrete data with an analytical expression or other methods, depending on the specific implementation situation.

[0077] In this embodiment, by respectively fitting the point cloud data in the current frame image with the remaining point cloud data in the current frame image to generate various seed line segments; fitting the point cloud data on the extension lines of various seed line segments with the point cloud data on the same seed line segment to generate each straight line; merging the straight lines with a parallelism greater than the first threshold or the coincident straight lines among each straight line as the straight lines extracted from the point cloud data, the extraction of the straight lines is realized, so as to facilitate the subsequent determination of the under-constrained situation of the current environment through the parallelism of the straight lines.

[0078] Based on the above embodiment:

[0079] As a preferred embodiment, respectively fitting the point cloud data in the current frame image with the remaining point cloud data in the current frame image includes:

[0080] Respectively performing least squares fitting on the point cloud data in the current frame image and a preset number of the remaining point cloud data in the current frame image.

[0081] As a preferred embodiment, in the process of extracting the seed line segments formed by the point cloud data, to reduce the computational amount and improve the generation efficiency of the seed line segments, in the specific implementation, respectively perform least squares fitting on the point cloud data in the current frame image and a preset number of the remaining point cloud data in the current frame image.

[0082] It should be noted that the preset quantity in this embodiment can be set to be not less than 2, so as to be able to generate representative seed line segments. In specific implementation, the preset quantity can also be set to be less than the total number of the remaining point cloud data in the current frame image. For example, the preset quantity is set to an integer not less than 2 and not greater than 10, so that the point cloud data in the current frame image is fitted with the preset quantity of point cloud data around it, which can not only obtain representative seed line segments, but also effectively reduce the calculation amount.

[0083] In addition, during the generation process of the seed line segments, the point cloud data is fitted by the least squares method. As a fitting method, the least squares method has higher calculation efficiency during the fitting process of the point cloud data.

[0084] In this embodiment, the point cloud data in the current frame image is respectively fitted with the preset quantity of the remaining point cloud data in the current frame image by the least squares method, realizing the generation of the seed line segments, with higher generation efficiency.

[0085] Based on the above embodiments:

[0086] As a preferred embodiment, obtaining the information entropy corresponding to the parallelism of each straight line includes:

[0087] Respectively obtain the included angle between the normal vector of each straight line and the x-axis as the inclination angle of each straight line;

[0088] Obtain the distribution histogram of the inclination angles in a preset number of angular intervals; wherein, the sizes of the angular intervals are equal and continuous, and the angular range composed of the angular intervals is from 0 degrees to 180 degrees;

[0089] Determine the probability of each inclination angle in each angular interval according to the distribution histogram;

[0090] Determine the information entropy of each straight line according to the probability of each inclination angle in each angular interval.

[0091] In the above embodiments, there is no limitation on the way of obtaining the information entropy. As a preferred embodiment, in this embodiment, the information entropy is determined by obtaining the probability of the inclination angles of each straight line in each angular interval.

[0092] Specifically, first obtain the inclination angles of each straight line. In this embodiment, the included angle between the normal vector of each straight line and the x-axis is respectively obtained as the inclination angle of each straight line. Set multiple angular intervals, the sizes of the angular intervals are equal and continuous, and the angular range composed of the angular intervals is from 0 degrees to 180 degrees. Based on the angular intervals and the inclination angles, generate the distribution histogram of all inclination angles in each angular interval. The distribution histogram can show the distribution of the inclination angles of each straight line in different angular intervals. Further obtain the probabilities p i of the inclination angles appearing in each angular interval, and according to the probabilities pi The information entropy is obtained through the calculation formula of information entropy. The calculation formula of information entropy is as follows:

[0093]

[0094] Among them, in the above formula, p i is the probability of the inclination angle appearing in each angular interval, and n is the number of angular intervals.

[0095] In this embodiment, by respectively obtaining the included angle between the normal vector of each straight line and the x-axis as the inclination angle of each straight line; obtaining the distribution histogram of each inclination angle in a preset number of angular intervals; wherein, the sizes of each angular interval are equal and continuous, and the angular range composed of each angular interval is from 0 degree to 180 degrees; determining the probability of each inclination angle in each angular interval according to the distribution histogram; and determining the information entropy of each straight line according to the probability of each inclination angle in each angular interval, the acquisition of the information entropy is realized, so as to facilitate subsequent determination of the under-constrained situation of the current environment according to the information entropy.

[0096] Based on the above embodiment:

[0097] As a preferred embodiment, determining the under-constrained situation of the current environment according to the information entropy includes:

[0098] If the information entropy is not greater than the second threshold, it is confirmed that the straight lines are parallel and the current environment is under-constrained;

[0099] If the information entropy is not less than the third threshold, it is confirmed that the directions of the straight lines are dispersed and the current environment meets the constraint requirements;

[0100] If the information entropy is greater than the second threshold and less than the third threshold, and there is a pair of target angular intervals with a distance not less than the fourth threshold and corresponding probabilities not less than the fifth threshold, it is confirmed that the directions of the straight lines are dispersed and the current environment meets the constraint requirements; otherwise, it is confirmed that the straight lines are parallel and the current environment is under-constrained;

[0101] Among them, the target angular interval is the angular interval containing the inclination angle, and the second threshold is less than the third threshold.

[0102] As a preferred embodiment, the specific process of determining the under-constrained situation of the current environment according to the information entropy in this embodiment is as follows:

[0103] After obtaining the information entropy corresponding to the parallelism of each straight line, compare the information entropy with the second threshold and the third threshold. Among them, the second threshold is less than the third threshold. If the information entropy is not greater than the second threshold, the inclination angle distributions of the straight lines are concentrated, and it is confirmed that the straight lines are parallel and the current environment is under-constrained. If the information entropy is not less than the third threshold, the inclination angle distributions of the straight lines are dispersed, and it is confirmed that the directions of the straight lines are dispersed and the current environment meets the constraint requirements.

[0104] If the information entropy is greater than the second threshold and less than the third threshold, in order to further determine the under-constrained situation of the current environment based on the information entropy, it is also necessary to determine whether there is a pair of target angle intervals whose distance is not less than the fourth threshold and the corresponding probability is not less than the fifth threshold; it should be noted that the target angle interval is an angle interval including the inclination angle; if so, it is confirmed that the line directions are dispersed and the current environment meets the constraint requirements; if not, it is confirmed that the lines are parallel and the current environment is under-constrained. Further, after detecting that the current environment is under-constrained, the point cloud matching data in the current environment can be removed, thus avoiding the influence of the under-constrained environment on subsequent mapping.

[0105] It should be noted that in this embodiment, there are no restrictions on the second threshold, the third threshold, the fourth threshold, and the fifth threshold, which are determined according to specific implementation situations.

[0106] In this embodiment, if the information entropy is not greater than the second threshold, it is confirmed that the lines are parallel and the current environment is under-constrained; if the information entropy is not less than the third threshold, it is confirmed that the line directions are dispersed and the current environment meets the constraint requirements; if the information entropy is greater than the second threshold and less than the third threshold, and there is a pair of target angle intervals whose distance is not less than the fourth threshold and the corresponding probability is not less than the fifth threshold, it is confirmed that the line directions are dispersed and the current environment meets the constraint requirements; otherwise, it is confirmed that the lines are parallel and the current environment is under-constrained; where the target angle interval is an angle interval including the inclination angle, and the second threshold is less than the third threshold. The detection of the under-constrained situation of the current environment based on the information entropy is realized, and the mapping quality is improved.

[0107] Based on the above embodiments:

[0108] As a preferred embodiment, after merging the lines with a parallelism greater than the first threshold or the overlapping lines among the lines as the lines for point cloud data extraction, it further includes:

[0109] Deleting the lines whose lengths do not meet the preset length.

[0110] As a preferred embodiment, after extracting the lines formed by the point cloud data, the extracted lines have different lengths. In order to remove the non-representative lines among all the lines and improve the calculation efficiency of the subsequent information entropy, the lines whose lengths do not meet the preset length among the lines are further deleted, so as to retain the lines that meet the preset length. There are no restrictions on the preset length in this embodiment, which is determined according to specific implementation situations.

[0111] In this embodiment, after merging the lines with a parallelism greater than the first threshold or the overlapping lines among the lines as the lines for point cloud data extraction, deleting the lines whose lengths do not meet the preset length improves the calculation efficiency.

[0112] To enable those skilled in the art to better understand the technical solution of this application, the following will further elaborate on the above application in conjunction with the appended Figure 3 drawings. Figure 3 FIG. is a flowchart of the underconstrained environment detection method provided in the embodiment of this application in an application scenario. In specific implementation, the acquisition and processing of point cloud data can be achieved through a robot, and the robot can specifically include a radar, a processor, a communication module, etc. As Figure 3 shown, the method includes:

[0113] S14: The radar starts.

[0114] S15: The radar continuously moves and scans the surrounding environment to collect point cloud data.

[0115] S16: The radar sends the collected point cloud data to the processor.

[0116] S17: The processor detects the underconstrained situation of the environment where the point cloud data of each frame of image is located.

[0117] S18: The processor deletes the point cloud data in the underconstrained environment and retains the point cloud data in the constrained environment.

[0118] S19: The processor sends the point cloud data in the constrained environment to the communication module.

[0119] S20: The communication module transmits the point cloud data in the constrained environment to the background for positioning and mapping.

[0120] The specific application scenario is as follows: When the robot performs positioning and mapping, the robot turns on the radar and moves in the current environment. The radar scans the surrounding environment frame by frame to collect the point cloud data of each frame of the current environment. The processor detects the underconstrained situation of the environment where the robot is located according to the point cloud data of each frame of image; specifically, by extracting the straight lines formed by the point cloud data, determining the parallelism between the extracted straight lines, and determining the underconstrained situation of the current environment according to the parallelism. After determining the underconstrained situation of the current environment according to the point cloud data of each frame of image, the processor retains the point cloud data in the constrained environment and deletes the point cloud data in the underconstrained environment to eliminate the influence of the point cloud data in the underconstrained environment on mapping. Further, the point cloud data in the constrained environment is transmitted to the background through the communication module for positioning and mapping.

[0121] In the above embodiment, the underconstrained environment detection method is described in detail, and this application also provides an embodiment corresponding to the underconstrained environment detection device. It should be noted that this application describes the embodiment of the device part from two perspectives, one is from the perspective of functional modules, and the other is from the perspective of hardware structure.

[0122] Figure 4The structural schematic diagram of an underconstrained environment detection device provided by an embodiment of the present application. As Figure 4 shown, the underconstrained environment detection device includes:

[0123] An acquisition module 10, configured to acquire the point cloud data of each frame of the image during the laser positioning process.

[0124] An extraction module 11, configured to extract the straight lines formed by the point cloud data.

[0125] A first determination module 12, configured to determine the parallelism between the extracted straight lines.

[0126] A second determination module 13, configured to determine the underconstrained situation of the current environment according to the parallelism.

[0127] In this embodiment, the underconstrained environment detection device includes an acquisition module, an extraction module, a first determination module, and a second determination module. By acquiring the point cloud data of each frame of the image during the laser positioning process; extracting the straight lines formed by the point cloud data; determining the parallelism between the extracted straight lines; and determining the underconstrained situation of the current environment according to the parallelism. It can be seen that the above solution extracts the straight lines formed by the point cloud data in the image and determines the parallelism of the straight lines; since there is a corresponding relationship between the straight line parallelism and the environmental constraints, the greater the straight line parallelism, the worse the environmental constraints at this time, so the underconstrained environment is detected through the straight line parallelism. The influence of the point cloud matching data on mapping under the underconstrained environment is avoided, and the mapping quality is improved.

[0128] Figure 5 The structural schematic diagram of another underconstrained environment detection device provided by an embodiment of the present application.

[0129] As Figure 5 shown, the underconstrained environment detection device includes:

[0130] A memory 20, configured to store a computer program.

[0131] A processor 21, configured to implement the steps of the underconstrained environment detection method mentioned in the above embodiment when executing the computer program.

[0132] The underconstrained environment detection device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.

[0133] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 may be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may further include an artificial intelligence (AI) processor, and the AI processor is used to process computational operations related to machine learning.

[0134] The memory 20 may include one or more computer-readable storage media, and the computer-readable storage media may be non-transitory. The memory 20 may further include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201. After the computer program is loaded and executed by the processor 21, it can implement the relevant steps of the underconstrained environment detection method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 20 may further include an operating system 202 and data 203, etc., and the storage method may be transient storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include, but is not limited to, the data involved in the underconstrained environment detection method.

[0135] In some embodiments, the underconstrained environment detection device may further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.

[0136] Those skilled in the art can understand that Figure 5 the structure shown in

[0137] In this embodiment, the underconstrained environment detection device includes a memory and a processor. The processor is configured to execute a computer program to implement the steps of the underconstrained environment detection method mentioned in the above embodiment. By collecting the point cloud data of each frame of the image during the laser positioning process; extracting the straight lines formed by the point cloud data; determining the parallelism between the extracted straight lines; and determining the underconstrained situation of the current environment according to the parallelism. It can be seen that the above solution extracts the straight lines formed by the point cloud data in the image and determines the parallelism of the straight lines; since there is a corresponding relationship between the straight line parallelism and the environmental constraints, the greater the straight line parallelism, the worse the environmental constraints at this time, so the detection of the underconstrained environment is realized through the straight line parallelism. The influence of the point cloud matching data on mapping under the underconstrained environment is avoided, and the mapping quality is improved.

[0138] Finally, the present application also provides an embodiment corresponding to a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the steps recorded in the above method embodiment.

[0139] It can be understood that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage media include: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks, and other media that can store program codes.

[0140] In this embodiment, a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements the steps recorded in the above method embodiment. By collecting the point cloud data of each frame of the image during the laser positioning process; extracting the straight lines formed by the point cloud data; determining the parallelism between the extracted straight lines; and determining the underconstrained situation of the current environment according to the parallelism. It can be seen that the above solution extracts the straight lines formed by the point cloud data in the image and determines the parallelism of the straight lines; since there is a corresponding relationship between the straight line parallelism and the environmental constraints, the greater the straight line parallelism, the worse the environmental constraints at this time, so the detection of the underconstrained environment is realized through the straight line parallelism. The influence of the point cloud matching data on mapping under the underconstrained environment is avoided, and the mapping quality is improved.

[0141] The above has introduced in detail a method, device, and computer-readable storage medium for underconstrained environment detection provided by this application. The various embodiments in the specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple. For related parts, reference can be made to the description in the method section. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0142] It should also be noted that in this specification, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article, or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article, or device including the said element.

Claims

1. An underconstrained environment detection method, characterized in that, it includes: Collect the point cloud data of each frame of the image during the laser positioning process; Extract the straight lines formed by the point cloud data; Determine the parallelism between the extracted straight lines; Determine the underconstrained situation of the current environment according to the parallelism; The determining the underconstrained situation of the current environment according to the parallelism includes: Obtain the information entropy corresponding to the parallelism of each straight line; Determine the underconstrained situation of the current environment according to the information entropy.

2. The underconstrained environment detection method according to claim 1, characterized in that, the extracting the straight lines formed by the point cloud data includes: Respectively fit the point cloud data in the current frame image with the remaining point cloud data in the current frame image to generate various sub-line segments; Fit the point cloud data located on the extension lines of each seed line segment with the point cloud data located on the same seed line segment to generate each straight line; Merge the straight lines with the parallelism greater than the first threshold or the overlapping straight lines among the straight lines as the straight lines extracted from the point cloud data.

3. The underconstrained environment detection method according to claim 2, characterized in that, the respectively fitting the point cloud data in the current frame image with the remaining point cloud data in the current frame image includes: Respectively perform least squares fitting on the point cloud data in the current frame image and the remaining point cloud data in the current frame image with a preset number.

4. The underconstrained environment detection method according to claim 1, characterized in that, the obtaining the information entropy corresponding to the parallelism of each straight line includes: Respectively obtain the included angle between the normal vector of each straight line and the x-axis as the inclination angle of each straight line; Obtain the distribution histogram of the inclination angles in a preset number of angular intervals; wherein, the sizes of the angular intervals are equal and continuous, and the angular range composed of the angular intervals is from 0 degree to 180 degrees; Determine the probability of each inclination angle in each angular interval according to the distribution histogram; Determine the information entropy of each straight line according to the probability of each inclination angle in each angular interval.

5. The underconstrained environment detection method according to claim 4, characterized in that, the determining the underconstrained situation of the current environment according to the information entropy includes: If the information entropy is not greater than the second threshold, it is confirmed that the straight lines are parallel and the current environment is underconstrained; If the information entropy is not less than the third threshold, it is confirmed that the directions of the straight lines are scattered and the current environment meets the constraint requirements; If the information entropy is greater than the second threshold and less than the third threshold, and there is a pair of target angular intervals with a distance not less than the fourth threshold and the corresponding probability not less than the fifth threshold, it is confirmed that the directions of the straight lines are scattered and the current environment meets the constraint requirements; otherwise, it is confirmed that the straight lines are parallel and the current environment is underconstrained; wherein, the target angular interval is the angular interval containing the inclination angle, and the second threshold is less than the third threshold.

6. The underconstrained environment detection method according to claim 2, characterized in that, After merging the lines with a parallelism greater than a first threshold or the overlapping lines among the respective lines as the lines for point cloud data extraction, the following steps are further included: Delete the lines whose lengths do not meet the preset length.

7. An underconstrained environment detection device, characterized in that, it includes: An acquisition module for acquiring point cloud data of each frame of the image during the laser positioning process; An extraction module for extracting the lines formed by the point cloud data; A first determination module for determining the parallelism between the respective lines that have been extracted; A second determination module for determining the underconstrained situation of the current environment according to the parallelism; The determination of the underconstrained situation of the current environment according to the parallelism includes: Obtaining the information entropy corresponding to the parallelism of the respective lines; Determining the underconstrained situation of the current environment according to the information entropy.

8. An underconstrained environment detection device, characterized in that, it includes: A memory for storing a computer program; A processor for implementing the steps of the underconstrained environment detection method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that, a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the underconstrained environment detection method according to any one of claims 1 to 6 are implemented.

Images