An environmental perception method and system for an intelligent lawn mowing robot

Abstract

An environment perception method and system for an intelligent lawn mowing robot, the device used includes a camera, an inertial measurement unit IMU, a global navigation satellite system GNSS, an odometer, a water drop sensor, a photosensitive sensor, and an ultrasonic sensor installed on the intelligent lawn mowing robot, including the following steps: S1, the intelligent lawn mowing robot obtains the environmental rainwater situation, compares it with the set threshold value to determine whether to continue working; S2, the intelligent lawn mowing robot obtains the ambient light intensity and uses the camera and ultrasonic module for fusion to realize environmental perception and target detection and tracking; S3, the intelligent lawn mowing robot obtains its current high-precision position information through a fusion positioning algorithm based on Kalman filtering; S4, the scene judgment is performed through the information obtained in steps S2 and S3; S5, the corresponding scene information is used as the input of the decision-making system to make a decision. The present invention improves the obstacle perception ability and positioning accuracy under low light.

Description

Technical Field

[0001] The present invention belongs to the field of garden machinery and artificial intelligence, and in particular relates to an environment perception method and system for an intelligent lawn mowing robot. Background Art

[0002] With the continuous development of science and technology and the continuous increase of human resources, the market of smart lawn mower robots has also become hot. However, most of the current smart lawn mower robots use a single sensor to perceive the external environment or their own status. For example, most of the current more advanced smart lawn mower robots only use ultrasonic sensors to detect external obstacles, which makes it difficult to cope with complex outdoor environments.

[0003] At present, many intelligent lawn mower robots use pre-buried wires and other methods to construct mowing areas. Furthermore, electronic virtual fences are used to construct areas, and the Global Positioning System (GPS) and Inertial Measurement Unit (IMU) are used to perceive their own positions. However, the accuracy of GPS is difficult to guarantee, and the cumulative error of IMU may also cause errors in the path of the lawn mower. In other fields, methods of fusion perception of vision and lidar have been proposed, but lidar is not applicable in the field of intelligent lawn mower robots due to its cost and other factors. As for environmental perception using visual sensors, some researchers have proposed an intelligent lawn mower robot based on panoramic images, which improves the environmental perception ability of the lawn mower robot by building a panoramic map. In addition, there are also proposals to use cameras and ultrasound to perceive obstacles and avoid them.

[0004] However, the above solutions can only cope with simple environments. In complex outdoor environments, vision, ultrasonic and other sensors need to face changes in ambient light caused by factors such as uneven grass, shadows of trees and buildings, dark clouds, and cloudy days. These factors will greatly affect the sensor's perception effect. In addition, a series of dynamic and static factors such as moving pets, walking adults, running children, tables and chairs, etc. will make it difficult for these isolated sensors to provide enough information for the processor to process accordingly to obtain better perception effects, and it is difficult for the decision system to comprehensively handle too many environmental parameters. Summary of the invention

[0005] In order to overcome the difficulty of environmental perception faced by the above-mentioned intelligent lawn mowing robot in complex outdoor environments, the present invention provides an environmental perception method and system for an intelligent lawn mowing robot, which divides the perceived environment into different scenes through the fusion perception of multiple sensors, and effectively converts complex environmental information into a limited number of different scenes as input to the decision-making system.

[0006] The present invention provides an environment perception method for an intelligent lawn mowing robot, comprising installing a camera, an inertial measurement unit IMU, a global navigation satellite system GNSS, an odometer, a water drop sensor, a photosensitive sensor, and an ultrasonic sensor on the intelligent lawn mowing robot, wherein:

[0007] The camera is used to perceive the semantic information of the environment and perform multi-target detection;

[0008] Ultrasonic sensors are used to supplement the perception of the camera from normal perception to inability to perceive in low light conditions, and perform target detection through decision-level fusion;

[0009] The photosensitive sensor is used to obtain the light intensity and determine the fusion perception weight of the camera and ultrasound according to the light intensity, so that the target can be better detected in various lighting environments;

[0010] The water drop sensor is used to determine the rain condition in the environment and compare it with the threshold value as the perception parameter for whether to continue working;

[0011] IMU, GNSS, and odometer are used for fusion positioning to achieve boundary judgment;

[0012] The specific steps include:

[0013] S1. The intelligent lawn mowing robot obtains the environmental rain conditions, compares them with the set threshold and determines whether to continue working;

[0014] S2, the intelligent lawn mowing robot obtains the ambient light intensity and uses the camera and ultrasonic module to integrate to achieve environmental perception and target detection and tracking;

[0015] S3, the intelligent lawn mowing robot obtains its current high-precision position information through a fusion positioning algorithm based on Kalman filtering;

[0016] S4, performing scene judgment based on the information obtained in step 2 and step 3;

[0017] S5. Make a decision using the corresponding scene information as input to the decision system.

[0018] Further, the step S2 specifically includes:

[0019] S21: Obtain data from the photosensor to obtain light intensity p;

[0020] S22: Compare p with the set light intensity threshold w;

[0021] S23: If p>w, the camera will be used for target detection and tracking to achieve the detection of multiple targets in the environment and the extraction of semantic information, and compared with the pre-built semantic map to make the next decision;

[0022] S24: If p < w, it indicates that the ambient light intensity is low, and the camera cannot effectively obtain environmental information such as semantics. There is a transition period before the camera completely fails. At this time, target detection using the fusion of the camera and ultrasonic waves is performed, and different weights are assigned to the recognition results of the two according to the size of p for decision-level fusion to achieve obstacle recognition;

[0023] S25: Output the fused result to achieve target detection and tracking.

[0024] Specifically, the fused result M can be represented by formula (1):

[0025] M = f1(p)x1 + f2(p)x2 (1)

[0026] Where x1 represents the target detection result of the camera, f1(p) represents the weight value of the camera target detection result with respect to the change in the light intensity p, x2 represents the target detection result of the ultrasonic wave, and f2(p) represents the weight value of the ultrasonic wave target detection result with respect to the change in the light intensity p.

[0027] Furthermore, when the camera and ultrasonic wave perform target detection, joint calibration is required to achieve spatial and temporal synchronization between the two.

[0028] Furthermore, the step S3 includes:

[0029] Obtain the data of GNSS, preprocess the data, obtain the IMU data and process it. When it is not the initial moment, perform error compensation on the IMU according to the feedback of the Kalman update. Obtain the odometer data and process it.

[0030] Where the state and measurement models X′ and Z of the Kalman filter can be described as

[0031] X′ = AX + W (2)

[0032] Z = HX + ν (3)

[0033] X represents the state parameter vector of GNSS, IMU, and odometer. A represents the corresponding state transition matrix, which is a conjecture model for the target state transition. W is the corresponding system error. Z represents the measurement observation vector. H represents the conversion matrix from the state parameter to the observation parameter. ν represents the observation error.

[0034] Then perform the Kalman filtering steps on it, including two parts: prediction and update.

[0035]

[0036] P k|k-1 = AP k-1|k-1 A T + Q k(5)

[0037]

[0038]

[0039] P k|k =[IK k H k ]P k|k-1 (8)

[0040] In the above formulas, formula (4) represents the state one-step prediction, formula (5) represents the state one-step prediction mean square error, formula (6) represents the filter gain, formula (7) represents the state estimation, and formula (8) represents the state estimation covariance; where represents the state prediction at time k-1, represents the state estimation at time k-1, P k|k-1 is the covariance forecast at time k-1, K k is the gain at time k, and P k|k They correspond to the state update and covariance update at time k, Q k is the process excitation noise covariance (covariance of the system process), which represents the error between the state transition matrix and the actual process, R k is the measured covariance, which can generally be observed and is a known condition of the filter.

[0041] Further, the step S4 specifically includes:

[0042] According to the information of the ambient brightness, map boundary, and environmental obstacles obtained in step S1, step S2, and step S3, the environment is divided into three levels of scene sets L1, L2, and L3. L1 includes low brightness scenes and normal brightness scenes, L2 includes boundary scenes and non-boundary scenes, and L3 includes obstacle-free scenes, simple obstacle scenes, and multiple dynamic obstacle scenes.

[0043] Furthermore, the priority and impact range of the three levels of scene sets are L1>L2>L3, that is, the L1 scene is judged first, and the L1 scene affects the L2 scene set, and the L2 scene affects the L3 scene set. For a better understanding, an example is used here to illustrate. Assume that L1 contains two scenes, L11 and L12, and L2 contains two scenes, L21 and L22. After perception, it is first judged that the environment is the L11 scene. At this time, the L2 scene is perceived and judged according to the decision corresponding to the L11 scene. If the perception is the L21 scene, the L21 decision is made under the decision corresponding to the L11 scene.

[0044] Furthermore, the map boundary includes the outer boundary of the map and the inner area boundary such as the boundary of trees, flower beds, houses and other obstacles that cannot be moved by humans, that is, the boundary when building the map. The recognition process includes using the camera and the prior semantic map for judgment and using the fused positioning information and the constructed map for boundary judgment.

[0045] To further distinguish the L3 scenario, according to the target information obtained in step S2, the number of static targets is set to N s , the number of dynamic targets is set to N d , N s , N d Compare with the set corresponding threshold to achieve L3 obstacle scene distinction.

[0046] An environment perception system for an intelligent lawn mowing robot, comprising: an environment perception module, a position positioning module, a data analysis module, a data recording module, and a data transmission module;

[0047] The environmental perception module is used to obtain environmental data around the intelligent lawn mowing robot, including rainfall conditions, light intensity data, semantic information, obstacle information, etc.

[0048] The location positioning module obtains high-precision positioning information of the intelligent lawn mowing robot, including odometer information, IMU information, and GNSS information;

[0049] The data analysis module is used to identify the scene of the complex environment around the intelligent mowing robot to obtain the result of the current scene judgment, and at the same time establish rule-based decision output information;

[0050] Data recording module, used to record sensor data and analysis result initialization information;

[0051] The data transmission module is used to transmit the analyzed data and recorded data to the main controller, including rainfall conditions, light intensity data, semantic information, obstacle information, odometer information, IMU information, GNSS information, and scene judgment information.

[0052] Compared with the prior art, the present invention has the following advantages:

[0053] 1. Traditional intelligent lawn mowing robots do not have the fusion perception function of complex environments. The present invention makes full use of the camera, ultrasonic wave, water drop sensor, photosensitive sensor, IMU, GNSS, odometer and other sensors carried by the intelligent lawn mowing robot to fuse information, thereby improving the perception ability of the environment.

[0054] 2. The present invention performs weighted fusion on the detection results of the camera and ultrasound based on the data measured by the photosensitive sensor to improve the obstacle perception capability under low light conditions.

[0055] 3. The Kalman-based adaptive fusion method designed by the present invention ensures the correct weighting of data and improves the accuracy of positioning

[0056] 4. The present invention divides the complex environment into three levels according to the obtained sensor data, and subdivides the complex environment, which can be used as the input of the decision-making system more concisely and clearly compared with the numerous sensor data. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Attached Figure 1 A schematic diagram of the structure of a device used in an environment perception method for an intelligent lawn mowing robot according to Embodiment 1 of the present invention;

[0058] Attached Figure 2 This is a schematic diagram of the multi-sensor environmental state perception principle of an environmental perception method for an intelligent lawn mowing robot according to the first embodiment of the present invention;

[0059] Attached Figure 3 A schematic diagram of a self-state perception method of an environment perception method for an intelligent lawn mowing robot according to Embodiment 1 of the present invention;

[0060] Attached Figure 4 A multi-scenario hierarchical perception flow chart of an environment perception method for an intelligent lawn mowing robot provided in Embodiment 1 of the present invention;

[0061] Attached Figure 5 For the attached Figure 2 The camera and ultrasonic fusion perception flow chart;

[0062] Attached Figure 6 For the attached Figure 3 GNSS, IMU and odometer fusion positioning flow chart;

[0063] Attached Figure 7 A schematic diagram of the structure of an environment perception system for an intelligent lawn mowing robot provided in the second embodiment of the present invention;

[0064] Attached Figure 8 A schematic diagram of the structure of an environment perception system for an intelligent lawn mowing robot provided in Embodiment 3 of the present invention; DETAILED DESCRIPTION

[0065] In order to make the technical solution of the present invention clearer, the technical solution provided by the present invention will be described in detail in conjunction with specific embodiments, and the present invention will be further described in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only partial embodiments of the present invention, which are used to explain the present invention, rather than to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0066] Embodiment 1

[0067] Attached Figure 1 This is a structural schematic diagram of an apparatus for an environment perception method and system for an intelligent lawn mowing robot as described in Example 1 of the present invention, comprising a camera unit 1, an IMU unit 2, a GNSS unit 3, an odometer unit 4, a water drop sensor unit 5, a photosensor unit 6, an ultrasonic sensor unit 7 and a controller.

[0068] in:

[0069] The camera is used to perceive the semantic information of the environment and perform multi-target detection;

[0070] Ultrasonic sensors are used to supplement the perception of the camera from normal perception to inability to perceive in low light conditions, and perform target detection through decision-level fusion;

[0071] The photosensitive sensor is used to obtain the light intensity and determine the fusion perception weight of the camera and ultrasound according to the light intensity, so that the target can be better detected in various lighting environments;

[0072] The water drop sensor is used to determine the rain condition in the environment and compare it with the threshold value as the perception parameter for whether to continue working;

[0073] IMU, GNSS, and odometer are used for fusion positioning to achieve boundary judgment;

[0074] This embodiment is applicable to the perception of complex lawn mower working environments. Unlike the common intelligent lawn mowers that use a set of perception solutions for any environment, this method integrates the information obtained by multiple sensors in a complex environment and divides the environment into three levels of scene sets as shown in the attached figure. Figure 4 As shown in the figure, the complex environment is subdivided into a limited number of scenes as the input of the decision-making system. The environmental perception is mainly divided into two parts: 1. Figure 2 1. External environment perception as shown; 2. Figure 3 The self-state awareness shown.

[0075] The overall solution includes the following steps:

[0076] S110. The intelligent lawn mowing robot obtains data through a water droplet sensor, processes it to obtain the rainfall value at the current moment, and compares it with a set threshold to determine whether the lawn mower should continue to work and make environmental perception decisions or return to the charging station for decisions.

[0077] S120. The intelligent lawn mowing robot obtains the environmental light intensity for scene judgment at the L1 level to determine whether the current scene meets the conventional brightness scene shown in the appendix. If it meets, it executes the corresponding decision and adopts a single camera perception decision to perform L2 boundary scene, L3 scene surrounding environment semantic information and obstacle information perception and decision-making. If it meets the low brightness environment scene, the camera and ultrasonic module are used for fusion to achieve environmental perception and target detection for L2 and L3 perception and decision-making. It should be noted that before the camera and ultrasonic perform environmental detection work, the camera and ultrasonic have been jointly calibrated to achieve spatial and temporal synchronization between the two, so that the target information detected by the two can be corresponding, and the target result detected by the ultrasonic under low light can compensate the detection result of the camera. The overall process is as shown in the appendix. Figure 4 shown. Figure 2 and Figure 5 shown.

[0078] S121. Obtain the data of the photosensitive sensor to get the light intensity p.

[0079] S122. Compare p with the set light intensity threshold w.

[0080] S123. If p > w, the camera will be used for target detection and tracking.

[0081] S124. If p < w, the camera and ultrasonic will be used for target detection, and different weights will be assigned to the results of the two according to the size of p for fusion.

[0082] S125. Output the fused result to achieve target detection and tracking.

[0083] Specifically, the fused detection result M can be expressed by formula (1):

[0084] M = f1(p)x1 + f2(p)x2 (1)

[0085] where x1 represents the target detection result of the camera, f1(p) represents the weight value of the camera target detection result changing with the light intensity p, x2 represents the target detection result of the ultrasonic, and f2(p) represents the weight value of the ultrasonic target detection result changing with the light intensity p.

[0086] S130, the intelligent lawn mowing robot obtains the data information of GNSS, IMU and odometer for relevant preprocessing, then obtains the current high-precision position information of itself based on the fusion positioning algorithm of Kalman filter, and compensates the error of IMU according to the feedback of Kalman update. Figure 3 This describes the process of self-position fusion perception. Figure 6 Shown is Figure 3 The specific flow chart of fusion positioning is described.

[0087] S131, the state and measurement models of the system can be described as

[0088] X′=AX+W (2)

[0089] Z=HX+ν (3)

[0090] X represents the state parameter vector of GNSS, IMU and odometer, A represents the corresponding state transfer matrix, which is a conjecture model for the target state transition, W is the corresponding system error, Z represents the measurement observation vector, H represents the conversion matrix from state parameters to observation parameters, and ν represents the observation error.

[0091]

[0092] X=merge(X IMU ,X 里程计 ,X GNSS ) (10)

[0093]

[0094] W is the corresponding error set, X represents X IMU ,X 里程计 ,X GNSS The combination of the three parameters, Z GNSS and Z 里程计 represents the observation parameters of GNSS and odometer, H GNSS and H 里程计 Represents the conversion matrix from the state parameters of GNSS and odometer to the observation parameters. GNSS and v 里程计 are their respective observation errors.

[0095] represents the transformation matrix from the inertial sensor frame coordinate system to the Earth-centered Earth-fixed coordinate system, ω a ω g represents noise, and the superscript T represents the transpose of the matrix. A is the state transfer matrix, as shown below:

[0096]

[0097] Here, the × symbol indicates a skew-symmetric matrix, that is, if there is V×, it means it is a skew-symmetric matrix of V, E represents the unit matrix, the subscript e represents the Earth-centered Earth-fixed coordinate system, and the subscript i represents the inertial system. is the rotation angular velocity vector of the Earth-centered Earth-fixed coordinate system relative to the projection on the inertial system, f b is the specific force output by the accelerometer in the inertial sensor frame

[0098] merge(.) represents an operator that unifies common parameters and combines different parameters in variables, where X IMU ,X 里程计 ,X GNSS They represent the corresponding estimated parameters, as follows

[0099]

[0100]

[0101] X 里程计 =[δs] (15)

[0102] In the above parameters, b g 、b a , δs and δG are respectively g ,ω a , ε s and ε δG The random walk process, δt r , δZ wet The model is white noise, δN is set as a constant, and δZ is used after atmospheric correction wet The initial values ​​of and δI are set to zero. More specifically: the prefix symbol δ indicates the correction of the subsequent parameters, and φ describes the misalignment error vector between the Earth-centered Earth-fixed coordinate system and the coordinate system of the actual inertial sensor. δν e ,δr e e represents the velocity and position error vector in the Earth-centered Earth-fixed coordinate system, and b represents g ,b a are the bias of the gyroscope and accelerometer in the IMU respectively. r represents the receiver clock offset, G describes the clock deviation between various GNSS systems such as the Galileo satellite positioning system, the Beidou positioning system and the GPS, I represents the propagation delay of the ionosphere at the first frequency, and Z wet represents the tropospheric wet delay, and N represents the integer carrier phase ambiguity. In order to describe s, the speed measurement of the smart lawn mower is first described as in formula (16).

[0103]

[0104] Here, formula (16) actually describes the observation equation of the odometer. represents the transformation matrix from the sensor coordinate system to the vehicle coordinate system, It is the velocity parameter obtained by IMU in the sensor coordinate system. is the rotation angular velocity vector of the sensor coordinate system relative to the Earth-centered Earth-fixed coordinate system projected on it, The lever arm vector representing the center of the odometry in the sensor coordinate system. is the speed measurement value obtained by the odometer in the lawn mower body coordinate system, ε v represents the observation error, and s represents the scaling factor in the speed measurement of the smart lawn mower.

[0105] Z GNSS Indicates the observation information of GNSS, H GNSS represents the conversion matrix corresponding to the GNSS state parameters, v GNSS Represents the observation error of GNSS observation. Since satellite positioning is performed by multiple satellites, Z GNSS =(Z GNSS1 , Z GNSS2 、···、Z GNSSi ), i represents the number of GNSS, and similarly H GNSS and v GNSS It is also composed of this structure, where the information of the i-th GNSS is expressed as follows:

[0106]

[0107]

[0108]

[0109] in represents the pseudorange and carrier phase observations, represents the direction cosines of the unit vector from the receiver to the satellite, represents the lever arm vector of the receiver antenna phase center relative to the IMU center in the Earth-centered Earth-fixed coordinate system, γ i is the frequency-dependent coefficient, Yes and Z GNSSi The correlation coefficient, A i =[0 1 0], yes The corresponding observation error.

[0110] S132: Perform Kalman filtering, including prediction and update.

[0111] Perform a temporal update of the state estimate and the estimated error covariance:

[0112]

[0113] P k|k-1 =AP k-1|k-1 A T +Q k (5)

[0114] Then the measurement update of the state estimate and the estimation error covariance is performed:

[0115]

[0116]

[0117] P k|k =[IK k H k ]P k|k-1 (8)

[0118] In the above formulas, formula (4) represents the state one-step prediction, formula (5) represents the state one-step prediction mean square error, formula (6) represents the filter gain, formula (7) represents the state estimation, and formula (8) represents the state estimation covariance; where represents the state prediction at time k-1, represents the state estimation at time k-1, P k|k-1 is the covariance forecast at time k-1, K k is the gain at time k, and P k|k They correspond to the state update and covariance update at time k, Q k is the process excitation noise covariance (covariance of the system process), which represents the error between the state transition matrix and the actual process, R k is the measured covariance, which can generally be observed and is a known condition of the filter.

[0119] S140, Attachment Figure 4Starting from the overall scene division, it includes three levels of scene sets, where the L1 scene includes the L2 scene set, and the L2 scene includes the L3 scene set. The three levels of scene influence range L1>L2>L3. First, the light intensity information is detected as input to obtain the specific scene of the current L1 level. Assuming that the detected scene is a regular brightness scene, the decision of this scene is adopted, and the pure visual detection method is used to detect the semantic information and obstacle targets of the surrounding environment; then the L2 boundary scene detection is started, and the high-precision positioning information obtained by the previous semantic information and fusion positioning is used as the input of the scene detection to obtain whether the current scene is a boundary scene. Assuming that the detected scene is a non-boundary scene, the corresponding non-boundary scene decision is executed. Finally, the L3 scene is judged, and the target detection information and semantic information obtained by vision are used as input. The detected dynamic and static obstacle information is compared with the set threshold for judgment. Assuming that it is a multi-dynamic obstacle scene, the corresponding multi-dynamic obstacle scene decision is executed.

[0120] In the above process, the definition of map boundaries includes the outer boundary of the map and the inner boundary of the area, such as trees, flower beds, houses and other obstacles that cannot be moved by humans, that is, the boundary when building the map. The recognition process includes using the camera and the prior semantic map to make judgments and the fusion positioning information obtained and the constructed map to make judgments.

[0121] Embodiment 2

[0122] Reference Figure 7 The second embodiment of the present invention provides an environment perception system for an intelligent lawn mowing robot, including: an environment perception module, a position positioning module, a data analysis module, a data recording module, and a data transmission module.

[0123] The environmental perception module is used to obtain environmental data around the intelligent lawn mowing robot, including rainfall conditions, light intensity data, semantic information, obstacle information, etc.

[0124] The location positioning module obtains high-precision positioning information of the intelligent lawn mowing robot, including odometer information, IMU information, and GNSS information;

[0125] The data analysis module is used to identify the complex environment around the intelligent mowing robot, establish rule-based decision output information, and obtain the results of the current scene judgment;

[0126] Data recording module, used to record sensor data and analysis result initialization information;

[0127] The data transmission module is used to transmit the analyzed data and recorded data to the main controller, including rainfall conditions, light intensity data, semantic information, obstacle information, odometer information, IMU information, GNSS information, and scene judgment information.

[0128] On the basis of the above technical solutions, the device further includes: a noise data processing module, which is used to denoise the sensor data with noise caused by the environment.

[0129] Embodiment 3

[0130] Reference Figure 8 The third embodiment of the present invention provides an environment perception system for an intelligent lawn mowing robot, including: an environment perception device, a position positioning device, a data analysis device, a communication device, and a main control system.

[0131] Among them, the environmental perception device is used to obtain environmental data around the intelligent lawn mowing robot, including rainfall conditions, light intensity data, semantic information, obstacle information, etc.; through error processing of the data, feature extraction and analysis processing, information for judging the scene category is obtained.

[0132] The position positioning device obtains high-precision positioning information of the intelligent lawn mowing robot, including odometer information, IMU information, and GNSS information; processes the data and performs Kalman filtering fusion to obtain high-precision positioning information.

[0133] The data analysis device is used for scene recognition of the complex environment around the intelligent lawn mowing robot, and performs scene judgment according to the input scene judgment parameters to obtain the result of the current scene judgment, and at the same time establishes rule-based decision output information;

[0134] The communication device is used for data transmission and communication between devices, realizes information synchronization and real-time update, integrates multiple communication methods and communication protocols to meet the needs of different scenarios, and at the same time ensures the confidentiality and integrity of data during the communication process to avoid information leakage.

[0135] It should be noted that in the above-mentioned embodiments of the environmental perception device for intelligent lawn mowing robots, the various modules included are only divided according to functional logic and are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other and are not used to limit the scope of protection of the present invention.

[0136] Finally, the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention is described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. An environment perception method for an intelligent lawn mowing robot, characterized in that: The devices used include a camera, an Inertial Measurement Unit (IMU), a Global Navigation Satellite System (GNSS), an odometer, a water droplet sensor, a photosensitive sensor, and an ultrasonic sensor installed on an intelligent lawn mowing robot, where: The camera is used to perceive the semantic information of the environment and perform multi-object detection; The ultrasonic sensor is used for supplementary perception during the period when the camera changes from normal perception to non-perception under low light conditions, and target detection is carried out through decision-level fusion; The photosensitive sensor is used to obtain the light intensity and determine the fusion perception weights of the camera and the ultrasonic sensor according to the light intensity, so as to detect targets in various lighting environments; The water droplet sensor is used to judge the rainfall situation in the environment and compare it with a threshold value as a perception parameter for whether to continue working; The IMU, GNSS, and odometer are used for fusion positioning to achieve boundary judgment; Specifically, it includes the following steps: S1. The intelligent lawn mowing robot obtains the environmental rainfall situation and compares it with a set threshold value to judge whether to continue working; S2. The intelligent lawn mowing robot obtains the environmental light intensity and uses the camera and ultrasonic modules for fusion to achieve environmental perception, target detection, and tracking; specifically including: S21: Obtain the data of the photosensitive sensor to get the light intensity p; S22: Compare p with the set light intensity threshold value w; S23: If p > w, the camera will be used for target detection and tracking to achieve the detection of multiple environmental targets and the extraction of semantic information, and compare it with the pre-constructed semantic map to make the next decision; S24: If p < w, it indicates that the environmental light intensity is low and the camera cannot effectively obtain semantic environmental information. There is a transition period before the camera completely fails. At this time, the camera and the ultrasonic sensor are used for fusion target detection, and different weights are assigned to the recognition results of the two according to the size of p for decision-level fusion to achieve the recognition of obstacles; S25: Output the fused result to achieve target detection and tracking; specifically, the fused result M is represented by formula (1): M = f1(p)x1 + f2(p)x2 (1) Where x1 represents the target detection result of the camera, f1(p) represents the weight value of the camera target detection result changing with the light intensity p, x2 represents the target detection result of the ultrasonic sensor, and f2(p) represents the weight value of the ultrasonic target detection result changing with the light intensity p; S3. The intelligent lawn mowing robot obtains its own high-precision position information through a fusion positioning algorithm based on Kalman filtering; S4. Perform scene judgment based on the information obtained in steps S2 and S3; specifically including: According to the information of environmental brightness, map boundaries, and environmental obstacles obtained in steps S1, S2, and S3, the environment is divided into three levels of scene sets L1, L2, and L3; L1 includes low-brightness scenes and regular-brightness scenes, L2 includes boundary scenes and non-boundary scenes, and L3 includes obstacle-free scenes, simple obstacle scenes, and multi-dynamic obstacle scenes; The priorities and influence ranges of the three levels of scene sets are L1 > L2 > L3, that is, the L1 scene is judged first, and the L1 scene will affect the L2 scene set, and the L2 scene affects the L3 scene set; For the map boundary including the map outer boundary and the inner area boundary, which are the boundaries of obstacles that cannot be moved by humans, i.e. the boundaries when building the map, the recognition process includes using the camera and the prior semantic map to make judgments and using the fused positioning information and the built map to make boundary judgments; For the distinction of L3 scenarios, the number of static targets is set to N according to the target information obtained in step S2. s , the number of dynamic targets is set to N d , N s , N d Compare with the set corresponding threshold to realize the distinction of L3 obstacle scene; S5. Make a decision using the corresponding scene information as input to the decision system.

2. The environment perception method for an intelligent lawn mowing robot according to claim 1, characterized in that: When the camera and ultrasound are used for target detection, joint calibration is required to achieve spatial and temporal synchronization between the two.

3. The environment perception method for an intelligent lawn mowing robot according to claim 1, characterized in that: The step S3 comprises: Obtain GNSS data and pre-process the data; obtain IMU data and process it; when it is not at the initial moment, perform error compensation on the IMU based on the feedback of Kalman update; obtain odometer data and process it; The state and measurement model of the Kalman filter is X ′ and Z can be described as X ′ =AX+W (2) Z=HX+ν (3) X represents the state parameter vectors of GNSS, IMU, and odometer, A represents the corresponding state transfer matrix, which is a conjecture model for the target state transition, W is the corresponding system error, Z represents the measurement observation vector, H represents the conversion matrix from state parameters to observation parameters, and ν represents the observation error; Then perform the Kalman filtering step, including prediction and update. P k|k-1 =AP k-1|k-1 From T +Q k (5) P k|k =[I-K k H k ]P k|k-1 (8) In the above formulas, formula (4) represents the state one-step prediction, formula (5) represents the state one-step prediction mean square error, formula (6) represents the filter gain, formula (7) represents the state estimation, and formula (8) represents the state estimation covariance; where represents the state prediction at time k-1, represents the state estimation at time k-1, P k|k-1 is the covariance forecast at time k-1, K k is the gain at time k, and P k|k They correspond to the state update and covariance update at time k, Q k is the process excitation noise covariance, which represents the error between the state transition matrix and the actual process, R k is the measured covariance, which is a known condition of the filter.

4. A system for implementing the environment perception method for an intelligent lawn mowing robot according to claim 1, characterized in that: include: Environmental perception module, location positioning module, data analysis module, data recording module, data transmission module; The environmental perception module is used to obtain environmental data around the intelligent lawn mowing robot, including rainfall conditions, light intensity data, semantic information, and obstacle information; The location positioning module obtains high-precision positioning information of the intelligent lawn mowing robot, including odometer information, IMU information, and GNSS information; The data analysis module is used to identify the scene of the complex environment around the intelligent mowing robot to obtain the result of the current scene judgment, and at the same time establish rule-based decision output information; A data recording module is used to record sensor data and analysis result initialization information; The data transmission module is used to transmit the analyzed data and recorded data to the main controller, including rainfall conditions, light intensity data, semantic information, obstacle information, odometer information, IMU information, GNSS information, and scene judgment information.

5. The system according to claim 4, characterized in that Also includes: The noise data processing module is used to denoise the sensor data with noise caused by the environment.

Images