Method and device for detecting position change of reference station and readable storage medium

Abstract

The embodiment of the invention discloses a reference station position change detection method and device and a readable storage medium, when a self-moving device is switched from a non-working state to a working state, the self-moving device travels for a certain distance, in the traveling process, a visual image and RTK positioning are obtained, and the self-moving device is repositioned according to the visual image and the RTK positioning. When the relocation is successful, it is determined that the base station is not moved. By adopting the scheme, the self-moving equipment judges whether the base station is moved or not by combining the visual image and the motion characteristics of the self-moving equipment, an additional sensor does not need to be installed, the situation that the self-moving equipment is abnormal in positioning after the base station is moved is avoided, and the purpose of improving the positioning accuracy of the self-moving equipment is achieved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of lawn mowers, and in particular to a method and device for detecting position changes of a reference station, and a readable storage medium. Background Art

[0002] Real-time kinematic (RTK) positioning technology is a high-precision positioning method based on the Global Navigation Satellite System (GNSS). Visual Simultaneous Localization and Mapping (SLAM) technology is a robot autonomous positioning and environment map construction method based on visual sensors.

[0003] In the field of robotics, RTK technology and visual SLAM technology are often combined to position the robot, thereby making full use of the advantages of both. However, once the base station is moved, the robot positioning will be abnormal. Therefore, it is necessary to ensure that the base station is not moved. In a common way, a gyroscope or acceleration sensor is installed on the base station. If the sensor detects that the base station moves significantly, it is considered that the base station has been moved.

[0004] However, the above detection method requires installing additional sensors for the reference station, which is costly. Summary of the invention

[0005] The embodiments of the present application provide a method, device and readable storage medium for detecting the position change of a base station, which determines whether the base station has been moved by combining visual images and the motion characteristics of a self-moving device, without installing additional sensors, thereby improving the accuracy of positioning the self-moving device.

[0006] In a first aspect, an embodiment of the present application provides a method for detecting a change in the position of a reference station, which is applied to a self-moving device, and the method includes:

[0007] When the self-moving device is switched from a non-working state to a working state, controlling the self-moving device to move;

[0008] During the movement of the self-moving device, controlling the self-moving device to obtain visual images and real-time dynamic carrier phase differential RTK positioning;

[0009] repositioning the self-moving device based on the visual image and the RTK positioning;

[0010] When the self-moving device is successfully repositioned, it is determined that the reference station has not been moved, and the reference station is used to provide a differential signal to the self-moving device, so that the self-moving device obtains RTK positioning according to the differential signal.

[0011] In a second aspect, an embodiment of the present application provides a device for detecting a change in the position of a reference station, which is integrated in a self-moving device, and the device includes:

[0012] A traveling module, used for controlling the self-moving device to travel when the self-moving device is switched from a non-working state to a working state;

[0013] An acquisition module, used for controlling the self-mobile device to acquire visual images and real-time dynamic carrier phase differential RTK positioning during the movement of the self-mobile device;

[0014] a processing module, configured to perform repositioning of the self-moving device based on the visual image and the RTK positioning;

[0015] The determination module is used to determine that the reference station has not been moved when the self-moving device is successfully relocated, and the reference station is used to provide a differential signal to the self-moving device so that the self-moving device obtains RTK positioning according to the differential signal.

[0016] In a third aspect, an embodiment of the present application provides a self-mobile device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the self-mobile device implements the method described in the first aspect or various possible implementations of the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed by a processor, they are used to implement the method described in the first aspect or various possible implementation methods of the first aspect.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product comprising a computing program, and when the computer program is executed by a processor, the method described in the first aspect or various possible implementation methods of the first aspect is implemented.

[0019] The detection method, device and readable storage medium for the change in the position of the base station provided in the embodiments of the present application are such that when the self-moving device switches from a non-working state to a working state and travels a certain distance, visual images and RTK positioning are acquired during the travel, and the self-moving device is repositioned based on the visual images and RTK positioning. When the repositioning is successful, it is determined that the base station has not been moved. With this solution, the self-moving device determines whether the base station has been moved by combining the visual image and the motion characteristics of the self-moving device, without the need to install additional sensors, thereby avoiding abnormal positioning of the self-moving device caused by the movement of the base station, thereby achieving the purpose of improving the positioning accuracy of the self-moving device. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 It is a schematic diagram of an implementation environment of a method for detecting a change in a reference station position provided in an embodiment of the present application;

[0022] Figure 2 is a flow chart of a method for detecting a change in a reference station position provided in an embodiment of the present application;

[0023] Figure 3 It is a flow chart of repositioning in the method for detecting the change of the reference station position provided in an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of an environment map in a method for detecting a change in a reference station position provided in an embodiment of the present application;

[0025] Figure 5 It is a flowchart of projecting feature points in an environment map to a current visual image during the first repositioning in the method for detecting a reference station position change provided in an embodiment of the present application;

[0026] Figure 6 is another flow chart of a method for detecting a change in a reference station position provided in an embodiment of the present application;

[0027] Figure 7 is a schematic diagram of a detection device for a reference station position change provided in an embodiment of the present application;

[0028] Figure 8 A schematic diagram of the structure of a self-moving device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In the field of robotics, positioning technology is particularly important. Accurate positioning results can greatly ensure that mobile devices can successfully complete tasks and are safe. Common positioning technologies include VSLAM positioning technology, RTK positioning, and VSLAM positioning integrated with RTK. Below, these positioning methods are introduced separately.

[0030] 1. VSLAM positioning technology.

[0031] VSLAM technology is a method for autonomous robot positioning and environmental mapping based on visual sensors. The goal of VSLAM technology is to estimate the robot's position and the map of the environment in an unknown environment through the robot's own movement and observation. VSLAM technology mainly uses cameras as sensors and realizes positioning and mapping functions by processing the image sequence taken by the camera. VSLAM technology includes the following steps:

[0032] (1) Feature extraction and matching.

[0033] The robot extracts distinguishing feature points from the images taken by the camera, and then matches the feature points between consecutive images to calculate the relative motion between adjacent frames.

[0034] (2) Motion estimation.

[0035] After the robot obtains the correspondence between feature points of adjacent images through feature point matching, it calculates the relative motion between adjacent images based on these correspondences. This process is called motion estimation. Commonly used methods include visual odometry (VO) and optical flow.

[0036] (3) Posture optimization.

[0037] Since there will be errors in the motion estimation process, the entire trajectory of the robot needs to be optimized to reduce the cumulative error. Commonly used optimization methods include nonlinear optimization, graph optimization, etc.

[0038] (4) Map construction.

[0039] During the robot's motion, the observed feature points are projected into the global world coordinate system based on the results of motion estimation to form a three-dimensional feature point map, namely the environment map.

[0040] (5) Closed-loop detection.

[0041] When the robot returns to the area it has passed before, the closed loop detection module identifies the duplication and introduces the closed loop information into the position optimization to eliminate the accumulated error. The area the robot has passed before includes but is not limited to the area where the base station is located and the area outside the area where the base station is located.

[0042] (6) Relocation.

[0043] Similar to closed-loop detection, when the robot plans a path and moves according to the environment map, if the positioning is lost, the current visual data is re-matched with the data of the environment map to quickly restore its own posture. The reasons for the robot's positioning loss include: the robot is moved while running, the robot is moved to another place and then restarted after being turned off. Commonly used visual relocalization methods include Bag of Words (BOW) model, etc.

[0044] BOW is a relocalization method based on the bag-of-words model. A visual dictionary is constructed in advance using training data to map feature points to visual words. During the relocalization process, the robot determines the similarity between the current frame and each key frame contained in the environment map, and obtains the relocalization result based on the key frame with the greatest similarity. Among them, the current frame is the image frame taken at the current position after the robot loses its positioning.

[0045] In the process of determining the similarity, the robot determines the sparse vector of the current frame and the sparse vector of each key frame. For the sake of clarity, the sparse vector of the current frame is referred to as the first vector, and the sparse vector of the key frame is referred to as the second vector. The robot determines the similarity between the first vector and each second vector, thereby obtaining the similarity between the current frame and each key frame.

[0046] The BOW method has good relocalization performance, especially in large-scale environments.

[0047] 2. RTK positioning.

[0048] RTK positioning is a high-precision positioning method based on GNSS. RTK positioning technology uses the carrier phase observation value in the satellite signal to achieve centimeter-level positioning accuracy, usually with an error of less than 2 centimeters (cm). Therefore, RTK positioning technology is widely used in surveying, drones, autonomous driving and other fields.

[0049] RTK positioning has certain limitations. For example, it is necessary to set up a fixed base station and a communication device to transmit the error information calculated by the base station to the robot in order to achieve centimeter-level positioning accuracy. This positioning refers to positioning relative to the base station. If the base station moves, the robot's output positioning will also move accordingly, resulting in positioning deviation.

[0050] In addition, the positioning accuracy of RTK positioning will also be affected by factors such as multipath effects, occlusion, and signal interference.

[0051] 3. VSLAM positioning integrated with RTK.

[0052] RTK positioning has high accuracy, but is easily affected by the occlusion of houses and trees. VSLAM positioning technology has slightly lower accuracy, but is not affected by the occlusion of the surrounding environment. Combining RTK positioning and VSLAM positioning can make full use of the advantages of both, thereby improving the accuracy and reliability of positioning and mapping.

[0053] In a combined solution, RTK data is used as a high-precision global positioning reference, and VSLAM technology provides relative motion information, thereby overcoming the short signal interruption of RTK positioning or the instability of VSLAM technology in feature-deficient environments. The relative motion information of RTK data and VSLAM is used to estimate the position of the camera, that is, the position and posture of the robot, through Kalman filtering, nonlinear optimization and other solutions.

[0054] During RTK positioning, if the base station moves, it will cause positioning deviation of the robot. Therefore, when the robot uses RTK positioning technology, it is necessary to ensure that the base station is not moved. Common judgment methods include the following:

[0055] a. Monitor whether the coordinates of the base station are changing.

[0056] Monitor the coordinates of the base station itself for a long time. For example, based on the single-point positioning of the base station itself, monitor the position coordinates of the base station. If the position coordinates of the base station change significantly, it is considered that the base station has been moved. The disadvantage of this solution is that it can only be detected when the position coordinates of the base station change significantly. In other words, the base station can only be detected to be moved when the moving distance of the base station is greater than the threshold distance. If the moving distance of the base station is relatively small, such as 1 meter, 2 meters, etc., it is impossible to detect that the base station has been displaced.

[0057] b. Use a gyroscope or accelerometer to detect the motion state of the base station.

[0058] In this solution, a gyroscope or acceleration sensor is installed on the base station, and the motion state of the base station is detected by the gyroscope or acceleration sensor. If the base station is detected to move significantly, it is considered that the base station has been moved. The disadvantage of this solution is that it requires the installation of additional sensors, and it can only be detected when the base station is tilted or moves significantly. If the base station is slightly translated, it is impossible to detect that the base station has moved.

[0059] c. Use geomagnetic sensors to detect the motion status of the base station.

[0060] In this solution, a geomagnetic sensor is installed on the base station to detect changes in geomagnetism. When the base station is moved, the magnetic field around the base station will change. When the geomagnetic sensor detects changes in the magnetic field around the base station, it is considered that the base station has been moved. The disadvantage of this solution is that it requires additional sensors to be installed on the base station, and it can only be detected when the base station angle changes.

[0061] d. Combined with the position of the robot base station, determine whether the position of the reference station has changed.

[0062] In this solution, the robot enters the base station and determines the size of the deviation between the current position and the base station position. When the deviation is relatively large, the user is prompted to confirm whether the base station position has changed or the reference station position has changed. The disadvantage of this solution is that the robot needs to be located at the base station. Even so, if the RTK signal at the base station is weak, it cannot be judged. Moreover, after the user moves the base station, for example, the robot is specifically a lawn mowing robot, and the user's home has multiple lawns, the user moves the base station from one lawn to another to facilitate the robot to automatically mow the lawn. At this time, when the robot determines that the deviation between the current position and the base station position is relatively large, manual intervention is required to make a judgment, which is not smart enough.

[0063] Based on this, the embodiments of the present application provide a method, device and readable storage medium for detecting the position change of a base station, which determines whether the base station has been moved by combining visual images and the motion characteristics of a self-moving device, without installing additional sensors, thereby improving the accuracy of positioning the self-moving device.

[0064] Figure 1 Schematic diagram of the implementation environment of the method for detecting the change of the reference station position provided in the embodiment of the present application. Figure 1 The implementation environment includes: a self-mobile device 11, a base station 12, and a maintenance station 13. A network connection is established between the self-mobile device 11 and the base station 12, and the base station 12 is used to provide a differential signal for the self-mobile device 11. The self-mobile device 11 obtains positioning information using a positioning module during its movement, and corrects the positioning information using a differential signal to obtain RTK positioning.

[0065] The self-mobile device 11 is also called a mobile station, a robot, an autonomous mobile device, etc. It is a device that can autonomously move in a working area and perform tasks based on an environmental map, including but not limited to a lawn mowing robot, an outdoor navigation robot, an automatic spraying equipment, etc.

[0066] The reference station 12 is also called an RTK reference station, and is used to provide a differential signal for the self-moving device 11. In outdoor robot applications, the movement of the reference station 12 may cause abnormal RTK positioning of the self-moving device 11. When the positioning is abnormal, the position of the reference station 12 needs to be recalibrated or the map needs to be rebuilt.

[0067] The maintenance station 13 is also called a base station, a pedestal, a charging station, etc., and is usually arranged at a fixed position. The mobile device 11 performs maintenance work such as charging and cleaning at the maintenance station 13 .

[0068] Figure 2 1 is a flow chart of a method for detecting a change in the position of a reference station provided in an embodiment of the present application. The execution subject of this embodiment is a self-moving device. This embodiment includes:

[0069] 201. When the self-moving device is switched from a non-working state to a working state, control the self-moving device to move.

[0070] Generally, the self-mobile device may be suspended, dormant or shut down for a long time, during which the probability of the reference station being moved is higher. Moreover, when the self-mobile device is suspended, it is easier to determine whether the reference station has been moved. Therefore, in the embodiment of the present application, before the self-mobile device switches from a non-working state to a working state or works, it is first determined whether the reference station has been moved.

[0071] In one method, once the self-moving device switches from a non-working state to a working state, repositioning is triggered. During the repositioning process, the self-moving device moves forward, collects visual images and RTK positioning during the movement, and performs repositioning based on the visual images and RTK positioning.

[0072] In another way, after the mobile device switches from the non-working state to the working state, relocation is triggered only when a preset condition is met, and the preset condition is, for example, a change in the surrounding environment of the mobile device. That is, after the mobile device switches from the non-working state to the working state, the environment images before and after the state switching are compared to determine whether the surrounding environment has changed. Relocation is triggered only when the surrounding environment changes.

[0073] Among them, the surrounding environment of the self-moving device changes, and the possible reasons are as follows: Reason 1, the self-moving device moves during the non-working state. Movement includes the self-moving device not being displaced but changing in direction, being displaced but not changing in direction, being displaced and changing in direction, etc. For example, the user moves the self-moving device from place A to place B, and the environmental features of places A and B are different; for another example, the direction of the self-moving device changes, and the environmental images collected by visual sensors such as cameras are different at different directions.

[0074] Reason 2: The self-moving device did not move during the non-working state, but the surrounding environment changed. For example, when the self-moving device entered the non-working state, there were dynamic obstacles around it. When the self-moving device switched from the non-working state to the working state, the dynamic obstacles had left. For another example, the orientation of the self-moving device changed, and the environment images collected by visual sensors such as cameras were different in different orientations.

[0075] 202. During the movement of the self-moving device, control the self-moving device to obtain visual images and real-time dynamic carrier phase differential RTK positioning.

[0076] In the embodiment of the present application, a visual sensor, a positioning module, and an RTK sensor are provided on the mobile device, and the RTK sensor is used to receive a differential signal from a reference station. During the movement of the mobile device, the visual sensor is continuously used to collect visual images, for example, the mobile device collects visual images at a speed of 10 frames per second.

[0077] At the same time, the self-mobile device uses the RTK sensor to receive the differential signal from the base station, and uses the differential signal to correct the positioning information determined by the positioning module to obtain RTK positioning. For example, the self-mobile device receives the differential signal at a rate of 10 times per second, thereby generating 10 RTK positioning per second. RTK positioning corresponds to the visual image one by one.

[0078] 203. Reposition the self-moving device according to the visual image and the RTK positioning.

[0079] After the self-mobile device obtains a frame of visual image and the corresponding RTK positioning each time, the self-mobile device is repositioned according to the visual image and the RTK positioning.

[0080] 204. When the self-moving device is relocated successfully, determine that the reference station has not been moved.

[0081] After each relocation, if the relocation is successful, the self-mobile device considers that the base station has not been moved. After that, the self-mobile device performs tasks according to the environment map.

[0082] If the repositioning fails, the self-mobile device executes further judgment logic to determine whether the reference station is moved, etc. Alternatively, if the self-mobile device fails to reposition according to the visual image and RTK positioning, the default reference station is moved.

[0083] The method for detecting the change in the position of the base station provided in the embodiment of the present application is that when the self-moving device switches from a non-working state to a working state and travels a certain distance, a visual image and RTK positioning are obtained during the travel, and the self-moving device is repositioned according to the visual image and RTK positioning. When the repositioning is successful, it is determined that the base station has not been moved. With this solution, the self-moving device determines whether the base station has been moved by combining the visual image and the motion characteristics of the self-moving device, without the need to install additional sensors, thereby avoiding abnormal positioning of the self-moving device after the base station is moved, thereby achieving the purpose of improving the positioning accuracy of the self-moving device.

[0084] Optionally, in the above embodiment, the non-working state includes any one of the following states: a pause state, a sleep state or a shutdown state.

[0085] In the embodiment of the present application, the main purpose is to determine whether the base station has been moved when the self-mobile device switches from a non-working state to a working state. For example, the self-mobile device automatically enters a pause state due to low battery, sensor abnormality, rain, etc. After a period of time, when the self-mobile device is fully charged, the sensor is normal, or the rain stops, it switches to a working state. At this time, the self-mobile device needs to determine whether the base station has been moved.

[0086] For another example, after the self-mobile device completes the task and returns to the maintenance station, it enters the dormant state. After a period of time, the self-mobile device recognizes the work instruction and enters the working state again. The work instruction can be issued by the user pressing a physical button, issued by the user through an application (APP), or issued by voice, etc., and the embodiments of the present application are not limited thereto.

[0087] For example, the user controls the mobile device to enter the pause state through physical buttons or APP. After a period of time, the mobile device re-enters the working state. At this time, the mobile device needs to determine whether the base station has been moved.

[0088] For another example, the user turns off the self-mobile device through a physical button or an APP. After a period of time, the self-mobile device is restarted. At this time, the self-mobile device needs to determine whether the base station has been moved.

[0089] The reason why it is determined whether the base station has been moved after the mobile device switches from a non-working state to a working state is that the time when the mobile device is in the non-working state may be relatively long, that is, the time when the mobile device is in a suspended state, a dormant state or a powered-off state is relatively long, during which the probability of the base station being moved is higher.

[0090] In addition, when the self-mobile device is in working state, it is positioned or mapped by combining RTK positioning technology and VSLAM positioning technology. If the base station moves, the RTK data will be abnormal, which will lead to abnormal positioning. If the base station is moved during the creation of the environmental map, an abnormal environmental map will be created. Once the environmental map is abnormal, the positioning of the self-mobile device will inevitably be abnormal.

[0091] With this solution, when the self-mobile device switches from a paused state, a dormant state or a shutdown state to a working state, it is determined whether the base station has been moved, thereby avoiding positioning abnormalities after re-entering the working state due to the base station being moved during the non-working state, thereby improving the positioning accuracy of the self-mobile device.

[0092] Figure 3 This is a flow chart of repositioning in a method for detecting a change in the position of a reference station provided in an embodiment of the present application. This embodiment includes:

[0093] 301. Determine a predicted position and a predicted angle from a mobile device.

[0094] When the self-mobile device performs repositioning on the self-mobile device according to the visual image and RTK positioning, each time the visual image and RTK positioning are acquired, a predicted position and a predicted angle are determined according to the visual image and RTK positioning. The predicted angle and predicted position are used to perform repositioning.

[0095] When determining the predicted position, after the mobile device obtains the current RTK positioning, the current RTK positioning is used as the predicted position.

[0096] When determining the predicted angle, the self-mobile device determines the predicted angle of the self-mobile device based on the current RTK positioning and the previous positioning, that is, the self-mobile device calculates the orientation angle of the self-mobile device based on two adjacent RTK positionings, and uses the orientation angle as the predicted angle. The previous positioning is the RTK positioning that was obtained before the self-mobile device obtained the current RTK positioning and is adjacent to the current RTK positioning. For example, the self-mobile device determines 10 RTK positionings per second in sequence, recorded as RTK positioning 1, RTK positioning 2...RTK positioning 10. When the current RTK positioning is RTK positioning 10, the self-mobile device determines the predicted angle based on RTK positioning 9 and RTK positioning 10.

[0097] For another example, when the self-mobile device switches from a non-working state to a working state, it is determined that the surrounding environment changes before and after the self-mobile device switches. After returning to the working state, the first RTK positioning obtained is RTK positioning 1. At this time, the self-mobile device determines the predicted angle based on RTK positioning 1 and the target RTK positioning. The target RTK positioning is the RTK positioning that was most recently obtained and saved when the self-mobile device entered the non-working state.

[0098] Take the current RTK positioning as P N (x N ,y N ), the previous position is P N-1 (x N-1 ,y N-1 ) as an example, the predicted angle θ can be calculated by the following formula (1):

[0099] θ=atan2(y N -y N-1 , x N -x N-1 )Formula (1)

[0100] It should be noted that since the self-mobile device switches from a non-working state to a working state, if the surrounding environment changes before and after the self-mobile device switches its state, it moves forward and is in a moving state. During the moving process, the self-mobile device continuously obtains visual images and RTK positioning. Each time after obtaining the visual image and RTK positioning, repositioning is performed. If the repositioning is successful, there is no need to move forward. If the repositioning is unsuccessful, it keeps moving. Therefore, the predicted position and predicted angle are constantly changing.

[0101] 302. Reposition the self-moving device according to the predicted position, the predicted angle, and the current visual image.

[0102] The self-moving device combines the predicted position, predicted angle, and current visual image to perform relocation on the self-moving device. The predicted position, predicted angle, and other characteristics characterize the motion characteristics of the self-moving device. It can be seen that the self-moving device combines the current visual image and motion characteristics to perform relocation, and then determines whether the base station has been moved based on the relocation result, without installing additional sensors, thereby improving the accuracy of the self-moving device's positioning.

[0103] Optionally, in the above embodiment, when the self-mobile device performs relocation of the self-mobile device according to the predicted position, the predicted angle and the current visual image, the feature points in the environment map are projected into the current visual image according to the predicted position and the predicted angle to obtain a first matching point number. When the first point number is greater than a first threshold, it is determined that the relocation of the self-mobile device is successful.

[0104] Exemplarily, after the mobile device determines the predicted angle and predicted position, the feature points in the environment map are projected into the current visual image according to the predicted angle and predicted position, and matched with the feature points of the current visual image, thereby obtaining a first number of matching points. Feature points are, for example, three-dimensional (3D) feature points, etc., which are not limited by the embodiments of the present application. Feature point matching schemes include, but are not limited to, scale-invariant feature transform (SIFT) matching method, oriented brief (ORB) feature point matching method, etc.

[0105] The environmental map is a map stored locally on the mobile device. The mobile device pre-creates the environmental map using the VSLAM positioning technology integrated with RTK. The environmental map contains three-dimensional visual feature points, and the environmental map and RTK are in the same coordinate system. Among them, the VSLAM positioning technology integrated with RTK includes but is not limited to Laser Spot Detector-Simultaneous Localization and Mapping (LSD-SLAM), ORB-SLAM, Semi-Direct Monocular Visual Odometry (SVO), Dense Tracking and Mapping i (DTAM), etc.

[0106] In the embodiment of the present application, the environment map includes feature points with distinguishability, and the feature points are, for example, 2D feature points, 3D feature points, etc. Figure 4 It is a schematic diagram of an environment map in a method for detecting a change in a base station position provided in an embodiment of the present application.

[0107] Please refer to Figure 4 ,The thick black solid lines represent the boundaries, including the outer boundary of the working area and the boundary of the internal restricted area.,The black points are three-dimensional feature points.

[0108] During the feature point matching process, for any feature point in the environment map, if the mobile device can find the corresponding feature point in the current visual image, it means that the feature point is matched successfully. The mobile device accumulates the number of successfully matched feature points to obtain a first point number.

[0109] Afterwards, the self-mobile device determines whether the first point number is greater than the first threshold. If the first point number is greater than the first threshold, it means that the base station has not been moved and the self-mobile device is successfully relocated. Afterwards, the self-mobile device plans a path based on the current RTK positioning position in the environment map.

[0110] If the first point number is less than or equal to the first threshold, it is possible that the base station has been moved, or the mobile device has been moved, etc.

[0111] With this solution, the mobile device determines the predicted position and predicted angle based on the current RTK positioning and the current visual image, determines the first point based on the predicted position and predicted angle, and then determines whether the base station has been moved based on the first point, with high accuracy and fast speed.

[0112] Optionally, in the above embodiment, when the self-mobile device fails to relocate, that is, when the first point number is less than or equal to the first threshold, the self-mobile device is positioned purely visually based on the BOW dictionary. During the pure visual positioning process, the self-mobile device determines the target key frame from multiple key frames of the environment map. Afterwards, the feature points in the target key frame are projected into the current visual image to obtain a matching second point number. When the second point number is greater than the second threshold, the self-mobile device determines that the reference station is moved, and the second threshold is greater than the first threshold.

[0113] In the pure visual positioning process based on the BOW dictionary, the self-mobile device determines the sparse vector of the current visual image and the sparse vectors of each environmental frame of the environmental image. For the sake of clarity, the sparse vector of the current visual image is referred to as the first vector, and the sparse vector of the key frame is referred to as the second vector. After that, the self-mobile device determines the similarity between the first vector and each second vector, thereby obtaining the similarity between the current visual image and each key frame.

[0114] Next, the mobile device sorts the key frames in descending order of similarity, and uses one or more key frames with the highest sorting as target key frames.

[0115] After that, the self-mobile device performs feature point matching. That is, the feature points in the target key frame are projected into the current visual image to obtain the second matching point number. After obtaining the second point number, the self-mobile device determines whether the second point number is greater than the second threshold. If the second point number is greater than the second threshold, it means that the base station has been moved. If the second point number is less than or equal to the second threshold, it means that the pure visual positioning based on the BOW dictionary has failed. Among them, the second point number is greater than the first point number, which can prevent the successful repositioning by mistake.

[0116] With this solution, after the mobile device fails to relocate based on the current RTK positioning and the current visual image, it further obtains the second point based on the BOW dictionary pure visual relocation, and then determines whether the base station has been moved based on the second point, with high accuracy and fast speed.

[0117] Optionally, in the above embodiment, when the self-mobile device fails to relocate based on pure vision of the BOW dictionary, that is, when the second score is less than or equal to the first score, the self-mobile device determines that it cannot determine whether the reference station is moved. Then, it continues to move forward.

[0118] For example, if the mobile device performs two repositioning operations and both repositioning operations fail, the mobile device continues to move forward. During the movement, the mobile device continues to acquire visual images and RTK positioning, and performs Figure 2 The scheme shown in the flowchart. Among them, there are two types of relocalization: relocalization based on the current visual image and current RTK positioning, and pure visual relocalization based on the BOW dictionary.

[0119] With this solution, if both repositioning operations fail, the mobile device continues to move forward to obtain new visual images and RTK positioning, and re-judges whether the base station has been moved, thereby achieving the purpose of accurately determining whether the base station has been moved.

[0120] Figure 5 This is a flowchart of projecting feature points in an environment map to a current visual image during the first relocation in a method for detecting a change in the position of a reference station provided in an embodiment of the present application. This embodiment includes:

[0121] 501. Determine a posture matrix corresponding to a current posture of the self-moving device according to the predicted position and the predicted angle.

[0122] For example, the self-mobile device calculates the current pose matrix T based on the predicted position and predicted angle vw .

[0123] It should be noted that, although here, the self-mobile device calculates the current moment pose matrix T according to the predicted position and predicted angle vw However, the present application embodiment is not limited to this. In other feasible implementations, the self-mobile device can also obtain the current moment posture matrix T by repositioning or other methods. vw .

[0124] 502. Determine the position coordinates of the feature points in the environment map in the camera coordinate system according to the posture matrix and the transformation matrix, wherein the transformation matrix is ​​the transformation matrix from the center of the mobile device to the center of the camera.

[0125] For example, in the process of projecting the 3D feature points in the environment map to the current visual image, due to the environment map Figure 1 Generally, it is a map in the world coordinate system. Therefore, we must first determine a conversion formula that is used to convert the 3D feature points in the environment map from the world coordinate system to the camera coordinate system.

[0126] For example, the self-mobile device is based on the current posture matrix T vw Determine the two parameters R vw ,t vw , these two parameters are used to determine the conversion formula of 3D feature points in the environment map from the world coordinate system to the camera coordinate system.

[0127] Assuming that the coordinates of a 3D feature point in the environment map in the world coordinate system are P, and the position of the feature point in the camera coordinate system is X (x, y, z), the conversion formula is as follows:

[0128] X=T cv (R vw P+t vw )Formula (2)

[0129] Among them, T cv is the transformation matrix from the mobile center to the camera center, which is a fixed parameter determined by the installation position structure of the camera of the mobile device. vw ,t vw According to the current position T vw The two parameters obtained by inversion are shown in formula (3):

[0130]

[0131] After obtaining the conversion formula, the self-mobile device can determine the position coordinates of each feature point in the environment map in the camera coordinate system, that is, the above-mentioned X. After that, the self-mobile device projects the position coordinates in the camera coordinate system into the current visual image. For example, the self-mobile device is implemented through the projection function π(X). Among them, X is the input information, that is, the position X(x, y, z) of the feature point in the camera coordinate system. When the omnidirectional model is used, the calculation process is shown in formula (4):

[0132]

[0133] Among them, f x 、f y , c x 、c y and ξ are the intrinsic parameters of the camera, which are calibrated before the camera leaves the factory. ‖X‖ represents the calculated Euclidean distance.

[0134] u x 、u yIt represents the pixel coordinates of the feature point position X(x, y, z) in the camera coordinate system projected onto the image plane, and is used to represent the position of the feature point on the image plane when the feature point is matched. During the feature point matching process, the mobile device calculates the position of a 3D feature point in the environment image on the current visual image. If the feature point can be searched near this position, the descriptor matching of the feature point is performed.

[0135] 503. Project the feature points in the environment map into the current visual image according to the position coordinates of each feature point in the environment map in the camera coordinate system to obtain a first number of matching points.

[0136] During the feature point matching process, for any feature point in the environment map, if the match is successful, the number of successfully matched feature points is accumulated from the mobile device to obtain a first point number.

[0137] With this solution, the mobile device calculates the pose matrix at the current moment according to the predicted position and predicted angle, and then projects the feature points in the environment map to the current visual image according to the position matrix and transformation matrix, with fast projection speed and high accuracy.

[0138] Optionally, in the above embodiment, when the self-moving device switches from a non-working state to a working state, when controlling the self-moving device to move, first, the self-moving device is controlled to acquire an environmental image, and based on the acquired environmental image and the environmental image before the state switching, it is determined whether the surrounding environment before and after the state switching of the self-moving device has changed. Afterwards, if the environmental image indicates that the surrounding environment of the self-moving device has changed, the self-moving device is controlled to move.

[0139] Exemplarily, after the mobile device switches from the non-working state to the working state, relocation is triggered only when a preset condition is met, and the preset condition is, for example, that the surrounding environment of the mobile device changes. That is, after the mobile device switches from the non-working state to the working state, the environment images before and after the state switching are compared to determine whether the surrounding environment has changed. Relocation is triggered only when the surrounding environment changes.

[0140] With this solution, only when the surrounding environment changes before and after the mobile device status switches, the current visual image and the current RTK positioning are combined to determine whether the base station has been moved, thereby improving the accuracy of base station position change detection.

[0141] Optionally, in the above embodiment, after the mobile device switches from the non-working state to the working state, if the environmental image indicates that the surrounding environment of the mobile device has not changed, the mobile device further compares the target positioning and the initial positioning. When the distance between the target positioning and the initial positioning is greater than a preset distance, the mobile device determines that the base station has been moved. The target positioning is the RTK positioning obtained before the mobile device enters the non-working state, and the initial positioning is the RTK positioning after the mobile device switches to the working state.

[0142] Exemplarily, after the mobile device switches from a non-working state to a working state, if the environmental image indicates that the surrounding environment of the mobile device has not changed, one possible reason is that the mobile device did not move during the non-working state, that is, it did not move or rotate; another possible reason is that the mobile device moved during the non-working state, such as displacement or rotation, but the surrounding environment after the movement was similar to the surrounding environment before the movement. Take the self-moving device as a lawn mowing robot as an example. During the non-working state of the self-moving device, there is a large white wall in front and a lawn around it. There are no obvious features on the lawn, such as large trees, flower beds, etc. During the non-working state, the self-moving device is moved to another place, which is also facing the large white wall and surrounded by lawns. After the mobile device enters the working state, it is found that the surrounding environment has not changed.

[0143] In order to prevent misjudgment, after the self-mobile device determines that the surrounding environment has not changed before and after the state switching, it further determines whether the RTK positioning of the self-mobile device has changed before and after the state switching, that is, whether the distance between the target positioning before the state switching and the initial positioning after the state switching is greater than the preset distance. When the distance between the target positioning before the state switching and the initial positioning after the state switching is greater than the preset distance, the self-mobile device determines that the base station has been moved; when the distance between the target positioning before the state switching and the initial positioning after the state switching is less than or equal to the preset distance, the self-mobile device determines that the base station has not been moved.

[0144] With this solution, after the mobile device switches from a non-working state to a working state, if the surrounding environment has not changed before and after the mobile device state is switched, it is further determined whether the RTK positioning has changed before and after the mobile device state is switched, thereby accurately determining whether the base station has been moved to avoid misjudgment.

[0145] Optionally, in the above embodiment, when the self-mobile device switches from a non-working state to a working state, the target image and the initial visual image are compared, and when the similarity between the target image and the initial visual image is less than a preset similarity, it is determined that the surrounding environment of the self-mobile device has changed. The target image and the initial visual image are adjacent, the target image is an image taken before the self-mobile device enters the non-working state, and the initial visual image is an image taken after the self-mobile device switches to the working state.

[0146] For example, when the mobile device switches from a non-working state to a working state, in order to determine whether the surrounding environment changes before and after the state switching of the mobile device, the target image taken before the state switching and the initial visual image after the state switching are compared. The target image is one or more frames of images taken last time before the mobile device enters the non-working state, and the initial visual image is the first image taken after the mobile device switches from the non-working state to the working state.

[0147] In one method, the self-mobile device determines the similarity between the target image and the initial visual image by a feature point matching method. In this method, the self-mobile device extracts feature points from the target image and extracts feature points from the initial visual image. If the number of matched feature points exceeds a certain threshold, it is considered that the similarity between the target image and the initial visual image is greater than or equal to a preset similarity, indicating that the surrounding environment has not changed before and after the self-mobile device switches its state. If the number of matched feature points does not exceed the threshold, it is considered that the similarity between the target image and the initial visual image is less than the preset similarity, indicating that the surrounding environment has changed before and after the self-mobile device switches its state.

[0148] In another way, the mobile device compares the target image and the initial visual image pixel by pixel. If the number of similar pixels exceeds a preset pixel threshold, it is considered that the similarity between the target image and the initial visual image is greater than or equal to the preset similarity, indicating that the surrounding environment has not changed before and after the mobile device switches its state. If the number of similar pixels does not exceed the pixel threshold, it is considered that the similarity between the target image and the initial visual image is less than the preset similarity, indicating that the surrounding environment has changed before and after the mobile device switches its state.

[0149] By adopting this solution, the mobile device determines whether the surrounding environment has changed before and after the state of the mobile device is switched by image comparison, which has high accuracy and fast speed.

[0150] Optionally, in the above embodiment, when the self-mobile device switches from a non-working state to a working state, the target image and the initial visual image are compared, and when the target image and the initial visual image have a similarity greater than or equal to a preset similarity, it is determined that the surrounding environment before and after the self-mobile device switches state has not changed. Wherein, the target image and the initial visual image are adjacent, the target image is an image taken before the self-mobile device enters a non-working state, and the initial visual image is an image taken after the self-mobile device switches to a working state.

[0151] The specific judgment process can refer to the above-mentioned processing process when the similarity between the target image and the initial visual image is less than the preset similarity, which will not be repeated here.

[0152] Optionally, in the above embodiment, during the normal operation of the self-mobile device, that is, when it is in a working state, the latest visual image and the latest RTK positioning are acquired in real time. When the self-mobile device is switched from a working state to a non-working state, the latest visual image and the latest RTK positioning are stored, so that after the self-mobile device is switched from a non-working state to a working state, the latest visual image is used as a target image, and the latest RTK positioning is used as a target positioning.

[0153] Exemplarily, the self-mobile device continuously acquires visual images and RTK positioning during operation. Each time the latest visual image and the latest RTK positioning are acquired, the latest visual image is used to overwrite the visual image acquired last time, and the latest RTK positioning is used to overwrite the RTK positioning acquired last time. Afterwards, once the self-mobile device switches from a working state to a non-working state, the latest visual image and the latest RTK positioning are stored in a non-volatile memory such as a flash memory, an embedded multimedia card (EMMC), etc.

[0154] Afterwards, when the self-mobile device switches from the non-working state to the working state, the self-mobile device uses the latest visual image stored in the non-volatile memory as the target image, uses the latest RTK positioning stored in the non-volatile memory as the target positioning, and judges whether the surrounding environment has changed before and after the self-mobile device switches based on the target image and the initial visual image taken for the first time after entering the working state. If the surrounding environment has not changed before and after the self-mobile device switches, it is determined whether the base station has been moved based on the target positioning and the initial positioning obtained for the first time after entering the working state. If the surrounding environment has changed before and after the self-mobile device switches, the self-mobile device moves forward and determines whether the base station has been moved based on the current RTK positioning, the current visual image, etc.

[0155] With this solution, the self-moving device in the working state continuously obtains the latest visual image and the latest RTK positioning. Once the self-moving device switches from the working state to the non-working state, the latest visual image and the latest RTK positioning are used as the target image and the target positioning respectively, so as to judge whether the base station moves when the self-moving device switches from the non-working state to the working state.

[0156] Optionally, in the above embodiment, the settlement status of RTK positioning is a fixed solution.

[0157] Exemplarily, there are three solution states for RTK positioning: fixed solution, floating point solution and single point solution. Among them, the positioning accuracy is the highest when the solution is fixed, generally less than 2 cm. The floating point solution has a larger error, and the error is the largest when the solution is single point. Fixed solution and floating point solution are distinguished by "ambiguity". When the ambiguity is an integer, it is a fixed solution, and when the ambiguity is not an integer, it is a floating point solution. The embodiments of the present application may adopt any ambiguity calculation method. A single point solution means that the differential signal of any base station is not used, so the accuracy is similar to that of an ordinary global positioning system (GPS).

[0158] Therefore, in order to accurately determine whether the reference station is moved, the solution states of the RTK positioning involved in the above embodiments, such as target positioning, initial positioning, current RTK positioning, latest RTK positioning, etc., are all fixed solutions.

[0159] By adopting this solution, the solution state of the RTK positioning involved in the embodiment of the present application is a fixed solution with high accuracy, thereby achieving the purpose of accurately determining whether the base station has been moved.

[0160] Figure 6 This is another flow chart of the method for detecting the position change of the reference station provided in an embodiment of the present application.

[0161] This embodiment includes:

[0162] 601. Before the mobile device enters the non-working state, at least one latest RTK positioning and latest visual image are saved. Afterwards, when the mobile device resumes the working state, the latest RTK positioning becomes the target positioning, and the latest visual image becomes the target image.

[0163] For example, when the self-mobile device is working normally, the latest visual image and the latest RTK positioning are continuously obtained, and once the self-mobile device is switched from the working state to the non-working state, the latest visual image and the latest RTK positioning are used as the target image and the target positioning, respectively. The non-working state includes the pause state, the sleep state or the shutdown state.

[0164] 602. After the mobile device resumes its working state, determine whether the surrounding environment has changed since the mobile device state was switched. If the surrounding environment has not changed since the mobile device state was switched, execute step 603; if the surrounding environment has changed since the mobile device state was switched, execute step 606.

[0165] For example, after the mobile device resumes working state, it captures the surrounding environment to obtain the current visual image. Afterwards, the mobile device determines the similarity between the current visual image and the target image, and determines whether the surrounding environment has changed before and after the mobile device switches state based on the similarity.

[0166] 603. The self-mobile device determines whether the distance between the target positioning and the initial positioning is greater than a preset distance. If the distance between the target positioning and the initial positioning is greater than the preset distance, execute step 604; if the distance between the target positioning and the initial positioning is less than or equal to the preset distance, execute step 605.

[0167] For example, after the mobile device resumes working state, it is positioned based on RTK technology to obtain an initial position, and then it is determined whether the reference station has been moved according to the distance between the initial position and the target position.

[0168] 604. The self-mobile device determines that the base station has been moved and issues an alarm.

[0169] 605. The self-mobile device determines that the base station has not been moved.

[0170] 606. Triggering relocation of the mobile device.

[0171] For example, when the self-mobile device switches from a non-working state to a working state, if the environment image indicates that the surrounding environment of the self-mobile device has changed before and after the state switching, it is possible that the self-mobile device has been moved, or it is possible that the self-mobile device has not been moved to another location, but the surrounding environment has changed. In this case, the self-mobile device is triggered to perform relocation.

[0172] 607. The self-mobile device moves forward, and obtains the current visual image and the current RTK positioning during the movement.

[0173] 608. The mobile device determines a predicted position and a predicted angle according to the current visual image and the current RTK positioning.

[0174] 609. The mobile device projects the feature points in the environment map into the current visual image according to the predicted position and the predicted angle to obtain the first matching points.

[0175] 610. The self-mobile device determines whether the first score is greater than a first threshold. If the first score is greater than the first threshold, the relocation is successful and step 605 is executed; if the first score is less than or equal to the first threshold, step 611 is executed.

[0176] 611. The self-mobile device is relocated purely visually based on the BOW dictionary to obtain a second point.

[0177] Exemplarily, a target key frame is determined from a plurality of key frames of the environment map, and feature points in the target key frame are projected into the current visual image to obtain a second number of matching points.

[0178] 612. The mobile device determines whether the second score is greater than a second threshold. If the second score is greater than the second threshold, the relocation is successful and step 604 is executed; if the second score is less than or equal to the second threshold, the relocation is considered to have failed and step 607 is executed.

[0179] For example, if the number of matching points is greater than the second threshold, it means that the mobile device has found a better relocation result than the predicted position, that is, the actual position of the mobile device is inconsistent with the predicted position, and it is determined that the reference station has been moved, for example, by at least 10 centimeters.

[0180] If the number of matched points is less than or equal to the second threshold, it means that the repositioning has failed. It may be that the surrounding scene has changed significantly, or the self-mobile device is located outside the environmental map, resulting in the failure of pure visual repositioning based on the BOW dictionary. At this time, it is impossible to determine whether the base station has been moved. At this time, the self-mobile device continues to move forward. For example, the self-mobile device actively moves forward; for example, the self-mobile device prompts the user so that the user controls the self-mobile device to continue moving forward. During the movement, continue to collect visual images and RTK positioning, and execute step 607 and subsequent steps.

[0181] It should be noted that the RTK positioning involved in the above embodiments is all RTK fixed solution positioning. This is because the RTK positioning error is relatively large when it is not a fixed solution, and it cannot be used to accurately determine whether the base station has been moved.

[0182] The following describes in detail the above-mentioned method for detecting the position change of the reference station by taking the self-moving device as a lawn mowing robot in combination with a specific application scenario.

[0183] Application scenario 1:

[0184] During the normal operation of the self-mobile device, the latest visual image and the latest RTK positioning are continuously obtained. When the self-mobile device switches from the working state to the non-working state, the latest visual image and the latest RTK positioning are saved in the non-volatile memory. The latest visual image shows that the self-mobile device faces a large white wall, surrounded by lawn A, and there are no obvious features on lawn A, such as large trees, flower beds, etc.

[0185] Case a: No movement occurred while the mobile device was inactive, but the base station was moved.

[0186] After a period of time, the self-mobile device re-enters the working state, and before working, it determines whether the base station has been moved. During the judgment process, the self-mobile device captures the current environment to obtain the initial visual image, performs positioning to obtain the initial positioning, and takes out the latest visual image and the latest RTK positioning from the non-volatile memory as the target image and target positioning, respectively.

[0187] Afterwards, the self-mobile device determines the similarity between the initial visual image and the target image. The initial visual image corresponds to lawn A, and the target image corresponds to lawn A. In this case, the self-mobile device determines that the initial visual image and the target image are very similar, and temporarily assumes that the surrounding environment has not changed before and after the state of the self-mobile device is switched. Afterwards, the self-mobile device determines the distance between the initial positioning and the target positioning. Since the distance between the initial positioning and the target positioning is relatively large, the self-mobile device determines that the base station has been moved.

[0188] Case b: Movement occurs while the mobile device is inactive, but the base station is moved.

[0189] After the mobile device entered the non-working state on lawn A, it was manually moved to lawn B. The environments of lawn A and lawn B are very different. After the mobile device switched to the working state on lawn B, it found that the surrounding environment had changed. After that, the mobile device moved forward and performed relocation.

[0190] During the process of traveling, each time the current visual image and the current RTK positioning are obtained, the predicted angle and predicted position are determined. Then, based on the predicted angle and predicted position, the 3D feature points of the environment image are projected into the current visual image to obtain the first matching point number.

[0191] Since the first number of points is relatively low, the self-mobile device performs pure visual relocation based on the BOW dictionary to obtain a second number of matching points, that is, the number of feature points matching the target key frame in the environment image and the current visual image. When the second number of points is greater than the second threshold, the self-mobile device considers that the pure visual relocation is successful, determines that the self-reference station has been moved, and issues an alarm. When the second number of points is less than or equal to the second threshold, the self-mobile device considers that the pure visual relocation has failed, continues to move forward, and executes the above relocation logic.

[0192] Case c: Movement occurs while the mobile device is inactive, but the base station is moved.

[0193] After the mobile device entered the non-working state on lawn A, it was manually moved to lawn B. The environments of lawn A and lawn B are very different. After the mobile device switched to the working state on lawn B, it found that the surrounding environment had changed. After that, the mobile device moved forward and performed relocation.

[0194] During the process of traveling, each time the current visual image and the current RTK positioning are obtained, the predicted angle and predicted position are determined. Then, based on the predicted angle and predicted position, the 3D feature points of the environment image are projected into the current visual image to obtain the first number of matching points. Since the first number of points is greater than the first threshold, the mobile device considers that the repositioning is successful and the reference station has not been moved.

[0195] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0196] Figure 7 Schematic diagram of a reference station position change detection device provided in an embodiment of the present application. The reference station position change detection device 700 includes:

[0197] A traveling module 71, used to control the self-moving device to travel when the self-moving device is switched from a non-working state to a working state;

[0198] An acquisition module 72 is used to control the self-mobile device to acquire visual images and real-time dynamic carrier phase differential RTK positioning during the movement of the self-mobile device;

[0199] A processing module 73, configured to reposition the self-moving device according to the visual image and the RTK positioning;

[0200] The determination module 74 is used to determine that the reference station has not been moved when the self-moving device is successfully relocated, and the reference station is used to provide a differential signal to the self-moving device so that the self-moving device obtains RTK positioning according to the differential signal.

[0201] In a feasible implementation, the processing module 73 is used to determine the predicted position of the self-mobile device based on the current RTK positioning obtained by the self-mobile device, the RTK positioning includes the current RTK positioning, and the visual image includes the current visual image; determine the predicted angle of the self-mobile device based on the current RTK positioning and the previous positioning, the previous positioning is the RTK positioning obtained before the self-mobile device obtains the current RTK positioning and is adjacent to the current positioning; and reposition the self-mobile device based on the predicted position, the predicted angle and the current visual image.

[0202] In a feasible implementation, when the processing module 73 performs repositioning on the self-moving device according to the predicted position, the predicted angle and the current visual image, it is used to project the feature points in the environment map into the current visual image according to the predicted position and the predicted angle to obtain a first matching point number; when the first point number is greater than a first threshold, it is determined that the repositioning of the self-moving device is successful.

[0203] In a feasible implementation, the processing module 73 is also used to determine a target key frame from multiple key frames of the environment map when the self-mobile device fails to be relocated, and the target key frame is the key frame with the highest similarity to the current visual image among the multiple key frames; project the feature points in the target key frame into the current visual image to obtain a second matching point number; when the second point number is greater than a second threshold, determine that the base station has been moved, and the second threshold is greater than the first threshold.

[0204] In a feasible implementation, the processing module 73 is further configured to, when the second number of points is less than or equal to a second threshold, determine that it is impossible to determine whether the reference station has been moved; and continue to control the self-moving device to move forward.

[0205] In a feasible implementation, the processing module 73 projects the feature points in the environment map into the current visual image according to the predicted position and the predicted angle to obtain the first number of matching points. The processing module 73 is used to determine the posture matrix corresponding to the current posture of the self-moving device according to the predicted position and the predicted angle; determine the position coordinates of the feature points in the environment map in the camera coordinate system according to the posture matrix and the transformation matrix, and the transformation matrix is ​​the transformation matrix from the center of the self-moving device to the center of the camera; and project the feature points in the environment map into the current visual image according to the position coordinates of each feature point in the environment map in the camera coordinate system to obtain the first number of matching points.

[0206] In a feasible implementation, the processing module 73 is used to control the self-mobile device to acquire an environment image when the self-mobile device is switched from a non-working state to a working state;

[0207] The moving module 71 is used to control the self-moving device to move if the environment image indicates that the surrounding environment of the self-moving device has changed.

[0208] In a feasible implementation, the processing module 73 is also used to compare the target positioning and the initial positioning when the self-mobile device is switched from a non-working state to a working state, if the environmental image indicates that the surrounding environment of the self-mobile device has not changed, the target positioning is the RTK positioning obtained before the self-mobile device enters the non-working state, and the initial positioning is the RTK positioning after the self-mobile device is switched to the working state; when the distance between the target positioning and the initial positioning is greater than the preset distance, it is determined that the base station is moved.

[0209] In a feasible implementation, the processing module 73 is also used to compare the target image and the initial visual image, the target image and the initial visual image are adjacent, the target image is an image taken before the self-mobile device enters the non-working state, and the initial visual image is an image taken after the self-mobile device switches to the working state; when the similarity between the target image and the initial visual image is less than a preset similarity, it is determined that the surrounding environment of the self-mobile device has changed.

[0210] In a feasible implementation, the processing module 73 is also used to compare the target image and the initial visual image, the target image and the initial visual image are adjacent, the target image is an image taken before the self-mobile device enters the non-working state, and the initial visual image is an image taken after the self-mobile device switches to the working state; when the similarity between the target image and the initial visual image is greater than or equal to a preset similarity, it is determined that the surrounding environment of the self-mobile device has not changed.

[0211] In a feasible implementation, the processing module 73, before comparing the target image with the initial visual image, is also used to obtain the latest visual image and the latest RTK positioning in real time when the self-mobile device is in a working state; when the self-mobile device is switched from a working state to a non-working state, the latest visual image and the latest RTK positioning are stored, so that after the self-mobile device is switched from a non-working state to a working state, the latest visual image is used as the target image, and the latest RTK positioning is used as the target positioning.

[0212] In a feasible implementation manner, the solution state of the RTK positioning is a fixed solution.

[0213] In a feasible implementation, the non-working state includes any one of the following states: a pause state, a sleep state, or a shutdown state.

[0214] The control device provided in the embodiment of the present application can execute the actions of the cleaning equipment in the above-mentioned embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0215] Figure 8 This is a schematic diagram of the structure of a self-moving device provided in an embodiment of the present application. Figure 8 As shown, the mobile device 800 includes:

[0216] Processor 81 and memory 82;

[0217] The memory 82 stores computer instructions;

[0218] The processor 81 executes the computer instructions stored in the memory 82 , so that the processor 81 executes the control method of the self-moving device as described above.

[0219] The specific implementation process of the processor 81 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0220] Optionally, the self-mobile device 800 further includes a communication component 83 , wherein the processor 81 , the memory 82 and the communication component 83 may be connected via a bus 84 .

[0221] An embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored. The computer instructions are executed by a processor to implement the method for detecting a change in a reference station position as described above.

[0222] An embodiment of the present application further provides a computer program product, which includes a computer program, and the computer program is executed by a processor to perform the method for detecting a change in the position of a reference station as described above.

[0223] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0224] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for detecting a change in the position of a reference station, characterized in that: include: When the self-mobile device is switched from a non-working state to a working state, controlling the self-mobile device to obtain an environmental image, and judging whether the surrounding environment of the self-mobile device has changed before and after the state switching according to the environmental image; When the surrounding environment of the self-moving device changes before and after the state switching, controlling the self-moving device to move forward; During the movement of the self-moving device, controlling the self-moving device to obtain visual images and real-time dynamic carrier phase differential RTK positioning; Determining a predicted position and a predicted angle of the self-moving device according to the RTK positioning, and repositioning the self-moving device according to the predicted position, the predicted angle, and the visual image; When the self-moving device is successfully repositioned, it is determined that the reference station has not been moved, and the reference station is used to provide a differential signal to the self-moving device, so that the self-moving device obtains RTK positioning according to the differential signal.

2. The method according to claim 1, characterized in that Determining the predicted position and predicted angle of the self-moving device according to the RTK positioning, and relocating the self-moving device according to the predicted position, the predicted angle and the visual image, includes: Determine a predicted position of the self-mobile device according to a current RTK positioning acquired by the self-mobile device, wherein the RTK positioning includes a current RTK positioning, and the visual image includes a current visual image; Determine the predicted angle of the self-mobile device according to the current RTK positioning and a previous positioning, wherein the previous positioning is an RTK positioning obtained before the self-mobile device obtains the current RTK positioning and is adjacent to the current positioning; Repositioning the self-moving device is performed based on the predicted position, the predicted angle, and the current visual image.

3. The method according to claim 2, characterized in that The repositioning of the self-moving device according to the predicted position, the predicted angle and the current visual image comprises: Projecting feature points in the environment map into the current visual image according to the predicted position and the predicted angle to obtain a first number of matching points; When the first points are greater than a first threshold, it is determined that the relocation of the self-mobile device is successful.

4. The method according to claim 2, characterized in that: Also includes: When the repositioning of the self-moving device fails, it is determined whether the reference station is moved according to the current visual image and the environment map.

5. The method according to claim 4, characterized in that Also includes: When it cannot be determined whether the reference station is moved according to the current visual image and the environment map, the self-moving device continues to be controlled to move forward and perform repositioning.

6. The method according to claim 1, characterized in that Also includes: When the surrounding environment of the self-mobile device does not change before and after the state switching, it is determined whether the base station moves according to the RTK positioning.

7. The method according to claim 6, characterized in that When the surrounding environment of the mobile device does not change before and after the state switching, judging whether the reference station moves according to the RTK positioning includes: When the surrounding environment of the self-mobile device does not change before and after the state switching, compare the target positioning and the initial positioning, the target positioning is the RTK positioning obtained before the self-mobile device enters the non-working state, and the initial positioning is the RTK positioning after the self-mobile device switches to the working state; When the distance between the target location and the initial location is greater than a preset distance, it is determined that the reference station is moved.

8. The method according to any one of claims 1 to 7, characterized in that: The solution state of the RTK positioning is a fixed solution.

9. The method according to any one of claims 1 to 7, characterized in that: The non-working state includes any one of the following states: a pause state, a sleep state or a shutdown state.

10. A device for detecting a change in the position of a reference station, characterized in that: Integrated in a self-mobile device, the device comprises: A processing module, used for controlling the self-mobile device to obtain an environmental image when the self-mobile device is switched from a non-working state to a working state, and judging whether the surrounding environment of the self-mobile device has changed before and after the state switching according to the environmental image; A moving module, used for controlling the self-moving device to move when the surrounding environment of the self-moving device changes before and after the state switching; An acquisition module, used for controlling the self-mobile device to acquire visual images and real-time dynamic carrier phase differential RTK positioning during the movement of the self-mobile device; The processing module is further used to determine the predicted position and predicted angle of the self-moving device according to the RTK positioning, and to reposition the self-moving device according to the predicted position, the predicted angle and the visual image; The determination module is used to determine that the reference station has not been moved when the self-moving device is successfully relocated, and the reference station is used to provide a differential signal to the self-moving device so that the self-moving device obtains RTK positioning according to the differential signal.

11. A self-propelled device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the mobile device implements the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images