Control method of self-moving device, self-moving device and storage medium

Abstract

The embodiment of the invention provides a control method of a self-moving device, the self-moving device and a storage medium, and relates to the technical field of intelligent control. The method comprises the following steps: respectively acquiring spatial information richness of a first view field range and a second view field range; taking the direction corresponding to the view field range with high spatial information richness in the first view field range and the second view field range as a target direction; movement of the self-moving device is controlled based on the sensor information and the target direction. According to the embodiment of the invention, the movement of the self-moving equipment is controlled in combination with the sensor information and the target direction with high spatial information richness, so that more accurate positioning of the self-moving equipment can be realized, and the environment perception capability of the self-moving equipment can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of intelligent control technology, and in particular to a control method for a self-moving device, a self-moving device, and a storage medium. Background Technology

[0002] With the rapid development of artificial intelligence and robotics, self-moving devices (such as service robots, driverless cars, and drones) are increasingly being used in industrial, commercial, and home scenarios. Autonomous navigation and control, as a core capability of self-moving devices, relies on the collaborative work of technologies such as multi-sensor fusion positioning and map building.

[0003] However, in practical applications, existing technologies still suffer from low positioning accuracy in self-moving devices, which restricts their intelligence level and adaptability to various scenarios. Summary of the Invention

[0004] This application provides a control method for a self-moving device to solve the problem of low positioning accuracy in the prior art.

[0005] According to one aspect of this application, a control method for a self-moving device is provided. The self-moving device includes a first sensor and a second sensor; a first field of view of the first sensor corresponds to the current moving direction of the self-moving device; and a second field of view of the second sensor corresponds to the opposite direction of the current moving direction. The method includes: The spatial information richness of the first field of view and the second field of view are obtained respectively; wherein, the spatial information richness is used to characterize the proportion of effective spatial information in the target spatial information within the corresponding field of view; the effective spatial information includes target spatial information used for localization and / or obstacle detection; The direction corresponding to the field of view with higher spatial information richness in the first and second fields of view is taken as the target direction. The movement of the self-moving device is controlled based on sensor information and target direction, wherein the sensor information includes at least one of first sensing information collected by a first sensor and second sensing information collected by a second sensor.

[0006] In one possible implementation, the first sensor and the second sensor are different sensors; The above-mentioned control of the movement of the self-moving device based on sensor information and target orientation includes: The actual movement direction of the self-moving device is determined based on the relationship between the target direction and the current movement direction. The movement of the self-moving device is controlled based on the actual direction of movement and sensor information.

[0007] In one possible implementation, the determination of the actual movement direction of the self-moving device based on the relationship between the target direction and the current movement direction includes: If the target direction is in the same direction as the current movement direction, then the current movement direction will be taken as the actual movement direction. If the target direction is opposite to the current movement direction, then the opposite direction of the current movement direction will be taken as the actual movement direction.

[0008] In yet another possible implementation, the above-mentioned control of the self-moving device's movement based on the actual direction of movement and sensor information includes: If the actual movement direction is the current movement direction, then control the self-moving device to maintain the current movement direction; if the actual movement direction is the opposite of the current movement direction, then control the self-moving device to change direction. The real-time displacement of the self-moving device is determined based on sensor information; Determine obstacle information in the target area based on sensor information; The movement of the self-moving device is controlled based on real-time displacement, obstacle information, and pre-movement path.

[0009] In yet another possible implementation, the aforementioned sensor information includes first sensing information and second sensing information; Determining the real-time displacement of a self-moving device based on sensor information includes: If the first sensing information is detected as a valid signal, then the first real-time displacement of the self-moving device is determined based on the first sensing information; and the real-time displacement is determined based on the first real-time displacement. If the first sensor information is detected as an invalid signal, the second real-time displacement of the self-moving device is determined based on the second sensor information, and the second real-time displacement is used as the real-time displacement.

[0010] In another possible implementation, the above-mentioned determination of real-time displacement based on the first real-time displacement includes: The second real-time displacement of the self-moving device is determined based on the second sensor information; The real-time displacement is determined based on the difference between the first real-time displacement and the second real-time displacement.

[0011] In another possible implementation, determining the real-time displacement based on the difference between the first and second real-time displacements includes: If the difference between the first real-time displacement and the second real-time displacement is less than a preset difference threshold, then the first real-time displacement is taken as the real-time displacement.

[0012] In another possible implementation, before controlling the movement of the self-moving device based on real-time displacement, obstacle information, and pre-movement path, the following is also included: If the difference between the first real-time displacement and the second real-time displacement is not less than the preset difference threshold, then the self-moving device is controlled to stop moving and a maintenance reminder is sent.

[0013] In yet another possible implementation, determining the first real-time displacement of the self-moving device based on the first sensing information includes: If it is detected that all the first sensing information is a valid signal within a preset time range, then the first real-time displacement of the self-moving device is determined based on the first sensing information. If it is detected that the first sensing information changes from an invalid signal to an effective signal within a preset time range, then the first sensor is controlled to reposition itself, and the real-time sensing information of the first sensor is obtained. Based on the real-time sensing information, the first real-time displacement of the self-moving device is determined.

[0014] In yet another possible implementation, the aforementioned sensor information includes first sensing information or second sensing information; controlling the movement of the self-moving device based on the sensor information and the target orientation includes: If the target direction is in the same direction as the current movement direction, the movement of the self-moving device is controlled based on the first sensor information; If the target direction is opposite to the current movement direction, the movement of the self-moving device is controlled based on the second sensor information.

[0015] In another possible implementation, the above-mentioned control of the movement of the self-moving device based on the first sensor information includes: The information acquisition frequency of the first sensor is increased to a first preset frequency, and the first sensing information is acquired based on the first preset frequency; the movement of the self-moving device is controlled based on the first sensing information. Controlling the movement of a self-moving device based on second sensor information includes: The information acquisition frequency of the second sensor is increased to a second preset frequency, and the second sensing information is acquired based on the second preset frequency; the movement of the self-moving device is controlled based on the second sensing information acquired by the second sensor.

[0016] According to another aspect of the embodiments of this application, a self-moving device is provided, the self-moving device including a controller, a first sensor and a second sensor; a first field of view of the first sensor corresponds to the current movement direction of the self-moving device; a second field of view of the second sensor corresponds to the opposite direction of the current movement direction; the controller is configured to: The spatial information richness of the first field of view and the second field of view are obtained respectively; wherein, the spatial information richness is used to characterize the proportion of effective spatial information in the target spatial information within the corresponding field of view; the effective spatial information includes target spatial information used for localization and / or obstacle detection; The direction corresponding to the field of view with higher spatial information richness in the first and second fields of view is taken as the target direction. The movement of the self-moving device is controlled based on sensor information and target direction, wherein the sensor information includes at least one of first sensing information collected by a first sensor and second sensing information collected by a second sensor.

[0017] In one possible implementation, the first sensor and the second sensor are different sensors; The above-mentioned control of the movement of the self-moving device based on sensor information and target orientation includes: The actual movement direction of the self-moving device is determined based on the relationship between the target direction and the current movement direction. The movement of the self-moving device is controlled based on the actual direction of movement and sensor information.

[0018] In one possible implementation, the determination of the actual movement direction of the self-moving device based on the relationship between the target direction and the current movement direction includes: If the target direction is in the same direction as the current movement direction, then the current movement direction will be taken as the actual movement direction. If the target direction is opposite to the current movement direction, then the opposite direction of the current movement direction will be taken as the actual movement direction.

[0019] In one possible implementation, controlling the movement of the self-moving device based on the actual direction of movement and sensor information includes: If the actual movement direction is the current movement direction, then control the self-moving device to maintain the current movement direction; if the actual movement direction is the opposite of the current movement direction, then control the self-moving device to change direction. The real-time displacement of the self-moving device is determined based on sensor information; Determine obstacle information in the target area based on sensor information; The movement of the self-moving device is controlled based on real-time displacement, obstacle information, and pre-movement path.

[0020] In another possible implementation, the aforementioned sensor information includes first sensing information and second sensing information; The above-mentioned method of determining the real-time displacement of a self-moving device based on sensor information includes: If the first sensing information is detected as a valid signal, then the first real-time displacement of the self-moving device is determined based on the first sensing information; and the real-time displacement is determined based on the first real-time displacement. If the first sensor information is detected as an invalid signal, the second real-time displacement of the self-moving device is determined based on the second sensor information, and the second real-time displacement is used as the real-time displacement.

[0021] In another possible implementation, the above-mentioned determination of real-time displacement based on the first real-time displacement includes: The second real-time displacement of the self-moving device is determined based on the second sensor information; The real-time displacement is determined based on the difference between the first real-time displacement and the second real-time displacement.

[0022] In another possible implementation, determining the real-time displacement based on the difference between the first and second real-time displacements includes: If the difference between the first real-time displacement and the second real-time displacement is less than a preset difference threshold, then the first real-time displacement is taken as the real-time displacement.

[0023] In another possible implementation, the controller described above is also configured as follows: If the difference between the first real-time displacement and the second real-time displacement is not less than the preset difference threshold, then the self-moving device is controlled to stop moving and a maintenance reminder is sent.

[0024] In another possible implementation, determining the first real-time displacement of the self-moving device based on the first sensing information includes: If it is detected that all the first sensing information is a valid signal within a preset time range, then the first real-time displacement of the self-moving device is determined based on the first sensing information. If it is detected that the first sensing information changes from an invalid signal to an effective signal within a preset time range, then the first sensor is controlled to reposition itself, and the real-time sensing information of the first sensor is obtained. Based on the real-time sensing information, the first real-time displacement of the self-moving device is determined.

[0025] In yet another possible implementation, the aforementioned sensor information includes either first sensing information or second sensing information; The above-mentioned control of the movement of the self-moving device based on sensor information and target orientation includes: If the target direction is in the same direction as the current movement direction, the movement of the self-moving device is controlled based on the first sensor information; If the target direction is opposite to the current movement direction, the movement of the self-moving device is controlled based on the second sensor information.

[0026] In yet another possible implementation, the above-mentioned control of the movement of the self-moving device based on the first sensor information includes: The information acquisition frequency of the first sensor is increased to a first preset frequency, and the first sensing information is acquired based on the first preset frequency; the movement of the self-moving device is controlled based on the first sensing information. The above-mentioned control of the movement of the self-moving device based on the second sensor information includes: The information acquisition frequency of the second sensor is increased to a second preset frequency, and the second sensing information is acquired based on the second preset frequency; the movement of the self-moving device is controlled based on the second sensing information acquired by the second sensor.

[0027] In yet another possible implementation, the first sensor is a lidar sensor and the second sensor is a vision sensor.

[0028] According to another aspect of this application, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method shown in the first aspect of this application.

[0029] According to another aspect of this application, a non-transitory computer-readable storage medium is provided that stores computer instructions for causing a computer to perform the steps of the method shown in the first aspect of this application.

[0030] According to another aspect of this application, a computer program product is provided, the computer program product including instructions that, when executed, cause a computer to perform the steps of the method shown in the first aspect of this application.

[0031] The beneficial effects of the technical solution provided in this application are: The self-moving device control method provided in this application can be achieved by setting a first sensor and a second sensor with different fields of view on the self-moving device, and obtaining the spatial information richness of the first field of view and the second field of view. The target direction corresponding to the field of view with higher spatial information richness is determined, and the movement of the self-moving device is controlled based on sensor information and the target direction. Since the higher the spatial information richness of the field of view, the higher the proportion of target spatial information used for positioning and / or obstacle detection within the target spatial information within the field of view, the higher the spatial perception of the sensor information determined based on that field of view, and the higher the accuracy of positioning based on that sensor information. Unlike existing technologies that rely on a single sensor for positioning and navigation, this application combines sensor information and the target direction with higher spatial information richness to control the movement of the self-moving device, enabling more accurate positioning, effectively improving the environmental perception capability of the self-moving device, and further enhancing its environmental adaptability and mobility safety. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 A schematic diagram of a self-moving device structure provided in an embodiment of this application; Figure 2 A flowchart illustrating a control method for a self-moving device provided in an embodiment of this application; Figure 3 This is a schematic diagram of the movement control process in a control method for a self-moving device provided in an embodiment of this application; Figure 4 This is a schematic diagram of a ramp scene in a control method for a self-moving device provided in an embodiment of this application; Figure 5 This is a schematic diagram of movement control in a control method for a self-moving device provided in an embodiment of this application; Figure 6 A flowchart illustrating an example of a self-moving device control method provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a self-moving device provided in an embodiment of this application.

[0034] Figure 8 This is a schematic diagram of the structure of a self-moving device provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0035] Embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0036] The accompanying drawings illustrate various structural schematics according to embodiments of the present disclosure. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0037] In the context of this disclosure, when a layer / element is referred to as being "above" another layer / element, the layer / element may be directly above the other layer / element, or there may be an intermediate layer / element between them. Additionally, if a layer / element is "above" another layer / element in one orientation, then when the orientation is reversed, the layer / element may be "below" the other layer / element.

[0038] Early self-moving devices used preset tracks or manual markings (such as electromagnetic guide lines and magnetic strips) to achieve path tracking, which was costly to deploy and lacked flexibility, and was only suitable for structured environments. With the rise of Simultaneous Localization and Mapping (SLAM) technology, devices have gradually shifted to trackless autonomous navigation, using lidar, cameras, or inertial measurement units (IMUs) to build environmental maps in real time and determine their own location.

[0039] The inventors discovered that current navigation technologies in dynamic scenarios still face the following challenges: Traditional SLAM assumes a static environment; when moving targets such as pedestrians and vehicles are present, point cloud or image features are easily contaminated, leading to positioning drift and map distortion. For example, in densely populated areas, LiDAR may mistakenly identify moving people as static obstacles, triggering unnecessary obstacle avoidance behavior. Furthermore, data fusion between LiDAR and visual sensors requires addressing time synchronization and coordinate alignment issues. In high-speed motion or scenarios with sudden changes in lighting, fusion errors increase significantly, affecting positioning accuracy.

[0040] To address the aforementioned technical issues, some embodiments of this application employ a method of setting up a first sensor and a second sensor with different fields of view on a self-moving device. The richness of spatial information within the first and second fields of view is then acquired, and the target direction corresponding to the field of view with higher spatial information richness is determined. This allows for control of the self-moving device's movement based on sensor information and the target direction. Since a higher degree of spatial information richness within the field of view results in a higher proportion of target spatial information used for positioning and / or obstacle detection, the spatial perception based on the sensor information determined within that field of view is higher, and the accuracy of positioning based on this sensor information is also higher. Unlike existing technologies that rely on a single sensor for positioning and navigation, this application combines sensor information and the target direction to control the movement of the self-moving device, enabling more precise positioning, effectively improving the device's environmental perception capabilities, and further enhancing its environmental adaptability and mobility safety.

[0041] The technical solutions of this application and how they solve the aforementioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0042] This application provides a control method for a self-moving device, which can be applied to self-moving devices, such as... Figure 1 As shown, the self-moving device includes a first sensor 101 and a second sensor 102; the first field of view Q1 of the first sensor 101 corresponds to the current moving direction of the self-moving device, that is, the first field of view Q1 can cover a part of the area corresponding to the current moving direction of the self-moving device; the second field of view Q2 of the second sensor 102 corresponds to the opposite direction of the current moving direction, that is, the second field of view Q2 can cover a part of the area corresponding to the opposite direction of the current moving direction of the self-moving device.

[0043] Wherein, the aforementioned current direction of movement can be the front end of the self-moving device pointing towards, and the opposite direction of the aforementioned current direction of movement can be the rear end of the self-moving device pointing towards. The aforementioned self-moving device can be a lawnmower, a robot vacuum cleaner, or other intelligent device, and is not specifically limited in this embodiment.

[0044] Optionally, the first sensor can be disposed at the front end of the self-moving device, and the second sensor can be disposed at the rear end of the self-moving device. Alternatively, the first and second sensors can also be disposed in the middle of the self-moving device, with the field of view of the first sensor facing the current direction of movement and the field of view of the second sensor facing the opposite direction of movement.

[0045] like Figure 2 As shown, the method includes: S101, respectively obtain the spatial information richness of the first field of view and the second field of view.

[0046] Spatial information richness is used to characterize the proportion of effective spatial information in the target spatial information within the corresponding field of view. Effective spatial information includes target spatial information used for localization and / or obstacle detection. For example, taking a target area with a slope as an example, the effective spatial information collected in the uphill direction is less, with most of it being open sky; while the downhill direction includes not only data on the slope but also data on the flat ground below the slope. Therefore, the spatial openness in the uphill direction is greater than that in the downhill direction, meaning that the effective spatial information corresponding to the uphill direction is less than that corresponding to the downhill direction. The aforementioned target area can be the movement area of ​​the self-moving device.

[0047] Optionally, if the aforementioned self-moving device is a sweeping robot, the aforementioned target area can be a room to be cleaned; if the aforementioned self-moving device is a lawnmower, the aforementioned target area can be an open area such as a meadow.

[0048] Specifically, the self-moving device can determine the spatial information richness within the first field of view based on the first sensing information collected by the first sensor, and determine the spatial information richness within the second field of view based on the second sensing information collected by the second sensor.

[0049] The first or second sensor mentioned above can be an IMU, a lidar sensor, or a vision sensor, etc., and is not specifically limited in this embodiment.

[0050] In some implementations, the first sensor can be a lidar sensor, the effective spatial information can be point cloud data, and the point cloud density within the field of view of the lidar sensor can be used as the richness of the spatial information.

[0051] In other embodiments, the second sensor can be a visual sensor, and the target spatial information can be image data. For example, if the target area is a grassland, the proportion of ineffective spatial information, i.e., sky pixels, to the total number of pixels can be obtained by statistically analyzing the image data, and the richness of spatial information can be determined based on the proportion of sky pixels. The proportion of sky pixels is negatively correlated with the richness of spatial information, i.e., the higher the proportion of sky pixels, the lower the richness of spatial information.

[0052] S102, take the direction corresponding to the field of view with high spatial information richness in the first field of view and the second field of view as the target direction.

[0053] Optionally, if the spatial information richness of the first field of view is higher than that of the second field of view, then the current movement direction can be taken as the target direction; if the spatial information richness of the second field of view is higher than that of the first field of view, then the opposite direction of the current movement direction can be taken as the target direction.

[0054] S103 controls the movement of the self-moving device based on sensor information and target direction.

[0055] The sensor information may include at least one of the first sensing information collected by the first sensor and the second sensing information collected by the second sensor. This sensor information is used for positioning and / or obstacle avoidance of the self-moving device.

[0056] Specifically, the location information of the self-moving device can be determined based on sensor information and target direction, and the movement of the self-moving device can be controlled based on the aforementioned location information. Alternatively, the location information of the self-moving device and target obstacles in the target area can be determined based on sensor information and target direction, and the movement and operating status of the self-moving device can be controlled based on the aforementioned location information and target obstacles.

[0057] If the self-moving device is a lawnmower, the above-mentioned operating states may include mowing state and non-mowing state; if the self-moving device is a robot vacuum cleaner, the above-mentioned operating states may include vacuuming state, mopping state, etc.

[0058] In some implementations, a target sensor can be determined from first and second sensing information based on a target direction, and the movement of the self-moving device can be controlled based on the sensing information collected by the target sensor.

[0059] In other implementations, the actual movement direction of the self-moving device can be determined based on the target direction, and the orientation of the self-moving device can be controlled based on the actual movement direction, so as to control the movement of the self-moving device based on the orientation, the first sensing information and the second sensing information.

[0060] This application embodiment sets up a first sensor and a second sensor with different fields of view on a self-moving device, and obtains the spatial information richness of the first and second fields of view. It then determines the target direction corresponding to the field of view with higher spatial information richness, and controls the movement of the self-moving device based on sensor information and the target direction. Since the higher the spatial information richness of the field of view, the higher the proportion of target spatial information used for positioning and / or obstacle detection within the field of view, the higher the spatial perception of the sensor information determined based on that field of view, and the higher the accuracy of positioning based on that sensor information. Unlike existing technologies that rely on a single sensor for positioning and navigation, this application combines sensor information and target direction to control the movement of the self-moving device, enabling more accurate positioning, effectively improving the environmental perception capability of the self-moving device, and further enhancing its environmental adaptability and mobility safety.

[0061] This application provides a possible implementation method in which the first sensor and the second sensor are different sensors; optionally, the positioning accuracy of the first sensor is greater than the positioning accuracy of the second sensor.

[0062] Specifically, the first sensor and the second sensor can be sensors of the same type, for example, the first sensor is a high-precision vision sensor and the second sensor is a low-precision vision sensor; the first sensor and the second sensor can also be sensors of different types; for example, the first sensor can be a lidar sensor and the second sensor can be a vision sensor, such as a binocular camera.

[0063] like Figure 3 As shown, the above-mentioned control of the movement of the self-moving device based on sensor information and target direction includes: S201, Based on the relationship between the target direction and the current direction of movement, determine the actual direction of movement of the self-moving device.

[0064] The relationship between the target direction and the current movement direction can be either in the same direction or in opposite directions.

[0065] Optionally, taking a lidar sensor as the first sensor and a vision sensor as the second sensor as an example, if the positioning accuracy of the lidar sensor is greater than that of the vision sensor, the lidar sensor can be oriented towards the target direction; that is, the lidar sensor should be oriented towards the direction with higher spatial information richness.

[0066] S202 controls the movement of the self-moving device based on the actual direction of movement and sensor information.

[0067] Specifically, if the target direction and the current movement direction are in the same direction, the current movement direction can be maintained as the actual movement direction, that is, the current orientation of the self-moving device can be maintained, and the movement of the self-moving device can be controlled based on sensor information.

[0068] If the target's actual direction of movement is opposite to the current direction of movement, then the opposite direction of the current direction of movement can be taken as the actual direction of movement, that is, the self-moving device can be controlled to change direction, and the movement of the self-moving device can be controlled based on sensor information.

[0069] In this embodiment, taking a lawnmower as an example of a self-moving device, a LiDAR sensor can be installed at the front end of the lawnmower, and a vision sensor can be installed at the rear end. The positioning accuracy of the LiDAR sensor is greater than that of the vision sensor, and the vision sensor can serve as an auxiliary to the LiDAR sensor; together, they achieve the positioning of the lawnmower. The actual direction of movement of the lawnmower can be determined based on the relationship between the target direction and the current direction of movement.

[0070] Furthermore, if the target direction is in the same direction as the current movement direction, the lidar sensor is facing the direction with higher spatial information richness, which can maintain the current movement direction of the lawnmower and control the movement of the lawnmower based on the sensor information; If the target direction is opposite to the current direction of movement, the lawnmower is controlled to change direction so that the lidar sensor is facing the direction with higher spatial information richness, and the movement of the lawnmower is controlled based on the sensor information.

[0071] This application embodiment determines the actual movement direction of the self-moving device by the relationship between the target direction and the current movement direction, and controls the movement of the self-moving device based on the actual movement direction and sensor information. This ensures that the target sensor with higher positioning accuracy among the first and second sensors is always facing the target direction. By using the target sensor as the main positioning sensor and the other sensors as auxiliary positioning sensors, the data utilization rate of the first and second sensors is effectively improved, and the positioning accuracy of the self-moving device is also improved.

[0072] This application provides a possible implementation method in which the actual movement direction of the self-moving device is determined based on the relationship between the target direction and the current movement direction, including: If the target direction is in the same direction as the current movement direction, then the current movement direction will be taken as the actual movement direction. If the target direction is opposite to the current movement direction, then the opposite direction of the current movement direction will be taken as the actual movement direction.

[0073] Taking a lidar sensor as the first sensor and a vision sensor as the second sensor as an example, the positioning accuracy of the lidar sensor is greater than that of the vision sensor, so the lidar sensor can be oriented towards the target direction; that is, the lidar sensor should be oriented towards the direction with higher spatial information richness.

[0074] In the embodiments of this application, such as Figure 4 As shown, taking a slope as the target area, less spatial information can be collected uphill, with most of it being open sky. Especially when the mobile device moves from the slope to the flat ground on the slope, the sensor in the current direction of movement cannot collect data from the flat ground on the slope. However, the downhill direction includes not only data from the slope but also data from the flat ground below the slope, so there is more spatial information to collect downhill. Therefore, the spatial information richness in the downhill direction is higher than that in the uphill direction, and the target direction is downhill. When the mobile device needs to go uphill, if the current direction of movement is uphill, the target direction is opposite to the current direction of movement. Therefore, the mobile device needs to be turned around, and the target direction, i.e., the downhill direction, is taken as the actual direction of movement. This ensures that the visual sensor 301 faces uphill and the lidar sensor 302 always faces downhill, enabling the collection of more accurate sensing information.

[0075] In this embodiment, taking a lawnmower as an example of a self-moving device, a LiDAR sensor can be installed at the front end of the lawnmower, and a vision sensor can be installed at the rear end. The positioning accuracy of the LiDAR sensor is greater than that of the vision sensor, and the vision sensor can serve as an auxiliary to the LiDAR sensor; together, they achieve the positioning of the lawnmower. The actual direction of movement of the lawnmower can be determined based on the relationship between the target direction and the current direction of movement.

[0076] Specifically, if the target direction is in the same direction as the current movement direction, the lidar sensor is facing the direction with higher spatial information richness, which can maintain the current movement direction of the lawnmower and control the movement of the lawnmower based on the sensor information; If the target direction is opposite to the current direction of movement, the lawnmower is controlled to change direction, that is, the rear end of the lawnmower moves forward and backward, so that the lidar sensor is facing the direction with higher spatial information richness, and the movement of the lawnmower is controlled based on the sensor information.

[0077] This application embodiment can determine the actual movement direction by the relationship between the target direction and the current movement direction; it ensures that the lidar sensor is always facing the target direction, using the lidar sensor as the main positioning sensor and the vision sensor as the auxiliary positioning sensor, effectively improving the data utilization rate of the lidar sensor and the vision sensor, while also improving the positioning accuracy of the self-moving device, so as to ensure the further improvement of the intelligence level and scene adaptability of the subsequent movement control of the self-moving device.

[0078] This application provides one possible implementation method, such as... Figure 5 As shown, the above-mentioned control of the movement of the self-moving device based on the actual direction of movement and sensor information includes: S301, if the actual movement direction is the current movement direction, then control the self-moving device to maintain the current movement direction; if the actual movement direction is the opposite direction of the current movement direction, then control the self-moving device to change direction.

[0079] Specifically, taking a lidar sensor as the first sensor and a vision sensor as the second sensor as an example, if the actual movement direction is the current movement direction, the lidar sensor faces the direction with higher spatial information richness, that is, the spatial information richness of the lidar sensor's field of view is greater than that of the vision sensor's field of view, and the current movement direction can be maintained; if the actual movement direction is the target direction, the lidar sensor faces the direction with lower spatial information richness, that is, the spatial information richness of the lidar sensor's field of view is less than that of the vision sensor's field of view, and the self-moving device can be controlled to change direction, so that the lidar sensor faces the direction with higher spatial information richness before proceeding with subsequent movement control steps.

[0080] S302 determines the real-time displacement of the self-moving device based on sensor information.

[0081] Specifically, the real-time displacement of the self-moving device within a unit of time can be determined based on the information from the first sensor and the information from the second sensor.

[0082] The aforementioned unit of time can be determined based on the sensor data acquisition time. The specific steps for determining the real-time displacement will be explained in detail below.

[0083] S303 determines obstacle information in the target area based on sensor information.

[0084] The obstacle information may include the type of obstacle and the real-time distance between the obstacle and the self-moving device.

[0085] Optionally, the above obstacle types may include static obstacles and dynamic obstacles.

[0086] Specifically, obstacle detection can be performed on the target area based on sensor information to obtain obstacle information. If the obstacle type is a static obstacle, the real-time distance between the obstacle and the self-moving device is obtained; if the obstacle type is a dynamic obstacle, the real-time distance between the obstacle and the self-moving device is obtained, and the movement direction of the dynamic obstacle is determined.

[0087] S304 controls the movement of the self-moving device based on real-time displacement, obstacle information, and pre-movement path.

[0088] The pre-movement path can be a path generated based on a pre-built map.

[0089] Specifically, taking a lawnmower as an example, if the obstacle is a dynamic obstacle such as a person or animal, and the obstacle is getting closer to the lawnmower in real time, the lawnmower will be stopped moving and the blades will be stopped rotating to pause the mowing operation and prevent the blades from accidentally injuring people or animals. Simultaneously, after pausing the mowing operation, the movement status of the dynamic obstacle can be determined based on real-time obstacle information. If the dynamic obstacle has moved away from the lawnmower, the lawnmower can be controlled to move along a pre-defined path based on real-time displacement.

[0090] If the obstacle type is a static obstacle and the static obstacle is located in the pre-movement path, the lawnmower will avoid the obstacle based on real-time displacement control.

[0091] If neither of the above two scenario types exists, the lawnmower can be moved normally along the pre-moving path based on real-time displacement control.

[0092] This embodiment controls the LiDAR sensor of the self-moving device to always face the direction with higher spatial information richness by controlling the actual movement direction. This allows the self-moving device to collect more accurate sensor information, enabling the determination of more precise obstacle information and real-time displacement data, effectively improving the environmental perception capability of the self-moving device and increasing its mobility efficiency. Simultaneously, by using the LiDAR sensor as the primary sensor and the vision sensor as the auxiliary sensor, the positioning accuracy of the vision sensor only needs to meet the safety requirements of the self-moving device. Therefore, the positioning accuracy of the vision sensor does not need to be too high, effectively reducing sensor costs. Real-time environmental modeling and dynamic obstacle avoidance are achieved based on the LiDAR and vision sensors, ensuring safe operation in environments with high pedestrian traffic.

[0093] This application provides a possible implementation method, wherein the sensor information includes first sensing information and second sensing information.

[0094] Optionally, the first sensing information may include point cloud data, and the second sensing information may include image data.

[0095] The above-mentioned method of determining the real-time displacement of a self-moving device based on sensor information includes: If the first sensing information is detected as a valid signal, then the first real-time displacement of the self-moving device is determined based on the first sensing information; and the real-time displacement is determined based on the first real-time displacement. If the first sensor information is detected as an invalid signal, the second real-time displacement of the self-moving device is determined based on the second sensor information, and the second real-time displacement is used as the real-time displacement.

[0096] Specifically, taking a LiDAR sensor as the first sensor and a vision sensor as the second sensor as an example, the first real-time displacement can be calculated based on the following method: For example, a lidar sensor can output a point cloud map containing point cloud data every 100ms. Key features such as edge points, planar points, and corner points can be extracted from the point cloud data. Then, the key features are matched with the pre-built map to obtain the position and orientation of the lidar sensor relative to the map coordinate system, so as to determine the first real-time displacement.

[0097] Understandably, LiDAR sensors and vision sensors can acquire data at the same frame rate, for example, both outputting sensing information at 100ms intervals. The two sensors can use system timestamps for data matching and fusion to ensure the synchronization of sensing data.

[0098] In some implementations, when the first sensing information is a valid signal, the first real-time displacement can be determined based on the first sensing information and directly used as the real-time displacement of the self-moving device.

[0099] In other embodiments, if the first sensing information is a valid signal, the first real-time displacement can be determined based on the first sensing information, and the second real-time displacement can be corrected based on the second sensing information to obtain the final real-time displacement. The specific calculation steps will be described in detail below.

[0100] Optionally, if there are abnormal situations such as the lidar sensor being covered by foreign objects or dirt, or the presence of low-reflectivity objects within the first field of view (e.g., tree trunks, metal doors, mirrors, etc.), these abnormal situations will result in excessively low point cloud density in the point cloud data. For example, if the number of valid point clouds collected within the lidar sensor's field of view is less than 10% of the maximum number of point clouds in the field of view, then the point cloud data is determined to be an invalid signal. Furthermore, unlike the lidar sensor which is the main sensor and the vision sensor which is an auxiliary sensor and a backup sensor, the second sensing information is always assumed to be valid. The situation where the second sensing information is invalid is not discussed in this embodiment.

[0101] Furthermore, if the first sensing information is detected as an invalid signal, the state of the lidar sensor can be detected in real time. If the first sensor recovers to an valid signal, global positioning can be performed based on the lidar sensor, and then the first real-time displacement of the self-moving device can be determined based on the first sensing information, and the real-time displacement can be determined based on the first real-time displacement.

[0102] This embodiment uses a lidar sensor as the primary sensor. If its first sensing information is a valid signal, the real-time displacement of the self-moving device can be determined based on the first real-time displacement. If the first sensing information is an invalid signal, the second sensing information of the secondary sensor, namely the vision sensor, can be activated, and the second real-time displacement can be used as the real-time displacement of the self-moving device. This embodiment achieves positioning or navigation using a lidar sensor as the primary sensor, with the lidar sensor always collecting directions with high spatial information richness. Simultaneously, it can dynamically switch between the lidar sensor and the vision sensor based on the signal state of the lidar sensor, effectively ensuring the accuracy of the real-time displacement information and laying a solid foundation for subsequent movement control of the self-moving device.

[0103] This application provides a possible implementation method in which the above-mentioned determination of real-time displacement based on the first real-time displacement includes: S401, determine the second real-time displacement of the self-moving device based on the second sensor information.

[0104] Taking the second sensor as a visual sensor as an example, the visual sensor can acquire an image every preset time interval, such as 100ms. The aforementioned second sensor information can include a sequence of images from multiple consecutive frames.

[0105] Specifically, feature detection algorithms can be used to extract features from each frame of image data in a sequence of multiple consecutive frames, thereby obtaining feature point data in each frame of image data. The feature detection algorithm can be a feature from Accelerated Segment Test (FAST), a Scale-Invariant Feature Transform (SIFT) algorithm, or an Oriented FAST and Rotated BRIEF (ORB) algorithm, and is not specifically limited in this embodiment.

[0106] Furthermore, by combining the inherent parameters of the visual sensor, such as the intrinsic or extrinsic parameters of the camera, feature matching can be performed on the feature point data between adjacent frames to obtain the relative motion displacement between adjacent frames; then, based on the relative motion displacement between each adjacent frame, the global displacement of the self-moving device, i.e., the second real-time displacement, can be calculated.

[0107] In this embodiment, since the visual sensor requires real-time continuous frame images for positioning, even when displacement detection is performed based on the first sensing information of the LiDAR sensor, the visual sensor also needs to simultaneously acquire the second sensing information. This is so that when the LiDAR tactile sensor malfunctions, i.e. when the first sensing information is invalid, the self-moving device can acquire the second sensing information of the visual sensor at any time for displacement detection. Since the visual sensor is always in working condition, there is no need to perform initialization operations such as repositioning, which can effectively ensure the continuity of real-time displacement detection.

[0108] S402, determine the real-time displacement based on the difference between the first real-time displacement and the second real-time displacement.

[0109] Specifically, the first real-time displacement can be calibrated based on the second real-time displacement to obtain the final real-time displacement. The specific calibration steps will be explained in detail below.

[0110] The embodiments of this application determine the real-time displacement of the self-moving device by using a first real-time displacement and a second real-time displacement, thereby realizing the information fusion of the first sensing information of the lidar sensor and the second sensing information of the vision sensor, and effectively improving the accuracy of the real-time displacement.

[0111] This application provides a possible implementation method in which the determination of real-time displacement based on the difference between the first real-time displacement and the second real-time displacement includes: If the difference between the first real-time displacement and the second real-time displacement is less than a preset difference threshold, then the first real-time displacement is taken as the real-time displacement.

[0112] The aforementioned difference threshold can be 10cm.

[0113] In some implementations, if the difference between the first real-time displacement and the second real-time displacement is less than a preset difference threshold, it indicates that the sensing data of the lidar sensor and the vision sensor are basically consistent, and the first real-time displacement corresponding to the lidar sensor can be used as the real-time displacement.

[0114] This application provides a possible implementation method, which, before controlling the movement of the self-moving device based on real-time displacement, obstacle information, and pre-movement path, further includes: If the difference between the first real-time displacement and the second real-time displacement is not less than the preset difference threshold, then the self-moving device is controlled to stop moving and a maintenance reminder is sent.

[0115] In other implementations, if the difference between the first real-time displacement and the second real-time displacement is not less than a preset difference threshold, it indicates a large discrepancy between the sensing data of the lidar sensor and the vision sensor, which may be due to errors in the sensing information caused by external foreign objects or signal interference from the lidar sensor and / or the vision sensor. Taking a lawnmower as an example, if the difference between the first real-time displacement and the second real-time displacement is detected to be not less than the preset difference threshold, the lawnmower can be controlled to stop moving and the blades of the lawnmower can be controlled to stop rotating, thereby pausing the mowing operation. At the same time, the lawnmower can control the lidar sensor and the vision sensor to perform self-checks, and after confirming a problem, it can send a maintenance reminder so that the user can receive the reminder and perform maintenance or restart the lawnmower.

[0116] This application embodiment determines the sensing and positioning difference between the LiDAR sensor and the vision sensor by measuring the difference between the first and second real-time displacements. If the difference is large, it indicates a fault in the LiDAR sensor and / or the vision sensor, allowing the self-moving device to stop and perform a self-check. If the difference is within a preset range, it indicates that the sensing data of the LiDAR sensor and the vision sensor are basically consistent, and the first real-time displacement corresponding to the LiDAR sensor can be used as the real-time displacement. This application embodiment implements data verification between the LiDAR sensor and the vision sensor, further ensuring accurate detection of the real-time displacement of the self-moving device when both the LiDAR sensor and the vision sensor are fault-free. In this application, the first sensing information collected by the LiDAR sensor is used as the main positioning data, and the first sensing information is verified based on the vision sensor, effectively ensuring the environmental awareness of the self-moving device and improving the user experience.

[0117] This application provides a possible implementation method in which the determination of the first real-time displacement of the self-moving device based on the first sensing information includes: If it is detected that all the first sensing information is a valid signal within a preset time range, then the first real-time displacement of the self-moving device is determined based on the first sensing information. If it is detected that the first sensing information changes from an invalid signal to an effective signal within a preset time range, then the first sensor is controlled to reposition itself, and the real-time sensing information of the first sensor is acquired. Based on the real-time sensing information, the first real-time displacement of the self-moving device is determined.

[0118] The preset duration can be 15 minutes.

[0119] Specifically, taking a laser sensor as the first sensor as an example, the validity status of the first sensing information collected by the laser radar sensor can be checked every 15 minutes. If the first sensing information is a valid signal, real-time displacement can be detected based on the main positioning sensor, i.e., the laser radar sensor. If the first sensing information is a invalid signal, the displacement of the self-moving device can be detected based on the auxiliary sensor, i.e., the vision sensor. If the signal status of the first sensing information is detected to switch from invalid to valid, the laser radar sensor can be controlled to reposition after the first sensing information becomes valid. After successful repositioning, the first real-time displacement of the self-moving device can be determined based on the real-time sensing information collected by the laser radar sensor.

[0120] This application embodiment performs validity detection on the LiDAR sensor signal based on a preset time period. When the main sensor, i.e., the LiDAR sensor, fails, a visual sensor is used for displacement detection, ensuring the continuity of sensing information and uninterrupted movement of the self-moving device. If the first sensing information of the LiDAR sensor recovers from an invalid state to a valid state, repositioning is first performed based on the LiDAR sensor, and then the displacement detection is switched from the auxiliary sensor, i.e., the visual sensor, back to the LiDAR sensor, effectively improving the reliability and accuracy of the displacement data.

[0121] This application provides a possible implementation method, wherein the sensor information includes first sensing information or second sensing information.

[0122] Controlling the movement of a self-moving device based on sensor information and target orientation includes: If the target direction is in the same direction as the current movement direction, the movement of the self-moving device is controlled based on the first sensor information; If the target direction is opposite to the current movement direction, the movement of the self-moving device is controlled based on the second sensor information.

[0123] The first or second sensor mentioned above can be a lidar sensor, an infrared sensor, an ultrasonic sensor, or a vision sensor, etc.; the first and second sensors can be the same type of sensor or different types of sensors, and no specific limitation is made in this embodiment.

[0124] In some implementations, if the target direction is the current movement direction, the field of view of the first sensor faces the direction with higher spatial information richness, and the field of view of the second sensor faces the direction with lower spatial information richness. Therefore, relative to the second sensing information, the spatial information richness corresponding to the field of view of the first sensor is higher than the spatial information richness corresponding to the field of view of the second sensor. The first sensing information can achieve more accurate positioning, and the movement of the self-moving device can be controlled based on the first sensing information. In other implementations, if the target direction is the opposite of the current movement direction, the field of view of the second sensor faces the direction with higher spatial information richness, while the field of view of the first sensor faces the direction with lower spatial information richness. In this case, the second sensor information can achieve more accurate positioning than the first sensor information, and the movement of the self-moving device can be controlled based on the second sensor information.

[0125] In this embodiment, the target sensor is determined by comparing the target direction with the current movement direction. If the target direction is the same as the current movement direction, the target sensor is the first sensor, and the movement of the self-moving device is controlled based on the first sensing information. If the target direction is opposite to the current movement direction, the target sensor is the second sensor, and the movement of the self-moving device is controlled based on the second sensing information. In other words, in this embodiment, the direction sensor with a higher field of view and richer spatial information is always used as the target sensor, so that the sensing information collected by the target sensor is more accurate and effective, and the self-moving device can achieve more accurate positioning or obstacle avoidance based on the target sensor.

[0126] This application provides a possible implementation method in which the movement of the self-moving device is controlled based on the first sensing information, including: The information acquisition frequency of the first sensor is increased to a first preset frequency, and the first sensing information is acquired based on the first preset frequency; the movement of the self-moving device is controlled based on the first sensing information. Controlling the movement of a self-moving device based on second sensor information includes: The information acquisition frequency of the second sensor is increased to a second preset frequency, and the second sensing information is acquired based on the second preset frequency; the movement of the self-moving device is controlled based on the second sensing information acquired by the second sensor.

[0127] In this embodiment, taking a lawnmower with a LiDAR sensor at the front and a vision sensor at the rear as an example, if the target area where the lawnmower is located is a slope, and the lawnmower is going uphill, the number of point clouds that the LiDAR sensor at the front can collect is much less than that on flat ground. Especially when moving from a slope to the top of the slope, the LiDAR sensor at the front cannot collect the point cloud on the top of the slope, which may lead to loss of positioning. In this state, positioning and obstacle detection can be performed based on the vision sensor at the rear. If the lawnmower changes from going uphill to going downhill, the lawnmower does not need to change direction or reverse. Positioning and navigation can still be performed based on the vision sensor at the rear, so as to further improve the accuracy of positioning and detection and the moving efficiency of the lawnmower.

[0128] In this application embodiment, when the target sensor is determined to be the first sensor and the movement of the self-moving device needs to be controlled based on the first sensing information, the information acquisition frequency of the first sensor can be increased from the initial frequency to a first preset frequency, further improving the quality and reliability of the first sensing information. Conversely, when the target sensor is determined to be the second sensor and the movement of the self-moving device needs to be controlled based on the second sensing information, the information acquisition frequency of the second sensor can be increased from the initial frequency to a first preset frequency, thereby improving the quality and reliability of the second sensing information. In other words, this application further improves the quality and effectiveness of sensing information by increasing the acquisition frame rate of the target sensor, laying a good foundation for the subsequent accurate and efficient positioning or obstacle avoidance operations of the self-moving device.

[0129] To better understand the control method of the self-moving device described above, the following will combine... Figure 6 A detailed example of a control method for a self-moving device according to this application is provided, taking a lawnmower as an example. A lidar sensor is installed at the front of the lawnmower and a vision sensor is installed at the rear of the lawnmower. The field of view of the lidar sensor faces the front of the lawnmower and is used to collect spatial data in front of it. The field of view of the vision sensor faces the rear of the lawnmower and is used to collect spatial data in the rear. The lidar sensor is connected to the controller motherboard of the lawnmower through a Mobile Industry Processor Interface (MIPI), and the vision sensor is connected to the controller motherboard through a Digital Signal Processor (DSP). The two sensors are time-stamp aligned through the system clock.

[0130] The method includes the following steps: S501 collects spatial information of the grassland to be operated based on a lidar sensor, and determines the spatial information richness of the first field of view and the second field of view based on the spatial information.

[0131] Specifically, the aforementioned lidar sensor can rotate 360° in the horizontal plane. After the lawnmower enters the work area, it can control the lidar sensor to rotate in the horizontal plane, collecting spatial information of the work area during the rotation. Alternatively, after the lawnmower enters the work area, it can control the lawnmower to rotate in the horizontal plane, collecting spatial information of the work area during the rotation.

[0132] The first field of view is the field of view of the lidar sensor, and the second field of view is the field of view of the vision sensor.

[0133] S502, take the direction corresponding to the field of view with higher spatial information richness in the first field of view and the second field of view as the target direction.

[0134] Specifically, within the first and second fields of view, if the spatial information richness of the first field of view is higher (i.e., the spatial information richness of the first field of view is greater than that of the second field of view), the direction of movement can be taken as the target direction; if the spatial information richness of the second field of view is higher (i.e., the spatial information richness of the first field of view is less than that of the second field of view), the opposite direction of movement can be taken as the target direction.

[0135] S503: If the target direction is the same as the direction of movement of the lawnmower, the current direction of movement is maintained; if the target direction is opposite to the direction of movement of the lawnmower, the lawnmower is controlled to change direction so that the lidar sensor faces the target direction.

[0136] S504 determines the real-time displacement of the lawnmower based on point cloud data collected by the lidar sensor and image data collected by the vision sensor.

[0137] In some implementations, if the point cloud data is detected as valid data, the first real-time displacement of the lawnmower is determined based on the point cloud data, and the second real-time displacement of the lawnmower is determined based on the image data. If the difference between the first real-time displacement and the second real-time displacement is less than 10cm, the first real-time displacement is taken as the real-time displacement of the lawnmower. Otherwise, the lawnmower stops moving and the cutting disc rotation is paused. The lidar sensor and vision sensor perform self-checks and send fault alerts to the user terminal.

[0138] In other implementations, if invalid point cloud data is detected, a second real-time displacement of the lawnmower is determined based on the image data, and the second real-time displacement is used as the real-time displacement of the lawnmower.

[0139] The S505 detects obstacle information in the working area based on point cloud data collected by a lidar sensor and / or image data collected by a vision sensor; and controls the movement of the lawnmower based on real-time displacement, obstacle information, and pre-movement path.

[0140] This application provides a self-moving device, such as... Figure 7 As shown, the self-moving device 70 includes a controller 701, a first sensor 702, and a second sensor 703; the first field of view of the first sensor corresponds to the current moving direction of the self-moving device; the second field of view of the second sensor corresponds to the opposite direction of the current moving direction. The controller 701 described above is configured as follows: The spatial information richness of the first field of view and the second field of view are obtained respectively; wherein, the spatial information richness is used to characterize the proportion of effective spatial information in the target spatial information within the corresponding field of view; the effective spatial information includes target spatial information used for localization and / or obstacle detection; The direction corresponding to the field of view with higher spatial information richness among the first and second fields of view is taken as the target direction. The movement of the self-moving device is controlled based on sensor information and target orientation.

[0141] The sensor information includes at least one of the first sensing information collected by the first sensor and the second sensing information collected by the second sensor.

[0142] This application embodiment can set up a first sensor and a second sensor with different fields of view on the self-moving device, and obtain the spatial information richness of the first field of view and the second field of view. The target direction corresponding to the field of view with higher spatial information richness is determined, and the movement of the self-moving device is controlled based on sensor information and target direction. Since the higher the spatial information richness of the field of view, the higher the proportion of target spatial information used for positioning and / or obstacle detection within the target spatial information within the field of view, the higher the spatial perception of the sensor information determined based on that field of view, and the higher the accuracy of positioning based on that sensor information. Unlike existing technologies that rely on a single sensor for positioning and navigation, this application combines sensor information and target direction to control the movement of the self-moving device, enabling more accurate positioning, effectively improving the environmental perception capability of the self-moving device, and further enhancing its environmental adaptability and mobility safety.

[0143] This application provides a possible implementation method in which the first sensor and / or the second sensor are visual sensors; Along the height direction of the self-moving device, the optical axis of the vision sensor is oriented toward the moving surface of the self-moving device, and the angle between it and the mounting plane is the first angle; the maximum angle of the field of view of the vision sensor in the vertical direction is the second angle, and the first angle is determined based on the height of the mounting plane above the ground and the second angle.

[0144] The aforementioned mounting plane is a plane parallel to the horizontal plane and the plane where the vision sensor is located.

[0145] like Figure 8As shown, taking a lawnmower as an example of a self-moving device, the lawnmower has a LiDAR sensor 701 at the front and a vision sensor 702 at the rear. The LiDAR sensor has a horizontal field of view (FOV) greater than 120° and a vertical FOV of β, where β > 80°. The vision sensor has a horizontal FOV greater than 120° and a vertical FOV of α, where α > 100°. The LiDAR sensor is connected to the lawnmower's controller motherboard via a MIPI interface, and the vision sensor is connected to the controller motherboard via a DSP. The two sensors are time-stamp aligned using the system clock and are fused using a loosely coupled method.

[0146] Optionally, the optical axis of the vision sensor is tilted downwards, with a tilt angle θ ∈ [9°, 20°]; the specific tilt angle θ is determined based on the installation height h of the vision sensor. For example, Where l is the blind zone distance of the vision sensor in the direction of the lawnmower, and l < 10cm.

[0147] This application provides a possible implementation method in which the first sensor and the second sensor are different sensors; The above-mentioned control of the movement of the self-moving device based on sensor information and target orientation includes: The actual movement direction of the self-moving device is determined based on the relationship between the target direction and the current movement direction. The movement of the self-moving device is controlled based on the actual direction of movement and sensor information.

[0148] This application provides a possible implementation method in which the actual movement direction of the self-moving device is determined based on the relationship between the target direction and the current movement direction, including: If the target direction is in the same direction as the current movement direction, then the current movement direction will be taken as the actual movement direction. If the target direction is opposite to the current movement direction, then the opposite direction of the current movement direction will be taken as the actual movement direction.

[0149] This application provides a possible implementation method in which the movement of the self-moving device is controlled based on the actual movement direction and sensor information, including: If the actual movement direction is the current movement direction, then control the self-moving device to maintain the current movement direction; if the actual movement direction is the opposite of the current movement direction, then control the self-moving device to change direction. The real-time displacement of the self-moving device is determined based on sensor information; Determine obstacle information in the target area based on sensor information; The movement of the self-moving device is controlled based on real-time displacement, obstacle information, and pre-movement path.

[0150] This application provides a possible implementation method, wherein the sensor information includes first sensing information and second sensing information; The above-mentioned method of determining the real-time displacement of a self-moving device based on sensor information includes: If the first sensing information is detected as a valid signal, then the first real-time displacement of the self-moving device is determined based on the first sensing information; and the real-time displacement is determined based on the first real-time displacement. If the first sensor information is detected as an invalid signal, the second real-time displacement of the self-moving device is determined based on the second sensor information, and the second real-time displacement is used as the real-time displacement.

[0151] This application provides a possible implementation method in which the above-mentioned determination of real-time displacement based on the first real-time displacement includes: The second real-time displacement of the self-moving device is determined based on the second sensor information; The real-time displacement is determined based on the difference between the first real-time displacement and the second real-time displacement.

[0152] This application provides a possible implementation method in which the determination of real-time displacement based on the difference between the first real-time displacement and the second real-time displacement includes: If the difference between the first real-time displacement and the second real-time displacement is less than a preset difference threshold, then the first real-time displacement is taken as the real-time displacement. This application embodiment provides a possible implementation, wherein the controller 701 is further configured as follows: If the difference between the first real-time displacement and the second real-time displacement is not less than the preset difference threshold, then the self-moving device is controlled to stop moving and a maintenance reminder is sent.

[0153] This application provides a possible implementation method in which the determination of the first real-time displacement of the self-moving device based on the first sensing information includes: If it is detected that all the first sensing information is a valid signal within a preset time range, then the first real-time displacement of the self-moving device is determined based on the first sensing information. If it is detected that the first sensing information changes from an invalid signal to an effective signal within a preset time range, then the first sensor is controlled to reposition itself, and the real-time sensing information of the first sensor is acquired. Based on the real-time sensing information, the first real-time displacement of the self-moving device is determined.

[0154] This application provides a possible implementation method, wherein the sensor information includes first sensing information or second sensing information; The above-mentioned control of the movement of the self-moving device based on sensor information and target orientation includes: If the target direction is in the same direction as the current movement direction, the movement of the self-moving device is controlled based on the first sensor information; If the target direction is opposite to the current movement direction, the movement of the self-moving device is controlled based on the second sensor information.

[0155] This application provides a possible implementation method in which the movement of the self-moving device is controlled based on the first sensing information, including: The information acquisition frequency of the first sensor is increased to a first preset frequency, and the first sensing information is acquired based on the first preset frequency; the movement of the self-moving device is controlled based on the first sensing information. The above-mentioned control of the movement of the self-moving device based on the second sensor information includes: The information acquisition frequency of the second sensor is increased to a second preset frequency, and the second sensing information is acquired based on the second preset frequency; the movement of the self-moving device is controlled based on the second sensing information acquired by the second sensor.

[0156] This application provides a possible implementation method in which the first sensor can be a lidar sensor and the second sensor can be a vision sensor.

[0157] The apparatus in this application embodiment can execute the method provided in this application embodiment, and the implementation principle is similar. The actions performed by each module in the apparatus of each embodiment of this application correspond to the steps in the method of each embodiment of this application. For detailed functional descriptions of each module of the apparatus, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0158] This application embodiment sets up a first sensor and a second sensor with different fields of view on a self-moving device, and obtains the spatial information richness of the first and second fields of view. It then determines the target direction corresponding to the field of view with higher spatial information richness, and controls the movement of the self-moving device based on sensor information and the target direction. Since the higher the spatial information richness of the field of view, the higher the proportion of target spatial information used for positioning and / or obstacle detection within the field of view, the higher the spatial perception of the sensor information determined based on that field of view, and the higher the accuracy of positioning based on that sensor information. Unlike existing technologies that rely on a single sensor for positioning and navigation, this application combines sensor information and target direction to control the movement of the self-moving device, enabling more accurate positioning, effectively improving the environmental perception capability of the self-moving device, and further enhancing its environmental adaptability and mobility safety.

[0159] This application provides an electronic device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of a control method for a self-moving device. Compared with related technologies, this application can achieve the following: By setting a first sensor and a second sensor with different fields of view on the self-moving device, and acquiring the spatial information richness of the first and second fields of view, the target direction corresponding to the field of view with higher spatial information richness is determined, so as to control the movement of the self-moving device based on sensor information and target direction. Since the higher the spatial information richness of the field of view, the higher the proportion of target spatial information used for positioning and / or obstacle detection in the target spatial information within the field of view, the higher the spatial perception of the sensor information determined based on the field of view, and the higher the accuracy of positioning based on the sensor information. Unlike the prior art that uses a single sensor for positioning and navigation, this application combines sensor information and target direction to control the movement of the self-moving device, which can achieve more accurate positioning of the self-moving device, effectively improve the environmental perception capability of the self-moving device, and further improve the environmental adaptability and movement safety of the self-moving device.

[0160] In one alternative embodiment, an electronic device is provided, such as Figure 9 As shown, Figure 9 The illustrated electronic device 90 includes a processor 901 and a memory 903. The processor 901 and the memory 903 are connected, for example, via a bus 902. Optionally, the electronic device 90 may further include a transceiver 904, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 904 is not limited to one type, and the structure of the electronic device 90 does not constitute a limitation on the embodiments of this application.

[0161] Processor 901 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 901 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0162] Bus 902 may include a pathway for transmitting information between the aforementioned components. Bus 902 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 902 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0163] The memory 903 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0164] The memory 903 stores computer programs that execute embodiments of this application, and its execution is controlled by the processor 901. The processor 901 executes the computer programs stored in the memory 903 to implement the steps shown in the foregoing method embodiments.

[0165] The electronic devices include, but are not limited to, mobile terminals such as mobile phones, laptops, and tablets, as well as fixed terminals such as digital TVs and desktop computers. These mobile or fixed terminals can communicate with the mobile devices via wireless or wired networks.

[0166] This application provides a non-transitory computer-readable storage medium storing computer instructions, which are used to cause a computer to perform the steps of the method shown in the first aspect of this application.

[0167] The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access, or it can include one or more data storage devices such as servers or data centers that can be integrated with media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media, or semiconductor media (e.g., solid-state disks (SSDs)).

[0168] This application provides a computer program product including instructions that, when executed, cause a computer to perform the steps of the method shown in the first aspect of this application.

[0169] The computer storage medium and computer program products provided in the embodiments of this application are used to execute the methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects corresponding to the cleaning equipment provided above, and will not be repeated here.

[0170] In the above embodiments, implementation can also be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line, DSL) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0171] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0172] The above description does not provide detailed technical specifications regarding the structure of each layer. However, those skilled in the art should understand that layers and regions of desired shapes can be formed using various technical means. Furthermore, to form the same structure, those skilled in the art can also design methods that are not entirely identical to those described above. Additionally, although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be advantageously combined.

[0173] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0174] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A control method for a self-moving device, characterized in that, The self-moving device includes a first sensor and a second sensor; a first field of view of the first sensor corresponds to the current movement direction of the self-moving device; a second field of view of the second sensor corresponds to the opposite direction of the current movement direction; the method includes: The spatial information richness of the first field of view and the second field of view are obtained respectively; wherein, the spatial information richness is used to characterize the proportion of effective spatial information in the target spatial information within the corresponding field of view; the effective spatial information includes target spatial information used for localization and / or obstacle detection; The direction corresponding to the field of view with high spatial information richness in the first field of view and the second field of view is taken as the target direction; The movement of the self-moving device is controlled based on sensor information and the target direction, wherein the sensor information includes at least one of first sensing information collected by the first sensor and second sensing information collected by the second sensor.

2. The method according to claim 1, characterized in that, The first sensor and the second sensor are different sensors; The method of controlling the movement of the self-moving device based on sensor information and the target direction includes: Based on the relationship between the target direction and the current direction of movement, the actual direction of movement of the self-moving device is determined; The movement of the self-moving device is controlled based on the actual direction of movement and the sensor information.

3. The method according to claim 2, characterized in that, Determining the actual movement direction of the self-moving device based on the relationship between the target direction and the current movement direction includes: If the target direction is in the same direction as the current movement direction, then the current movement direction is taken as the actual movement direction; If the target direction is opposite to the current movement direction, then the opposite direction of the current movement direction is taken as the actual movement direction.

4. The method according to claim 3, characterized in that, The method of controlling the movement of the self-moving device based on the actual direction of movement and the sensor information includes: If the actual movement direction is the current movement direction, then the self-moving device is controlled to maintain the current movement direction; if the actual movement direction is the opposite direction of the current movement direction, then the self-moving device is controlled to change direction. The real-time displacement of the self-moving device is determined based on the sensor information; Based on the sensor information, determine the obstacle information of the target area; The movement of the self-moving device is controlled based on the real-time displacement, the obstacle information, and the pre-movement path.

5. The method according to claim 4, characterized in that, The sensor information includes first sensing information and second sensing information; Determining the real-time displacement of the self-moving device based on the sensor information includes: If the first sensing information is detected as a valid signal, then the first real-time displacement of the self-moving device is determined based on the first sensing information; the real-time displacement is then determined based on the first real-time displacement. If the first sensing information is detected as an invalid signal, then the second real-time displacement of the self-moving device is determined based on the second sensing information, and the second real-time displacement is used as the real-time displacement.

6. The method according to claim 5, characterized in that, Determining the real-time displacement based on the first real-time displacement includes: The second real-time displacement of the self-moving device is determined based on the second sensing information; The real-time displacement is determined based on the difference between the first real-time displacement and the second real-time displacement.

7. The method according to claim 6, characterized in that, Determining the real-time displacement based on the difference between the first real-time displacement and the second real-time displacement includes: If the difference between the first real-time displacement and the second real-time displacement is less than a preset difference threshold, then the first real-time displacement is taken as the real-time displacement.

8. The method according to claim 7, characterized in that, Before controlling the movement of the self-moving device based on the real-time displacement, the obstacle information, and the pre-movement path, the method further includes: If the difference between the first real-time displacement and the second real-time displacement is not less than a preset difference threshold, then the self-moving device is controlled to stop moving and a maintenance reminder is sent.

9. A self-moving device, characterized in that, The self-moving device includes a controller, a first sensor, and a second sensor; the first field of view of the first sensor corresponds to the current direction of movement of the self-moving device; the second field of view of the second sensor corresponds to the opposite direction of movement; the controller is configured to: The spatial information richness of the first field of view and the second field of view are obtained respectively; wherein, the spatial information richness is used to characterize the proportion of effective spatial information in the target spatial information within the corresponding field of view; the effective spatial information includes target spatial information used for localization and / or obstacle detection; The direction corresponding to the field of view with high spatial information richness in the first field of view and the second field of view is taken as the target direction; The movement of the self-moving device is controlled based on sensor information and the target direction, wherein the sensor information includes at least one of first sensing information collected by the first sensor and second sensing information collected by the second sensor.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.

Images