Method for processing depth data, computer readable medium and electronic device

Abstract

The application discloses a kind of processing method of depth data, computer readable medium and electronic equipment, the method includes: obtaining current frame depth image from the moving direction of mobile device, and extracting current frame depth data from current frame depth image;Determine the depth drop gradient in the process of mobile device movement;Wherein, depth drop gradient describes the change gradient of depth data in the process of mobile device movement;Abnormality check is carried out to current frame depth data;When determining that current frame depth data is normal, current frame depth data is corrected according to depth drop gradient processing.The application technical scheme preserves the accuracy of depth data by data correction, thereby avoid or reduce the influence of depth data inaccuracy caused by depth data jump, and then the mobile device can be according to relatively accurate depth data Path planning is carried out in the process of back charging, effectively improve the back charging accuracy of mobile device.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of data processing, and particularly relates to a deep data processing method, a computer readable medium and an electronic device. BACKGROUND

[0002] The operation of a self-moving device usually relies on electric energy driving. When the electric quantity is insufficient, the self-moving device needs to return to a charging pile for charging. In the process of returning to the charging pile for charging, the self-moving device needs to measure the positional relationship between the self-moving device and the charging pile, so as to be able to accurately move to the charging pile for charging. At present, a commonly used method for determining the positional relationship between the self-moving device and the charging pile is to calculate the distance between the self-moving device and the charging pile through point cloud depth information, and then determine whether the self-moving device reaches the charging pile according to the distance. However, in the process of obtaining the point cloud depth information, some factors may cause the depth information to jump, such as unstable signal transmission, interference when the self-moving device moves, etc., which leads to inaccurate calculation of the distance between the self-moving device and the charging pile, and further leads to inaccurate return of the self-moving device to the charging pile.

[0003] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present application, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0004] The purpose of the present application is to provide a deep data processing method, a computer readable medium and an electronic device, so as to optimize the problem of inaccurate return of the self-moving device caused by depth information jumping in related technologies.

[0005] Other characteristics and advantages of the present application will become apparent from the following detailed description, or will be learned by practice of the present application.

[0006] According to an aspect of an embodiment of the present application, a deep data processing method is provided, comprising:

[0007] obtaining a current frame depth image in the moving direction of the self-moving device, and extracting current frame depth data from the current frame depth image, the current frame depth data representing the distance of the self-moving device from the charging pile at the current time;

[0008] determining a depth drop gradient in the moving process of the self-moving device; wherein the depth drop gradient is used to describe the change gradient of the depth data in the moving process of the self-moving device;

[0009] performing an abnormality check on the current frame depth data;

[0010] when it is determined that the current frame depth data is normal, performing correction processing on the current frame depth data according to the depth drop gradient.

[0011] According to one aspect of the embodiments of this application, a computer-readable medium is provided, on which a computer program is stored, which, when executed by a processor, implements the depth data processing method as described in the above technical solutions.

[0012] According to one aspect of the embodiments of this application, an electronic device is provided, the electronic device comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor executes the executable instructions to cause the electronic device to perform a depth data processing method as described in the above technical solutions.

[0013] According to one aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the depth data processing method as described in the above technical solutions.

[0014] In the technical solution provided in this application embodiment, by extracting the current frame depth data and the depth descent gradient during the movement of the mobile device from the current frame depth image, and performing anomaly checks on the current frame depth data, since the depth descent gradient during the movement of the mobile device reflects the distance change during the process of the mobile device approaching the charging pile, the current frame depth data can be corrected according to the depth descent gradient when the current frame depth data is normal. That is, by correcting the depth data through the depth descent gradient, the accuracy of the depth data from the mobile device to the charging pile is ensured, thereby avoiding or reducing the impact of inaccurate depth data caused by depth data jumps. As a result, the mobile device can plan the path for the recharging process based on more accurate depth data, effectively improving the recharging accuracy of the mobile device.

[0015] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0017] Figure 1 An exemplary system architecture block diagram illustrating the application of the technical solution of this application is shown schematically.

[0018] Figure 2 A schematic diagram of a self-moving device applying the technical solution of this application is shown.

[0019] Figure 3 A flowchart illustrating a method for processing depth data according to an embodiment of this application is shown schematically.

[0020] Figure 4 A flowchart illustrating a depth data processing method provided in another embodiment of this application is shown schematically.

[0021] Figure 5 A schematic block diagram of a depth data processing apparatus provided in one embodiment of this application is shown.

[0022] Figure 6 A schematic block diagram of a computer system architecture for implementing an electronic device according to embodiments of the present application is shown. Detailed Implementation

[0023] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0024] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or processes are not shown or described in detail to avoid obscuring aspects of this application.

[0025] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0026] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and processes / steps, nor do they necessarily have to be executed in the described order. For example, some processes / steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0027] Figure 1An exemplary system architecture block diagram illustrating the application of the technical solution of this application is shown schematically.

[0028] like Figure 1 As shown, the system architecture 100 may include terminal devices 110, a network 120, and a server 130. Terminal devices 110 may include smartphones, tablets, laptops, smart voice interaction devices, smart home appliances, in-vehicle terminals, self-moving devices, etc. Server 130 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Network 120 may be a communication medium of various connection types capable of providing a communication link between terminal devices 110 and server 130, such as a wired communication link or a wireless communication link.

[0029] Depending on the implementation requirements, the system architecture in this application embodiment can have any number of terminal devices, networks, and servers. For example, server 130 can be a server group composed of multiple server devices. In addition, the technical solutions provided in this application embodiment can be applied to terminal device 110, or to server 130, or can be implemented jointly by terminal device 110 and server 130. This application does not impose any special limitations on this.

[0030] For example, the technical solution of this application is implemented by a terminal device 110, which is a self-moving device, such as a mobile robot. The terminal device 110 acquires a current frame depth image along the movement direction of the self-moving device and extracts current frame depth data from the current frame depth image. The current frame depth data represents the distance of the self-moving device from the charging pile at the current moment. Then, the terminal device 110 determines the depth descent gradient during the movement of the self-moving device; wherein, the depth descent gradient describes the gradient of the change in depth data during the movement of the self-moving device. Next, the terminal device 110 performs anomaly checks on the current frame depth data, and when it is determined that the current frame depth data is normal, it corrects the current frame depth data according to the depth descent gradient.

[0031] For example, Figure 2 A schematic diagram of a self-moving device applying the technical solution of this application is shown. For example... Figure 2 As shown, the self-moving device 200 can also be the main body for implementing the technical solution of this application. The self-moving device 200 can be various mobile robots, such as sweeping robots, mopping robots, vegetable delivery robots, transport robots, lawn mowing robots, etc. The specific type and function of the self-moving device 200 are not limited in the embodiments of this application. It can be understood that the self-moving device 200 in this embodiment can also include other devices with self-moving functions.

[0032] In this embodiment, a camera device 210 is provided on the self-moving device 200 for capturing depth images along the movement direction of the self-moving device 200. The camera device 210 can be fixed or non-fixed and rotatable; this embodiment does not limit this.

[0033] The camera device 210 is connected to the control module 220 inside the self-moving device 200. The control module 220 is used to receive control commands from the self-moving device 200 or to generate various control commands for the self-moving device 200. The control module 220 is also connected to the drive components of the self-moving device 200, such as the steering shaft, steering wheels, motor, etc., of the self-moving device 200, and is used to control the movement and steering of the self-moving device 200.

[0034] In this embodiment, the camera device 210 captures a depth image along the movement direction of the mobile device 200. The current frame depth image is the depth image captured by the camera device 210 at the current moment. The camera device 210 transmits the current frame depth image to the control module 220. After receiving the current frame depth image, the control module 220 performs depth data extraction, depth descent gradient determination, and anomaly detection on the current frame depth image according to the depth data processing method provided in this embodiment, and then corrects the current frame depth data. The control module 220 can locate the mobile device 200 and plan the path during the recharging process based on the corrected depth data, so that the mobile device 200 can accurately reach the recharging point, i.e., the location of the charging pile.

[0035] For example, in one application scenario of the technical solution of this application, the self-moving device 200 is a lawnmower robot. When the lawnmower robot finishes its work in a certain work area or its battery is low, it needs to move to a charging device to recharge; this process is referred to as the lawnmower robot's recharging. The lawnmower robot corrects its depth data according to the depth data processing method provided in the embodiments of this application, and then accurately recharges based on the corrected depth data.

[0036] The following describes in detail the depth data processing method provided in this application, with reference to specific implementation methods.

[0037] Figure 3 A flowchart illustrating a depth data processing method according to an embodiment of this application is shown. This method can be implemented by a depth data processing apparatus provided in this application, which can be configured on a terminal device or server, or on a self-moving device. Figure 1 The terminal device 110 or server 130 shown below will be used as the execution subject to describe the specific implementation process of the depth data processing method.

[0038] likeFigure 3 As shown, the depth data processing method provided in this application embodiment includes steps 310 to 340, specifically including:

[0039] Step 310: Obtain the current frame depth image in the direction of movement of the mobile device, and extract the current frame depth data from the current frame depth image. The current frame depth data represents the distance between the mobile device and the charging pile at the current moment.

[0040] Specifically, a depth image, also known as a distance image, is an image that uses the distance from the image acquisition device to various points in the image acquisition environment as pixel values. The distance from the image acquisition device to various points in the image acquisition environment is what is referred to as depth. During the recharging process, the mobile device moves towards the charging station; therefore, the depth data in the depth image along the direction of movement of the mobile device represents the distance of the mobile device from the charging station at the current moment. The current frame depth image represents the depth image acquired at the current moment, and the depth data it includes is denoted as the current frame depth data.

[0041] Step 320: Determine the depth descent gradient during the movement of the self-moving device; wherein, the depth descent gradient is used to describe the gradient of the change in depth data during the movement of the self-moving device.

[0042] Specifically, the depth descent gradient is used to describe the changes in depth data during the movement of the mobile device. The depth descent gradient is positively correlated with the amount of change in depth data; that is, the larger the depth descent gradient, the greater the change in depth data, and vice versa. The depth descent gradient during the movement of the mobile device can be obtained by subtracting the depth data from the depth data in adjacent depth images. For example, the depth descent gradient at the current moment can be obtained by subtracting the depth data in the previous depth image from the depth data in the current frame.

[0043] In one embodiment of this application, the depth descent gradient can be calculated based on the depth data of multiple frames of depth images. The specific process includes: acquiring depth data of a preset number of frames during the movement of the mobile device, wherein the depth data of the preset number of frames includes the depth data of the current frame; calculating the depth difference between each two adjacent frames of depth data in the preset number of frames of depth data; and averaging the depth differences to obtain the depth descent gradient.

[0044] Specifically, during the movement of the self-moving device, it can acquire depth images along its direction of movement at a certain frequency. As the device moves, the number of acquired depth images increases. When calculating the depth descent gradient, the nearest depth image (within a preset number of frames) to the current moment can be extracted from the acquired depth images. This extracted image includes the current frame's depth image. Depth data is then extracted for each extracted frame to obtain the preset number of depth data frames. Next, the depth difference between any two adjacent frames is calculated, and then the average or weighted sum of these differences is obtained to obtain the depth descent gradient. Calculating the depth descent gradient using depth data from multiple frames takes into account the influence of historical depth data on the current depth data, ensuring a smooth transition between the calculated gradients and preventing abrupt changes in the gradient.

[0045] In one embodiment of this application, when performing weighted summation, the weight of the depth difference can be set according to the distance between the acquisition time of the depth data corresponding to the depth difference and the current time. For example, the depth difference corresponding to the depth data closer to the current time is given a larger weight, and the depth difference corresponding to the depth data farther from the current time is given a smaller weight. This is because the depth data closer to the current time often has a greater impact on the depth descent extraction at the current time. By setting different weights according to time-distance, the influence of depth descent gradients at different historical times on the depth descent gradient at the current time can be considered simultaneously while taking into account smoothness, thereby improving the accuracy of depth descent gradient calculation.

[0046] Step 330: Perform anomaly checks on the current frame depth data.

[0047] Specifically, anomaly checking refers to detecting whether the current frame depth data is abnormal, such as whether there are abrupt changes or incorrect depth measurements.

[0048] In one embodiment of this application, the specific process of anomaly checking includes: determining the current depth range of the self-moving device based on the current location information of the self-moving device; determining that the current frame depth data is abnormal when the current frame depth data is not within the depth range; and determining that the current frame depth data is normal when the current frame depth data is within the depth range.

[0049] Specifically, the location information of the charging station is generally known in advance. Combined with the current location information of the mobile device, the distance between the mobile device and the charging station can be determined. This distance serves as the reference for the current frame depth data, representing the depth interval within which the current frame depth data falls. This depth interval can be defined with the distance as the midpoint and the interval width as the upper and lower tolerance values. That is, the generated depth interval is [distance - interval width, distance + interval width]. For example, if the current distance between the mobile device and the charging station is 100cm and the interval width is set to 20cm, then the depth interval could be [90, 110]. If the current frame depth data is not within the depth interval represented by this distance, it is determined that the current frame depth data is abnormal; if the current frame depth data is within the depth interval represented by this distance, it is determined that the current frame depth data is normal. For example, if the depth interval is [90, 110], and the current frame depth data is 80cm, then the current frame depth data is determined to be abnormal; if the current frame depth data is 105cm, then the current frame depth data is determined to be normal.

[0050] In one embodiment of this application, the depth range can also be determined based on the current location information of the mobile device and preset location points on the mobile device's recharge path. Specifically, multiple location points are preset on the mobile device's recharge path, and the distance of each location point from the charging station is measured in advance. The location points are arranged in descending (or ascending) order of distance from the charging station, so that two adjacent location points can constitute a depth range. During anomaly checks, the two closest and adjacent target location points to the mobile device are first determined based on the mobile device's current location information, and then the depth range is determined based on the distance from the target location points to the charging station. For example, positioning points are pre-set at locations such as 10cm, 40cm, 80cm, and 150cm away from the charging station. When the mobile device is currently between positioning point 10cm and positioning point 40cm, the depth range is [10, 40]. If the current frame depth data is less than 10cm or greater than 40cm, it indicates that the depth data is incorrect, and the current frame depth data is confirmed to be abnormal. If the current frame depth data is between [10, 40], it confirms that the current frame depth data is normal.

[0051] In one embodiment of this application, the anomaly detection process may further include: determining the magnitude of the depth descent gradient and a second preset threshold; when the depth descent gradient is greater than the second preset threshold, determining that the current frame depth data is abnormal; when the depth descent gradient is less than the second preset threshold, determining that the current frame depth data is normal.

[0052] Specifically, the second preset threshold is determined based on the changes in depth data during the movement of the mobile device. For example, if the depth data changes significantly, the second preset threshold is set to a larger value; if the depth data changes slightly, the second preset threshold is set to a smaller value. When the depth descent gradient is greater than the second preset threshold, it indicates that the depth data change has exceeded the normal range, thus the current frame's depth data is determined to be abnormal. When the depth descent gradient is less than the second preset threshold, it indicates that the depth data change is within the normal range, thus the current frame's depth data is determined to be normal. For example, if the second preset threshold is set to 0, when the depth descent gradient is less than 0, it indicates that the distance between the mobile device and the charging station has decreased, which is a normal phenomenon during the mobile device's recharging process, and the current frame's depth data can be considered normal. When the depth descent gradient is greater than 0, it indicates that the distance between the mobile device and the charging station has increased, which is an abnormal phenomenon during the mobile device's recharging process, and the current frame's depth data can be considered abnormal.

[0053] In one embodiment of this application, anomaly checks can also be performed based on the changes in the moving distance and depth data of the self-moving device. Specifically, the difference between the depth data of the previous frame and the moving distance of the self-moving device is calculated. The moving distance of the self-moving device refers to the distance moved by the self-moving device from the moment the previous frame data is acquired to the moment the current frame data is acquired. Then, this difference is compared with the depth data of the current frame. If the difference is greater than a third preset threshold, the depth data of the current frame is considered abnormal; if the difference is less than the third preset threshold, the depth data of the current frame is considered normal. For example, if the depth data of the previous frame is 100cm, the moving distance of the self-moving device is 20cm, and the third preset threshold is set to 5cm, the difference between the depth data of the previous frame and the moving distance of the self-moving device is 80cm. If the depth data of the current frame is 74cm, the difference between the difference and the current frame depth data is less than the third preset threshold, and the current frame depth data is normal; if the depth data of the current frame is 76cm, the difference between the difference and the current frame depth data is greater than the third preset threshold, and the current frame depth data is abnormal.

[0054] It should be noted that this application involves comparison or judgment steps, and the "equal to" cases in the comparison or judgment results can be categorized into "greater than" or "less than" cases according to actual needs. For example, when judging the magnitude of the depth descent gradient and the second preset threshold, the case where the depth descent gradient is equal to the second preset threshold can be categorized into the case where the depth descent gradient is greater than the second preset threshold or the case where the depth descent gradient is less than the second preset threshold.

[0055] Step 340: When it is determined that the current frame depth data is normal, the current frame depth data is corrected according to the depth descent gradient.

[0056] Specifically, when the depth data of the current frame is determined to be normal, in order to further improve the accuracy of the depth data, the depth data of the current frame is corrected based on the depth descent gradient. For example, the depth descent gradient is multiplied by a coefficient to obtain the correction amount, and then the correction amount is subtracted from the depth data of the current frame to obtain the corrected depth data of the current frame.

[0057] In the technical solution provided in this application embodiment, by extracting the current frame depth data and the depth descent gradient during the movement of the mobile device from the current frame depth image, and performing anomaly checks on the current frame depth data, since the depth descent gradient during the movement of the mobile device reflects the distance change during the process of the mobile device approaching the charging pile, the current frame depth data can be corrected according to the depth descent gradient when the current frame depth data is normal. That is, by correcting the depth data through the depth descent gradient, the accuracy of the depth data from the mobile device to the charging pile is ensured, thereby avoiding or reducing the impact of inaccurate depth data caused by depth data jumps. As a result, the mobile device can plan the path for the recharging process based on more accurate depth data, effectively improving the recharging accuracy of the mobile device.

[0058] Figure 4 A flowchart illustrating a depth data processing method according to an embodiment of this application is shown schematically. This embodiment is a further optimization of the above-described embodiment. Figure 4 As shown, the depth data processing method provided in this embodiment includes:

[0059] Step 401: Obtain the current frame depth image in the direction of movement of the mobile device, and extract the current frame depth data from the current frame depth image. The current frame depth data represents the distance between the mobile device and the charging pile at the current moment.

[0060] Step 402: Determine the depth descent gradient during the movement of the self-moving device; wherein, the depth descent gradient is used to describe the gradient of the change in depth data during the movement of the self-moving device.

[0061] Step 403: Perform anomaly checks on the current frame depth data. If the current frame depth data is determined to be normal, proceed to steps 404-406; if the current frame depth data is determined to be abnormal, proceed to steps 407-409.

[0062] Steps 401-403 are the same as steps 310-330 in the previous embodiments, and will not be repeated here.

[0063] The depth descent gradient can be calculated based on the depth data of a preset number of frames, as shown in equation (1) below:

[0064]

[0065] Where grad represents the depth descent gradient, d i This represents the depth data of the i-th frame (i.e., the historical depth data), and n represents the preset number of frames. The value of n should not be too large or too small. When the value of n is large, it is easy to accumulate errors from a large number of frames of depth data; when the value of n is too small, the depth data involved in the calculation is less, and the smoothness between the depth descent gradients at different times becomes worse. For example, the value of n can be between 10 and 50.

[0066] Step 404: Calculate the absolute value of the difference between the depth descent gradient and the current moving speed of the self-moving device.

[0067] Step 405: When the absolute value is greater than the first preset threshold, the depth data of the current frame is used as the corrected depth data of the current frame.

[0068] Step 406: When the absolute value is less than the first preset threshold, the difference between the current frame depth data and the absolute value is used as the corrected depth data of the current frame.

[0069] Steps 404-406 are the specific processes for correcting the depth data of the current frame when it is determined that the depth data of the current frame is normal. The depth descent gradient and the current moving speed of the self-moving device are compared. If the time for collecting depth data of a preset number of frames is taken as several units of time, and as can be seen from formula (1), the depth descent gradient is obtained by averaging the difference between the depth data of the preset number of frames, then the depth descent gradient can be expressed as the depth change distance per unit time, and the current moving speed of the self-moving device can be expressed as the moving distance of the self-moving device per unit time. Therefore, the depth descent gradient and the current moving speed can be compared as having the same dimension. When the difference between the two is large, that is, the absolute value of the difference between the two is greater than the first preset threshold, then the depth data of the current frame is taken as the depth data after correction of the current frame. When the difference between the two is small, that is, the absolute value of the difference between the two is less than the first preset threshold, then the difference between the depth data of the current frame and the absolute value is taken as the depth data after correction of the current frame. For details, please refer to the following formulas (2)-(3):

[0070] e = |V - grad| (2)

[0071]

[0072] Where V represents the current movement speed of the self-moving device, e represents the absolute value of the difference between the depth descent gradient and the current movement speed; d cur Indicates the depth data of the current frame; d prime This represents the corrected depth data for the current frame; tol represents the first preset threshold.

[0073] In one embodiment of this application, the first preset threshold tol can be determined based on the variance of the depth data in the depth data pool. The depth data pool is a data pool used to store a preset number of depth data points; generally, the depth data pool stores the preset number of depth data points most recent to the current time. The variance of the depth data in the depth data pool is calculated, and this variance is multiplied by a preset coefficient to obtain the first preset threshold tol, i.e., tol = var * 0.01, where var represents the variance of the depth data in the depth data pool.

[0074] Step 407: Obtain multiple historical depth data from the depth data pool based on the current moving speed of the self-moving device. The depth data pool is used to store a preset number of depth data.

[0075] Step 408: Calculate the depth mean of multiple historical depth data.

[0076] Step 409: Update the depth data of the current frame based on the depth mean.

[0077] Steps 407-409 describe the specific process of updating the current frame depth data when an anomaly is determined. Historical depth data refers to the depth data stored in the depth data pool. When acquiring historical depth data, the required number of depth data frames is first calculated based on the current moving speed of the mobile device and the depth image period. The depth image period is the period during which the mobile device acquires depth images, and it is the derivative of the frequency at which the mobile device acquires depth images. In this embodiment, the product of the current moving speed and the depth image period is used as the number of depth data frames. Then, the depth data closest to the current frame, in a number equal to the number of depth data frames, is retrieved from the depth data pool as multiple historical depth data sets. After obtaining multiple historical depth data sets, the average of these sets is used as the current frame depth data, thus updating the current frame depth data.

[0078] For example, the update process of the current frame depth data can be referred to equation (4).

[0079]

[0080] Where, d cur This represents the updated current frame depth data, d. i This represents the historical depth data of frame i; m represents the number of depth data frames; V represents the current moving speed of the mobile device; and T represents the depth image period.

[0081] According to equation (4), updating the current frame depth data based on historical depth data is equivalent to performing smoothing filtering on the current frame depth data by obtaining multiple historical depth data based on the current moving speed of the mobile device. In one embodiment of this application, before correcting the current frame depth data, smoothing filtering can also be performed on the current frame depth data using equation (4), and then the smoothed current frame depth data can be corrected. That is, the current frame depth data d in equation (3) cur It can be calculated according to equation (4).

[0082] In one embodiment of this application, after updating the current frame depth data, a correction process can be performed according to equations (2)-(3) to further ensure the accuracy of the depth data.

[0083] In one embodiment of this application, after correcting or updating the current frame depth data, the corrected or updated current frame depth data can be added to the depth data pool. When adding the current frame depth data to the depth data pool, it is first determined whether the amount of data in the depth data pool has reached its maximum value. If it has not reached the maximum value, the current frame depth data can be added directly to the depth data pool. If it has reached the maximum value, at least one frame of historical depth data that is far removed from the current time can be deleted from the depth data pool before adding the current frame depth data to the depth data pool.

[0084] In the technical solution provided in this application embodiment, by performing anomaly checks on the current frame depth data in the current frame depth image, normal and abnormal data in the depth data can be effectively identified. Different processing methods can then be applied to normal and abnormal data respectively to ensure the accuracy of the depth data. When the current frame depth data is normal, it is corrected based on the depth descent gradient, thus ensuring the accuracy of the depth data through the dual operations of anomaly checks and data correction. Simultaneously, when the current frame depth data is abnormal, it is updated based on historical depth data, thereby removing abnormal data from the depth data and performing frame interpolation using historical depth data. This ensures the integrity of the depth data and avoids the self-device from performing path planning during the recharging process based on abnormal data, reducing the impact of depth data jumps on recharging and thus improving the recharging accuracy of the self-device.

[0085] It should be noted that although the steps in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0086] The following describes embodiments of this application, which can be used to execute the depth data processing method in the above embodiments of this application. Figure 5 A schematic block diagram of a depth data processing apparatus according to an embodiment of this application is shown. Figure 5 As shown, the depth data processing apparatus provided in this application embodiment includes:

[0087] The depth data acquisition module 510 is used to acquire the current frame depth image in the direction of movement of the mobile device, and extract the current frame depth data from the current frame depth image. The current frame depth data represents the distance between the mobile device and the charging pile at the current moment.

[0088] The depth descent gradient determination module 520 is used to determine the depth descent gradient during the movement of the self-moving device; wherein, the depth descent gradient is used to describe the gradient of the change in depth data during the movement of the self-moving device.

[0089] The anomaly detection module 530 is used to perform anomaly detection on the current frame depth data;

[0090] The data correction module 540 is used to correct the current frame depth data according to the depth descent gradient when it is determined that the current frame depth data is normal.

[0091] In one embodiment of this application, the apparatus further includes,

[0092] The data update module is used to: obtain multiple historical depth data from the depth data pool based on the current moving speed of the self-moving device, the depth data pool being used to store a preset number of depth data; calculate the depth average of the multiple historical depth data; and update the current frame depth data based on the depth average.

[0093] In one embodiment of this application, the data update module is specifically used for:

[0094] The number of depth data frames required is calculated based on the current moving speed of the self-moving device and the depth image period, which is the period during which the self-moving device acquires depth images;

[0095] Historical depth data is obtained from the depth data pool, which contains depth data frames that are closest to the current frame.

[0096] In one embodiment of this application, the data correction module 540 is specifically used for:

[0097] Calculate the absolute value of the difference between the depth descent gradient and the current moving speed of the self-moving device;

[0098] When the absolute value is greater than the first preset threshold, the depth data of the current frame is used as the corrected depth data of the current frame;

[0099] When the absolute value is less than the first preset threshold, the difference between the current frame depth data and the absolute value is used as the corrected depth data of the current frame.

[0100] In one embodiment of this application, the data correction module 540 is further configured to:

[0101] The first preset threshold is determined based on the variance of the depth data in the depth data pool.

[0102] In one embodiment of this application, the anomaly checking module 530 is specifically used for:

[0103] Determine the current depth range of the self-moving device based on its current location information;

[0104] When the current frame depth data is not within the depth range, the current frame depth data is determined to be abnormal.

[0105] When the current frame depth data is within the depth range, it is determined that the current frame depth data is normal.

[0106] In one embodiment of this application, the depth descent gradient determination module 520 is specifically used for:

[0107] Acquire depth data for a preset number of frames during the movement of the mobile device, wherein the depth data for the preset number of frames includes the depth data of the current frame;

[0108] Calculate the depth difference between every two adjacent frames of depth data within a preset number of frames;

[0109] The depth descent gradient is obtained by averaging the depth differences.

[0110] In one embodiment of this application, the anomaly checking module 530 is specifically used for:

[0111] When the depth descent gradient is greater than the second preset threshold, the depth data of the current frame is determined to be abnormal; wherein, the second preset threshold is determined based on the changes in depth data during the movement of the mobile device;

[0112] When the depth descent gradient is less than the second preset threshold, the depth data of the current frame is determined to be normal.

[0113] The specific details of the depth data processing apparatus provided in the various embodiments of this application have been described in detail in the embodiments of the corresponding methods, and will not be repeated here.

[0114] Figure 6 A schematic block diagram of a computer system architecture for implementing an electronic device according to embodiments of the present application is shown.

[0115] It should be noted that, Figure 6 The computer system 600 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0116] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 602 or programs loaded from storage section 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for system processing. The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output interface 605 (I / O interface) is also connected to the bus 604.

[0117] The following components are connected to the input / output interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a local area network card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0118] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit 601, it performs various functions defined in the system of this application.

[0119] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0120] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and processing of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or processing, or using a combination of dedicated hardware and computer instructions.

[0121] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0122] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0123] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0124] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for processing depth data, characterized in that, include: Acquire the current frame depth image in the direction of movement of the mobile device, and extract the current frame depth data from the current frame depth image. The current frame depth data represents the distance of the mobile device from the charging pile at the current moment. Determine the depth descent gradient during the movement of the self-moving device; wherein the depth descent gradient is used to describe the gradient of the change in depth data during the movement of the self-moving device; Perform anomaly checks on the current frame depth data; When it is determined that the current frame depth data is normal, calculate the absolute value of the difference between the depth descent gradient and the current moving speed of the self-moving device; When the absolute value is greater than the first preset threshold, the current frame depth data is used as the current frame corrected depth data; When the absolute value is less than the first preset threshold, the difference between the current frame depth data and the absolute value is used as the corrected depth data of the current frame.

2. The method for processing depth data according to claim 1, characterized in that, When it is determined that the current frame depth data is abnormal, the method further includes: Based on the current moving speed of the self-moving device, multiple historical depth data are obtained from the depth data pool, which is used to store a preset number of depth data. Calculate the depth mean of multiple historical depth data sets; The depth data of the current frame is updated based on the mean depth.

3. The method for processing depth data according to claim 2, characterized in that, The step of obtaining multiple historical depth data from the depth data pool based on the current moving speed of the self-moving device includes: The required number of depth data frames is calculated based on the current moving speed of the self-moving device and the depth image period, where the depth image period is the period during which the self-moving device acquires depth images; The multiple historical depth data are obtained from the depth data pool, the number of which is the number of depth data frames closest to the current frame.

4. The method for processing depth data according to claim 1, characterized in that, The method further includes: The first preset threshold is determined based on the variance of the depth data in the depth data pool.

5. The method for processing depth data according to claim 1, characterized in that, The anomaly check on the current frame depth data includes: The depth range where the self-moving device is currently located is determined based on the current location information of the self-moving device; When the current frame depth data is not within the depth range, it is determined that the current frame depth data is abnormal; When the current frame depth data is within the depth range, it is determined that the current frame depth data is normal.

6. The method for processing depth data according to claim 1, characterized in that, Determining the depth descent gradient during the movement of the self-moving device includes: Acquire depth data for a preset number of frames during the movement of the self-moving device, wherein the depth data for the preset number of frames includes the depth data of the current frame; In the depth data of a preset number of frames, calculate the depth difference between each two adjacent frames of the depth data; The depth descent gradient is obtained by averaging the depth differences.

7. The method for processing depth data according to claim 6, characterized in that, The anomaly check on the current frame depth data includes: When the depth descent gradient is greater than a second preset threshold, the current frame depth data is determined to be abnormal; wherein, the second preset threshold is determined based on the depth data changes during the movement of the mobile device; When the depth descent gradient is less than the second preset threshold, the current frame depth data is determined to be normal.

8. A computer-readable medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method for processing depth data as described in any one of claims 1 to 7.

9. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor executes the executable instructions to cause the electronic device to perform the depth data processing method according to any one of claims 1 to 7.

Images