Method, device and equipment for detecting movement state of dynamic obstacle and storage medium

Abstract

The application is suitable for the technical field of computer vision, and provides a dynamic obstacle motion state detection method, device and equipment and a storage medium, the method comprising: acquiring an environment image pair at a first time and an environment image pair at a second time in a camera coordinate system; determining first point cloud data in a world coordinate system according to the environment image pair at the first time, and determining second point cloud data in the world coordinate system according to the environment image pair at the second time; determining a mass center position of a target dynamic obstacle at the first time according to the first point cloud data, determining a mass center position of the target dynamic obstacle at the second time according to the second point cloud data, and determining a motion state of the target dynamic obstacle according to the mass center position of the target dynamic obstacle at the first time and the mass center position of the target dynamic obstacle at the second time. Thus, the motion state of the target dynamic obstacle can be accurately determined, so that the mobile device can timely and accurately avoid the target dynamic obstacle.

Description

Technical Field

[0001] This application belongs to the field of computer vision technology, and in particular relates to a method, device, equipment and storage medium for detecting the motion state of dynamic obstacles. Background Technology

[0002] Self-moving devices (such as robots, autonomous vehicles, and drones) may encounter dynamic obstacles (such as pedestrians, vehicles, and animals) during their movement. Self-moving devices need to detect dynamic obstacles and determine their positions so that they can avoid them.

[0003] Currently, self-moving devices detect dynamic obstacles by: acquiring an image containing dynamic obstacles; processing and analyzing the image to determine the location of the dynamic obstacles in the image.

[0004] However, the aforementioned detection methods can only determine the position of dynamic obstacles, not their motion state. During the process of the self-moving device avoiding the dynamic obstacle based on its position, the motion state of the obstacle may change, potentially leading to a collision between the self-moving device and the obstacle with its altered motion state, thus increasing the risk of such a collision. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for detecting the motion state of dynamic obstacles, which can prevent collisions between the self-moving device and dynamic obstacles after the motion state has changed, thereby reducing the risk of collisions between the self-moving device and dynamic obstacles.

[0006] In a first aspect, embodiments of this application provide a method for detecting the motion state of a dynamic obstacle, the method comprising:

[0007] Acquire environmental image pairs at the first and second moments in the camera coordinate system. Each environmental image pair includes a first image and a second image. The first and second images are environmental images acquired from different perspectives at the same moment. The second moment is a time point after the first moment.

[0008] Based on the environmental image pair at the first moment in the camera coordinate system, determine the first point cloud data of the target dynamic obstacle at the first moment in the world coordinate system, and based on the environmental image pair at the second moment in the camera coordinate system, determine the second point cloud data of the target dynamic obstacle at the second moment in the world coordinate system.

[0009] Based on the first point cloud data, determine the centroid position of the target dynamic obstacle at the first moment, and based on the second point cloud data, determine the centroid position of the target dynamic obstacle at the second moment.

[0010] Based on the centroid position of the target dynamic obstacle at the first moment and the centroid position of the target dynamic obstacle at the second moment, the motion state of the target dynamic obstacle is determined. The motion state includes at least one parameter among the target dynamic obstacle's moving distance, moving speed, and moving direction.

[0011] In one possible implementation, determining first point cloud data of the target dynamic obstacle in the world coordinate system at a first moment based on environmental image pairs in the camera coordinate system, and determining second point cloud data of the target dynamic obstacle in the world coordinate system at a second moment based on environmental image pairs in the camera coordinate system, includes:

[0012] Based on the environmental image pair at the first moment in the camera coordinate system, a first disparity map at the first moment in the camera coordinate system is determined, and based on the environmental image pair at the second moment in the camera coordinate system, a second disparity map at the second moment in the camera coordinate system is determined. Both the first disparity map and the second disparity map include the disparity values ​​of all pixels of the target dynamic obstacle at the corresponding moment.

[0013] Based on the first disparity map at the first moment in the camera coordinate system, a first depth map at the first moment in the camera coordinate system is determined, and based on the second disparity map at the second moment in the camera coordinate system, a second depth map at the second moment in the camera coordinate system is determined. Both the first depth map and the second depth map include the distance values ​​of all pixels of the target dynamic obstacle at the corresponding moment.

[0014] Based on the first depth map at the first moment in the camera coordinate system, determine the first point cloud data of the target dynamic obstacle at the first moment in the world coordinate system, and based on the second depth map at the second moment in the camera coordinate system, determine the second point cloud data of the target dynamic obstacle at the second moment in the world coordinate system.

[0015] In one possible implementation, determining the first disparity map in the camera coordinate system based on the environmental image pair at the first moment in the camera coordinate system includes:

[0016] Obtain the environmental image pair in the camera coordinate system at the first moment;

[0017] The first image of the first moment of the environment image pair is input into the semantic segmentation network to obtain the position of the target dynamic obstacle in the first image;

[0018] The location of the target dynamic obstacle in the second image of the environmental image pair at the first moment is determined based on the environmental matching algorithm;

[0019] Based on the position of the target dynamic obstacle in the first image at the first moment and the position of the target dynamic obstacle in the second image at the first moment, determine the disparity value of all pixels of the target dynamic obstacle at the first moment;

[0020] Based on the disparity values ​​of all pixels of the target dynamic obstacle, determine the first disparity map in the camera coordinate system at the first moment.

[0021] In one possible implementation, based on the first depth map at the first moment in the camera coordinate system, the first point cloud data of the target dynamic obstacle at the first moment in the world coordinate system is determined, including:

[0022] Based on the first depth map at the first moment in the camera coordinate system, determine the first point cloud data of the target dynamic obstacle at the first moment in the camera coordinate system;

[0023] Obtain the mapping relationship between the camera coordinate system and the world coordinate system;

[0024] Based on the point cloud data of the target dynamic obstacle at the first moment in the camera coordinate system, and the mapping relationship, determine the first point cloud data of the target dynamic obstacle at the first moment in the world coordinate system.

[0025] In one possible implementation, the motion state of the target dynamic obstacle is determined based on the centroid position of the target dynamic obstacle at a first moment and the centroid position of the target dynamic obstacle at a second moment, including:

[0026] Based on the centroid position of the target dynamic obstacle at the first moment and the centroid position of the target dynamic obstacle at the second moment, determine the moving distance and direction of the target dynamic obstacle;

[0027] The speed of the target dynamic obstacle is determined based on the moving distance of the target dynamic obstacle and the preset duration; the preset duration is the difference between the second moment and the first moment.

[0028] In one possible implementation, when multiple dynamic obstacles exist, the first point cloud data consists of multiple sets, and the second point cloud data consists of multiple sets; different point cloud data correspond to different dynamic obstacles; the motion state detection method further includes:

[0029] Each second point cloud data is matched with each first point cloud data in turn to obtain multiple corresponding point cloud data pairs; the point cloud data pairs include the first point cloud data at the first moment and the second point cloud data at the second moment, which represent the same target dynamic obstacle.

[0030] In one possible implementation, when multiple dynamic obstacles exist, the first point cloud data consists of multiple sets, and the second point cloud data consists of multiple sets; different point cloud data correspond to different dynamic obstacles; the motion state detection method further includes:

[0031] Obtain the centroid position of each first point cloud data and the centroid position of each second point cloud data;

[0032] The centroid position of the first point cloud data and the centroid position of the second point cloud data that are closest to each other are determined as the centroid position of the same target dynamic obstacle.

[0033] Secondly, embodiments of this application provide a motion state detection device for dynamic obstacles, which is used to execute the motion state detection method in the first aspect or any possible implementation thereof. Specifically, the motion state detection device may include a module for executing the motion state detection method in the first aspect or any possible implementation thereof.

[0034] Thirdly, embodiments of this application provide a self-moving device, which includes a memory and a processor. The memory stores instructions; the processor executes the instructions stored in the memory, causing the self-moving device to perform the motion state detection method in the first aspect or any possible implementation thereof.

[0035] Fourthly, a readable storage medium is provided, which stores instructions that, when executed on a computer, cause the computer to perform the motion state detection method of the first aspect or any possible implementation thereof.

[0036] Fifthly, a computer program product containing instructions is provided, which, when executed on a self-moving device, causes the self-moving device to perform the motion state detection method in the first aspect or any possible implementation thereof.

[0037] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0038] This application acquires environmental image pairs at different times in the camera coordinate system, and further determines point cloud data at different times in the camera coordinate system based on these image pairs. Furthermore, this application can convert the point cloud data at different times in the camera coordinate system into point cloud data at different times in the world coordinate system, thus converting the relative motion of the target dynamic obstacle into absolute motion. Based on this, this application can determine the centroid position of the target dynamic obstacle at different times based on the point cloud data in the world coordinate system, and obtain the motion state of the target dynamic obstacle based on its centroid position. Combining this with the motion state of the target dynamic obstacle, accurate avoidance of the target dynamic obstacle can be achieved, preventing collisions between the mobile device and the target dynamic obstacle, thereby effectively reducing the risk of collisions between the mobile device and the target dynamic obstacle. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating the first method for detecting the motion state of a dynamic obstacle provided in this application embodiment;

[0041] Figure 2 This is a schematic diagram of the matching results of the centroid position of the target dynamic obstacle provided in the embodiments of this application;

[0042] Figure 3 This is a flowchart illustrating the second method for detecting the motion state of dynamic obstacles provided in this application embodiment;

[0043] Figure 4 This is a flowchart illustrating the third method for detecting the motion state of dynamic obstacles provided in this application embodiment;

[0044] Figure 5 This is a flowchart illustrating the fourth method for detecting the motion state of dynamic obstacles provided in this application embodiment;

[0045] Figure 6 This is a schematic block diagram of a motion state detection device for dynamic obstacles provided in an embodiment of this application. Detailed Implementation

[0046] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0047] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0048] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0049] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0050] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0051] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0052] When working in outdoor environments, self-moving devices need to identify obstacles in the environment, such as dynamic and static obstacles, and avoid the identified obstacles.

[0053] Dynamic obstacles are movable objects that can affect the movement path of a self-moving device. A dynamic obstacle may be one or more. Dynamic obstacles may include: parked vehicles, moving vehicles, walking pedestrians, standing pedestrians, and animals, etc.

[0054] For example, taking a self-moving device as a lawnmower robot, and the obstacle as a dynamic obstacle, when the lawnmower robot is working in an outdoor scene, it can collect environmental images, i.e., RGB images. The lawnmower robot can identify the RGB images and determine whether there are dynamic obstacles in them. When dynamic obstacles are present in the RGB images, the lawnmower robot can determine the position of the dynamic obstacles in the RGB images. Based on the position of the dynamic obstacles, the lawnmower robot can plan an avoidance path, i.e., the movement path of the lawnmower robot.

[0055] Suppose that after the lawnmower's path is determined, the path of a dynamic obstacle changes. At this point, the lawnmower's path and the obstacle's path may conflict, causing a collision between the lawnmower and the obstacle.

[0056] In summary, relying solely on the position of dynamic obstacles cannot accurately plan the lawnmower's movement path, making collisions with these obstacles unavoidable. This results in a relatively high risk of collisions between the lawnmower and dynamic obstacles.

[0057] To address the aforementioned issues, this application provides a method for detecting the motion state of dynamic obstacles. By analyzing and processing acquired environmental images, the centroid position of the dynamic obstacle can be determined, and the motion state of the dynamic obstacle can be determined based on its centroid position. This allows for accurate planning of the lawnmower robot's motion path, reducing the risk of collision between the lawnmower robot and the dynamic obstacle.

[0058] Please see Figure 1 , Figure 1 A flowchart illustrating the motion state detection method for a first dynamic obstacle provided in this application is shown. Figure 1 As shown, the motion state detection method for dynamic obstacles in this application embodiment may include:

[0059] S101. Obtain the environmental image pair at the first moment and the environmental image pair at the second moment in the camera coordinate system.

[0060] The camera coordinate system is a coordinate system centered on the camera, describing the position of dynamic obstacles from the camera's perspective. Typically, cameras have a high frame rate. The camera's frame rate indicates how many frames of images the camera can capture per second. For example, a frame rate of 30 FPS means the camera can capture 30 frames per second.

[0061] The first moment can be any moment when the camera begins acquiring the environmental image pair. For example, the first moment can be the moment when the camera begins acquiring the environmental image pair, or it can be any moment after the camera begins acquiring the environmental image pair.

[0062] Processing the environmental image pair allows us to determine the number and location of dynamic obstacles. The environmental image pair includes a first image and a second image. The first and second images are environmental images acquired at the same time from different viewpoints. Both the first and second images are two-dimensional images.

[0063] The second moment is a point in time after the first moment. If the first moment is the moment when the camera begins acquiring environmental image pairs, then the second moment is a moment after the camera begins acquiring environmental image pairs. Alternatively, if the first moment is any moment after the camera begins acquiring environmental image pairs, then the second moment is a moment at a preset time interval from any moment after the camera begins acquiring images.

[0064] The preset duration is the difference between the second and first moments. The preset duration can be stored in advance on the self-moving device. For example, the preset duration is 33ms.

[0065] In some embodiments, the self-moving device can acquire environmental image pairs at a first moment using a camera installed in the self-moving device. The self-moving device can also acquire environmental image pairs at a second moment using the camera. In this case, both the environmental image pairs acquired by the self-moving device at the first moment and the environmental image pairs at the second moment are image pairs in the camera coordinate system.

[0066] For example, the camera is a stereo camera. A stereo camera can be used to acquire pairs of environmental images. In this case, the environmental image pair includes two images, namely, a first image and a second image. In other words, the first image can be either the left or right image acquired by the stereo camera, and the second image can also be either the left or right image acquired by the stereo camera. When the first image is the left image, the corresponding second image is the right image. Or, when the first image is the right image, the corresponding second image is the left image.

[0067] In another embodiment, the self-moving device is communicatively connected to other storage devices. The self-moving device can retrieve pre-stored environmental image pairs at first and second moments in the camera coordinate system from the other storage devices.

[0068] In summary, the self-moving device can acquire environmental image pairs at the first and second moments in the camera coordinate system. This prepares the self-moving device for determining point cloud data of dynamic obstacles.

[0069] Considering that the environmental image pair may contain dynamic obstacles (i.e., the number of dynamic obstacles is greater than or equal to 1) or may not contain any dynamic obstacles (i.e., the number of dynamic obstacles is equal to 0), and since the environmental image pair includes a first image and a second image, the operation performed by the self-moving device will differ depending on the number of dynamic obstacles in the different images.

[0070] In the case where there are dynamic obstacles in both the first and second images, the self-moving device can continue to determine the point cloud data of the dynamic obstacles based on the first and second images.

[0071] Point cloud data typically refers to a collection of multiple points representing a dynamic obstacle in a three-dimensional coordinate system. Each point has corresponding geometric position information. In some embodiments, each point also has corresponding color information, etc. The geometric position information of each point can be represented by three-dimensional coordinates. A three-dimensional image of the dynamic obstacle can be constructed based on point cloud data.

[0072] If a dynamic obstacle exists in the first image but not in the second image, the self-moving device can re-acquire the first and second images until it is determined that a dynamic obstacle exists in both the first and second images. At this point, the self-moving device can continue to determine the point cloud data of the dynamic obstacle.

[0073] Furthermore, when there are no dynamic obstacles in the first image but there are dynamic obstacles in the second image, or when there are no dynamic obstacles in the first image and there are no dynamic obstacles in the second image, the operation performed by the self-moving device can be referred to the above description when there are dynamic obstacles in the first image but there are no dynamic obstacles in the second image, and will not be repeated here.

[0074] It should be understood that the number of dynamic obstacles in the first image and the number of dynamic obstacles in the second image may be the same or different. The position of the same dynamic obstacle in the first image may be different from the position of the same dynamic obstacle in the second image.

[0075] S102. Based on the environmental image pair at the first moment in the camera coordinate system, determine the first point cloud data of the target dynamic obstacle at the first moment in the world coordinate system, and based on the environmental image pair at the second moment in the camera coordinate system, determine the second point cloud data of the target dynamic obstacle at the second moment in the world coordinate system.

[0076] The world coordinate system is a three-dimensional coordinate system. It is used to describe the position of dynamic obstacles in the real world.

[0077] A target dynamic obstacle is a dynamic obstacle among multiple dynamic obstacles whose motion state needs to be detected. A target dynamic obstacle may include one or more.

[0078] The first point cloud data represents the set of points on the surface of the target dynamic obstacle at the first moment in three-dimensional space. The second point cloud data represents the set of points on the surface of the target dynamic obstacle at the second moment in three-dimensional space. Therefore, both the first and second point cloud data can be regarded as three-dimensional images in space.

[0079] When only one target dynamic obstacle exists in both the first and second images, after acquiring the environmental image pair in the camera coordinate system, the mobile device can convert the environmental image pair in the camera coordinate system at the first moment into an environmental image pair in the world coordinate system at the first moment. Based on the environmental image pair in the world coordinate system at the first moment, the mobile device can determine the first point cloud data of the target dynamic obstacle in the world coordinate system at the first moment. The mobile device can also convert the environmental image pair in the camera coordinate system at the second moment into an environmental image pair in the world coordinate system at the second moment. Based on the environmental image pair in the world coordinate system at the second moment, the mobile device can determine the second point cloud data of the target dynamic obstacle in the world coordinate system at the second moment.

[0080] In some embodiments, when multiple target dynamic obstacles exist in both the first and second images, the first point cloud data consists of multiple sets, and the second point cloud data consists of multiple sets, with different point cloud data corresponding to different target dynamic obstacles. Therefore, the self-moving device needs to sequentially perform image matching between each second point cloud data and each first point cloud data to obtain multiple corresponding point cloud data pairs. The self-moving device can determine the change in the centroid position of each target dynamic obstacle based on each point cloud data pair in the multiple point cloud data pairs.

[0081] The point cloud data includes first point cloud data at a first moment and second point cloud data at a second moment, representing the same target dynamic obstacle.

[0082] The centroid location is the geometric center of the point cloud data. The point cloud data can include first point cloud data or second point cloud data, etc.

[0083] In addition, the self-moving device can determine the first point cloud data of the target dynamic obstacle at the first moment in the camera coordinate system based on the environmental image pair at the first moment in the camera coordinate system. The self-moving device can convert the first point cloud data of the target dynamic obstacle at the first moment in the camera coordinate system into the first point cloud data of the target dynamic obstacle at the first moment in the world coordinate system.

[0084] In summary, the self-moving device can determine the point cloud data of dynamic obstacles in the world coordinate system. This prepares the self-moving device for determining the centroid position of the dynamic obstacles.

[0085] S103. Based on the first point cloud data, determine the centroid position of the target dynamic obstacle at the first moment, and based on the second point cloud data, determine the centroid position of the target dynamic obstacle at the second moment.

[0086] When only one target dynamic obstacle exists in both the first and second images, after determining the first and second point cloud data, the self-moving device can determine the centroid position of the target dynamic obstacle at the first moment based on the first point cloud data. The self-moving device can then determine the centroid position of the target dynamic obstacle at the second moment based on the second point cloud data. Therefore, the self-moving device can use the centroid position of the target dynamic obstacle instead of its actual position, making the subsequent determination of the target dynamic obstacle's motion state simpler and the calculations more straightforward.

[0087] In some embodiments, when multiple target dynamic obstacles exist in both the first and second images, the first point cloud data consists of multiple sets, and the second point cloud data consists of multiple sets, with different point cloud data corresponding to different target dynamic obstacles. The self-moving device can acquire the centroid positions of each first point cloud data set and each second point cloud data set. The self-moving device can match the centroid positions of each first point cloud data set and each second point cloud data set to obtain a matching result. The self-moving device can determine the centroid position of the closest first point cloud data set and the closest second point cloud data set in the matching result as the centroid position of the same target dynamic obstacle. Thus, the self-moving device can simultaneously determine the centroid positions of multiple target dynamic obstacles, achieving simultaneous detection of multiple target dynamic obstacles.

[0088] Specifically, the self-moving device can use an iterative closest point (ICP) algorithm to match the centroid positions of each first point cloud data set with the centroid positions of each second point cloud data set, obtaining a matching result. Typically, the preset time interval between the first and second moments is short; therefore, the movement distance of multiple target dynamic obstacles within this preset time interval is also short. In other words, the movement distance between the centroid positions of multiple sets of first point cloud data sets and multiple sets of second point cloud data sets corresponding to multiple target dynamic obstacles is short. Thus, based on the matching result, the self-moving device can determine the centroid position of the first point cloud data set and the second point cloud data set whose centroid positions are closest in distance among the multiple sets of first and second point cloud data sets as the centroid position of the same target dynamic obstacle.

[0089] like Figure 2 As shown, the four target dynamic obstacles include: target dynamic obstacle 1, target dynamic obstacle 2, target dynamic obstacle 3, and target dynamic obstacle 4. The centroid positions of the first point cloud data for each of these four target dynamic obstacles at the first moment are represented by the letters A, B, C, and D enclosed in circles. The centroid positions of the second point cloud data for each of these four target dynamic obstacles at the second moment are represented by the letters E, F, G, and H enclosed in squares. After the mobile device matches the centroid positions of each first point cloud data and each second point cloud data, it can be determined that A and E are both centroid positions of target dynamic obstacle 1, B and F are both centroid positions of target dynamic obstacle 2, C and G are both centroid positions of target dynamic obstacle 3, and D and H are both centroid positions of target dynamic obstacle 4.

[0090] S104. Determine the motion state of the target dynamic obstacle based on the centroid position of the target dynamic obstacle at the first moment and the centroid position of the target dynamic obstacle at the second moment.

[0091] Among them, motion state is used to represent the state of a target dynamic obstacle relative to the world coordinate system when it is performing mechanical movement.

[0092] The motion state can be characterized by at least one of the following parameters: the distance the target dynamic obstacle moves, its speed, and its direction of movement.

[0093] The distance the target dynamic obstacle moves is the length of the first vector. The starting point of the first vector is the centroid position of the target dynamic obstacle at the first moment, and the ending point is the centroid position of the target dynamic obstacle at the second moment.

[0094] The speed of the target dynamic obstacle is the ratio of the length of the first vector to the preset duration.

[0095] The direction of motion of the target dynamic obstacle is the direction of the first vector.

[0096] In some embodiments, when only one target dynamic obstacle exists in both the first and second images, after determining the centroid position of the target dynamic obstacle, the self-moving device can determine the moving distance and direction of the target dynamic obstacle based on the centroid positions of the target dynamic obstacle at the first and second moments. The self-moving device can determine the movement speed of the target dynamic obstacle based on its moving distance and a preset time duration. Therefore, the self-moving device can determine the motion state of the target dynamic obstacle based on its moving distance, direction of motion, and speed.

[0097] It should be understood that when there are multiple target dynamic obstacles in both the first and second images, the specific implementation process of determining the motion state of multiple target dynamic obstacles by the self-moving device is the same as that of determining the motion state of a single target dynamic obstacle, and will not be elaborated here.

[0098] The motion state detection method for dynamic obstacles in this application embodiment obtains first and second images from different perspectives at different times by acquiring environmental image pairs at first and second moments in the camera coordinate system. Further, based on the environmental image pairs at the first moment in the camera coordinate system, the first point cloud data of the target dynamic obstacle at the first moment in the world coordinate system can be determined, and based on the environmental image pairs at the second moment in the camera coordinate system, the second point cloud data of the target dynamic obstacle at the second moment in the world coordinate system can be determined. This allows the acquisition of point cloud data of the target dynamic obstacle at various moments in the camera coordinate system through the first and second images from different perspectives at various times in the camera coordinate system, and the conversion of the point cloud data of the target dynamic obstacle at various moments in the camera coordinate system into point cloud data of the target dynamic obstacle at various moments in the world coordinate system. This achieves the conversion of the relative motion of the target dynamic obstacle in the camera coordinate system into the absolute motion of the target dynamic obstacle in the world coordinate system.

[0099] Subsequently, based on the first point cloud data, the centroid position of the target dynamic obstacle at the first moment can be determined, and based on the second point cloud data, the centroid position of the target dynamic obstacle at the second moment can be determined. By utilizing the point cloud data of the target dynamic obstacle at each moment, the centroid position of the target dynamic obstacle can be obtained at each moment, thus representing the actual position of the target dynamic obstacle with its centroid position, reducing computational load and improving computational efficiency. Based on this, the motion state of the target dynamic obstacle can be determined based on its centroid position at the first and second moments. This allows the autonomous moving device to accurately avoid the target dynamic obstacle by combining its motion state with the objective dynamic obstacle's position.

[0100] This enables the detection of the motion state of multiple dynamic obstacles, obtaining their movement distance, speed, and direction. Based on these parameters, the system can accurately avoid multiple dynamic obstacles, reducing the risk of collisions between the mobile device and these obstacles.

[0101] In this embodiment, the self-moving device can determine the point cloud data of the target dynamic obstacle using various implementation methods. Furthermore, the specific implementation principle for determining the point cloud data of one target dynamic obstacle is the same as the specific implementation principle for determining the point cloud data of multiple target dynamic obstacles.

[0102] Based on the description in S102, the following will be combined with... Figure 3 This example illustrates a feasible implementation method for a self-moving device to determine the point cloud data of a target dynamic obstacle.

[0103] Please see Figure 3 , Figure 3 A flowchart illustrating the motion state detection method for a second dynamic obstacle provided in an embodiment of this application is shown. Figure 3 As shown, the motion state detection method for dynamic obstacles in this application embodiment may include:

[0104] S201. Based on the environmental image pair at the first moment in the camera coordinate system, determine the first disparity map at the first moment in the camera coordinate system, and based on the environmental image pair at the second moment in the camera coordinate system, determine the second disparity map at the second moment in the camera coordinate system.

[0105] The first disparity map and the second disparity map both include the disparity values ​​of all pixels of the target dynamic obstacle at the corresponding time. That is, the first disparity map includes the disparity values ​​of all pixels of the target dynamic obstacle at the first time. The second disparity map includes the disparity values ​​of all pixels of the target dynamic obstacle at the second time. The disparity value of each pixel is the difference between its coordinates in the first image and its coordinates in the second image.

[0106] The self-moving device can determine the disparity values ​​of all pixels of the target dynamic obstacle at the first moment based on the environmental image pair at the first moment in the camera coordinate system. Specifically, at the first moment, the self-moving device can determine the disparity values ​​of all pixels of the target dynamic obstacle at the first moment based on the coordinates of all pixels of the target dynamic obstacle in the first image and the coordinates of all pixels of the target dynamic obstacle in the second image. The self-moving device can then determine the first disparity map at the first moment in the camera coordinate system based on the disparity values ​​of all pixels of the target dynamic obstacle at the first moment.

[0107] The specific implementation principle of determining the second disparity map at the second moment in the camera coordinate system by the self-moving device is the same as that of determining the first disparity map at the first moment in the camera coordinate system, and will not be elaborated here.

[0108] Thus, the self-moving device can determine the first disparity map at the first moment in the camera coordinate system and the second disparity map at the second moment in the camera coordinate system.

[0109] S202. Based on the first disparity map at the first moment in the camera coordinate system, determine the first depth map at the first moment in the camera coordinate system, and based on the second disparity map at the second moment in the camera coordinate system, determine the second depth map at the second moment in the camera coordinate system.

[0110] Both the first and second depth maps include the distance values ​​of all pixels of the target dynamic obstacle at the corresponding time point. That is, the first depth map includes the distance values ​​of all pixels of the target dynamic obstacle at the first time point, and the second depth map includes the distance values ​​of all pixels of the target dynamic obstacle at the second time point. The distance value of each pixel represents the distance between each pixel and the camera.

[0111] After determining the first disparity map and the second disparity map, the self-moving device can determine the distance between all pixels of the target dynamic obstacle in the first disparity map and the camera, thereby obtaining the first depth map at the first moment in the camera coordinate system.

[0112] In some embodiments, the self-moving device can recover the distance values ​​of all pixels of the target dynamic obstacle at a first moment using the principle of triangulation, based on the disparity values ​​of all pixels of the target dynamic obstacle at the first moment. Thus, the self-moving device can determine the first depth map in the camera coordinate system at the first moment based on the distance values ​​of all pixels of the target dynamic obstacle at the first moment.

[0113] In this embodiment, the self-moving device employs binocular vision ranging to determine the distance values ​​of all pixels of the target dynamic obstacle. Specifically, it determines the distance values ​​of all pixels of the target dynamic obstacle based on their disparity values. Therefore, the distance values ​​of all pixels of the target dynamic obstacle are not affected by other factors in the external environment (such as fine particles like dust), resulting in higher accuracy. Compared to existing laser ranging or time-of-flight (TOF) ranging methods, this approach is more stable.

[0114] The specific implementation principle of determining the second depth map at the second moment in the camera coordinate system by the self-moving device is the same as that of determining the first depth map at the first moment in the camera coordinate system, and will not be elaborated here.

[0115] Thus, the mobile device can determine the first depth map at the first moment in the camera coordinate system and the second depth map at the second moment in the camera coordinate system.

[0116] S203. Based on the first depth map at the first moment in the camera coordinate system, determine the first point cloud data of the target dynamic obstacle at the first moment in the world coordinate system, and based on the second depth map at the second moment in the camera coordinate system, determine the second point cloud data of the target dynamic obstacle at the second moment in the world coordinate system.

[0117] After determining the first depth map and the second depth map, the self-moving device can determine the first point cloud data of the target dynamic obstacle in the world coordinate system at the first moment based on the first depth map in the camera coordinate system at the first moment. The self-moving device can determine the second point cloud data of the target dynamic obstacle in the world coordinate system at the second moment based on the second depth map in the camera coordinate system at the second moment.

[0118] In summary, the self-moving device can determine the first point cloud data of the target dynamic obstacle at the first moment in the world coordinate system and the second point cloud data of the target dynamic obstacle at the second moment in the world coordinate system.

[0119] Based on the description in S201, the following will be combined with... Figure 4 This paper details a feasible method for determining the first disparity map from a mobile device.

[0120] Please see Figure 4 , Figure 4 A flowchart illustrating the third method for detecting the motion state of dynamic obstacles provided in this application is shown. Figure 4 As shown, the motion state detection method for dynamic obstacles in this application embodiment may include:

[0121] S301. Obtain the first moment's environmental image pair in the camera coordinate system.

[0122] The specific implementation of S301 can be found in the description of the first moment's environmental image pair obtained by the self-moving device in S101, which will not be repeated here.

[0123] S302. Input the first image of the environmental image pair at the first moment into the semantic segmentation network to obtain the position of the target dynamic obstacle in the first image.

[0124] After acquiring the environmental image pair at the first moment, the self-moving device can input the first image of the environmental image pair at the first moment into the trained semantic segmentation network to obtain the semantic instance segmentation result. Based on the semantic instance segmentation result, the self-moving device can determine the position of the target dynamic obstacle in the first image.

[0125] After performing pixel-level semantic segmentation on the first image using a semantic segmentation network, the self-moving device can determine the position of the target dynamic obstacle in the first image, that is, determine the coordinates of all pixels of the target dynamic obstacle in the first image. Thus, the target dynamic obstacle in the first image can be accurately identified. When the self-moving device executes S304, it only needs to calculate the disparity values ​​of all pixels of the target dynamic obstacle, without needing to calculate the disparity values ​​of all pixels in the first image, reducing the computational load and improving the accuracy of the calculation.

[0126] In some embodiments, the semantic segmentation network may employ a mask-region-based convolutional neural network (Mask-RCNN).

[0127] S303. Determine the position of the target dynamic obstacle in the second image of the environmental image pair at the first moment according to the environment matching algorithm.

[0128] After acquiring the environmental image pair at the first moment, the self-moving device can determine the position of the target dynamic obstacle in the second image of the environmental image pair at the first moment according to the environment matching algorithm, that is, determine the coordinates of all pixels of the target dynamic obstacle in the second image.

[0129] In some embodiments, the environment matching algorithm may employ a semi-global matching (SGM) algorithm or a multi-view stereo algorithm (patch match), etc.

[0130] It should be understood that there is no temporal order between S302 and S303, and S302 and S303 can be executed simultaneously or sequentially.

[0131] Additionally, the self-moving device can input the second image into a semantic segmentation network to obtain the position of the target dynamic obstacle in the second image. At this point, the self-moving device can determine the position of the target dynamic obstacle in the first image based on an environment matching algorithm.

[0132] S304. Based on the position of the target dynamic obstacle in the first image at the first moment and the position of the target dynamic obstacle in the second image at the first moment, determine the disparity value of all pixels of the target dynamic obstacle at the first moment.

[0133] S305. Based on the disparity values ​​of all pixels of the target dynamic obstacle, determine the first disparity map in the camera coordinate system at the first moment.

[0134] The specific implementation methods of S304 and S305 can be found in the description of the first disparity map of the first moment in the camera coordinate system obtained by the self-moving device in S201, which will not be repeated here.

[0135] It should be understood that the specific implementation process of the self-moving device determining the second disparity map in the embodiments of this application can be found in the description of S301-S305, and will not be repeated here.

[0136] Based on the description in S203, the following will be combined with... Figure 5 This paper details a feasible implementation method for determining the first depth map on a mobile device.

[0137] Please see Figure 5 , Figure 5 A flowchart illustrating the fourth method for detecting the motion state of dynamic obstacles provided in this application is shown. Figure 5 As shown, the motion state detection method for dynamic obstacles in this application embodiment may include:

[0138] S401. Based on the first depth map at the first moment in the camera coordinate system, determine the first point cloud data of the target dynamic obstacle at the first moment in the camera coordinate system.

[0139] After determining the first depth map at the first moment, the self-moving device can determine the first point cloud data of the target dynamic obstacle at the first moment in the camera coordinate system based on the first depth map at the first moment in the camera coordinate system. At this time, the self-moving device can determine the relative motion of the target dynamic obstacle in the camera coordinate system based on the first point cloud data of the target dynamic obstacle at the first moment in the camera coordinate system.

[0140] Considering that the relative motion of the target dynamic obstacle will change over time, the self-moving device can convert the relative motion of the target dynamic obstacle into the absolute motion of the target dynamic obstacle, thereby ensuring that the motion state of the target dynamic obstacle does not change over time.

[0141] S402. Obtain the mapping relationship between the camera coordinate system and the world coordinate system.

[0142] The mapping relationship can be a transformation matrix, i.e., an odom matrix. The odom matrix can include rotation matrices and translation matrices.

[0143] In some embodiments, the self-moving device can determine its pose in the camera coordinate system based on a stereo image pair at a first moment and a stereo image pair at a second moment in the camera coordinate system. For example, the self-moving device can use algorithms such as simultaneous localization and mapping (SLAM), VINS-fusion, ORB_SLAM3, and semi-direct visualodometry (SVO) to determine its pose.

[0144] Pose can be characterized by factors such as translation distance and / or rotation angle of the self-moving device.

[0145] The self-moving device can determine the transformation matrix between the camera coordinate system and the world coordinate system based on its pose in the camera coordinate system. Therefore, the self-moving device can obtain the mapping relationship between the camera coordinate system and the world coordinate system.

[0146] It should be understood that there is no temporal order between S401 and S402, and S401 and S402 can be executed simultaneously or sequentially.

[0147] S403. Based on the point cloud data of the target dynamic obstacle at the first moment in the camera coordinate system and the mapping relationship, determine the first point cloud data of the target dynamic obstacle at the first moment in the world coordinate system.

[0148] After determining the first point cloud data of the target dynamic obstacle at the first moment in the camera coordinate system and the mapping relationship between the camera coordinate system and the world coordinate system, the self-moving device can determine the first point cloud data of the target dynamic obstacle at the first moment in the world coordinate system based on the point cloud data of the target dynamic obstacle at the first moment in the camera coordinate system and the mapping relationship.

[0149] It should be understood that the specific implementation process of the self-moving device determining the second depth map in the embodiments of this application can be found in the description of S401-S403, and will not be repeated here.

[0150] Please see Figure 6 , Figure 6 A schematic block diagram of a motion state detection device for dynamic obstacles provided in an embodiment of this application is shown. Figure 6 As shown, the motion state detection device 500 for dynamic obstacles in this application embodiment may include:

[0151] The acquisition module 501 is used to acquire environmental image pairs at a first moment and environmental image pairs at a second moment in the camera coordinate system. The environmental image pairs include a first image and a second image. The first image and the second image are environmental images acquired from different perspectives at the same moment. The second moment is a time point after the first moment.

[0152] The determining module 502 is used to determine the first point cloud data of the target dynamic obstacle in the world coordinate system at the first moment based on the environmental image pair in the camera coordinate system at the first moment, and to determine the second point cloud data of the target dynamic obstacle in the world coordinate system at the second moment based on the environmental image pair in the camera coordinate system at the second moment; to determine the centroid position of the target dynamic obstacle at the first moment based on the first point cloud data, and to determine the centroid position of the target dynamic obstacle at the second moment based on the second point cloud data.

[0153] The processing module 503 is used to determine the motion state of the target dynamic obstacle based on the centroid position of the target dynamic obstacle at the first moment and the centroid position of the target dynamic obstacle at the second moment. The motion state includes at least one parameter among the moving distance, moving speed and moving direction of the target dynamic obstacle.

[0154] In some embodiments, the determining module 502 is further configured to: determine a first disparity map in the camera coordinate system at a first moment based on the environmental image pair in the camera coordinate system at a first moment; and determine a second disparity map in the camera coordinate system at a second moment based on the environmental image pair in the camera coordinate system at a second moment, wherein both the first disparity map and the second disparity map include the disparity values ​​of all pixels of the target dynamic obstacle at the corresponding moment; determine a first depth map in the camera coordinate system at a first moment based on the first disparity map in the camera coordinate system at a first moment; and determine a second depth map in the camera coordinate system at a second moment based on the second disparity map in the camera coordinate system at a second moment, wherein both the first depth map and the second depth map include the distance values ​​of all pixels of the target dynamic obstacle at the corresponding moment; and determine first point cloud data of the target dynamic obstacle in the world coordinate system at a first moment based on the first depth map in the camera coordinate system at a first moment; and determine second point cloud data of the target dynamic obstacle in the world coordinate system at a second moment based on the second depth map in the camera coordinate system at a second moment.

[0155] In some embodiments, the determining module 502 is further configured to: acquire an environmental image pair at a first moment in the camera coordinate system; input the first image of the environmental image pair at the first moment into a semantic segmentation network to obtain the position of the target dynamic obstacle in the first image; determine the position of the target dynamic obstacle in the second image of the environmental image pair at the first moment according to an environment matching algorithm; determine the disparity value of all pixels of the target dynamic obstacle at the first moment according to the position of the target dynamic obstacle in the first image at the first moment and the position of the target dynamic obstacle in the second image at the first moment; and determine the first disparity map at the first moment in the camera coordinate system according to the disparity value of all pixels of the target dynamic obstacle.

[0156] In some embodiments, the determining module 502 is further configured to determine the first point cloud data of the target dynamic obstacle at the first moment in the camera coordinate system based on the first depth map at the first moment in the camera coordinate system; obtain the mapping relationship between the camera coordinate system and the world coordinate system; and determine the first point cloud data of the target dynamic obstacle at the first moment in the world coordinate system based on the point cloud data of the target dynamic obstacle at the first moment in the camera coordinate system and the mapping relationship.

[0157] In some embodiments, the processing module 503 is further configured to determine the moving distance and direction of the target dynamic obstacle based on the centroid position of the target dynamic obstacle at the first moment and the centroid position of the target dynamic obstacle at the second moment; and to determine the moving speed of the target dynamic obstacle based on the moving distance of the target dynamic obstacle and a preset duration; the preset duration is the difference between the second moment and the first moment.

[0158] In some embodiments, when there are multiple target dynamic obstacles, the first point cloud data is in multiple sets and the second point cloud data is in multiple sets; different point cloud data correspond to different target dynamic obstacles; the determining module 502 is further configured to sequentially match each second point cloud data with each first point cloud data to obtain multiple corresponding point cloud data pairs; the point cloud data pairs include first point cloud data at a first moment and second point cloud data at a second moment representing the same target dynamic obstacle.

[0159] In some embodiments, when there are multiple target dynamic obstacles, the first point cloud data is in multiple sets, and the second point cloud data is in multiple sets; different point cloud data correspond to different target dynamic obstacles; the processing module 503 is further configured to obtain the centroid position of each first point cloud data and the centroid position of each second point cloud data; and determine the centroid position of the first point cloud data and the second point cloud data that are closest to each first point cloud data as the centroid position of the same target dynamic obstacle.

[0160] It should be understood that the motion state detection device 500 for dynamic obstacles in this application embodiment can be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. It can also be implemented using software. Figure 1 , Figure 3 , Figure 4 as well as Figure 5 The method shown for controlling the movement of the device, when implemented through software. Figure 1 , Figure 3 , Figure 4 as well as Figure 5 In the motion state detection method for dynamic obstacles shown, the motion state detection device 500 and its various modules can also be software modules.

[0161] This application also provides a readable storage medium storing a computer program, which, when executed by a processor, implements the steps described in the above-described method embodiments.

[0162] This application provides a computer program product that, when run on a self-moving device, enables the self-moving device to implement the steps described in the various method embodiments above.

[0163] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0164] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0165] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0166] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0167] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0168] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0169] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0170] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for detecting the motion state of a dynamic obstacle, characterized in that, The method includes: Acquire environmental image pairs at a first moment and at a second moment in the camera coordinate system. The environmental image pairs include a first image and a second image. The first image and the second image are environmental images acquired from different perspectives at the same moment. The second moment is a time point after the first moment. If a dynamic obstacle exists in both the first and second images, the system determines the first point cloud data of the target dynamic obstacle in the world coordinate system at the first moment based on the environmental image pair in the camera coordinate system at the first moment, and determines the second point cloud data of the target dynamic obstacle in the world coordinate system at the second moment based on the environmental image pair in the camera coordinate system at the second moment. This includes: determining the first disparity map in the camera coordinate system at the first moment based on the environmental image pair in the camera coordinate system at the first moment, and determining the camera coordinate system at the second moment based on the environmental image pair in the camera coordinate system at the second moment. The second disparity map at the second moment in the camera coordinate system, wherein both the first disparity map and the second disparity map include the disparity values ​​of all pixels of the target dynamic obstacle at the corresponding moment; a first depth map at the first moment in the camera coordinate system is determined based on the first disparity map at the first moment in the camera coordinate system, and a second depth map at the second moment in the camera coordinate system is determined based on the second disparity map at the second moment in the camera coordinate system, wherein both the first depth map and the second depth map include the distance values ​​of all pixels of the target dynamic obstacle at the corresponding moment; and the distance map at the second moment in the camera coordinate system is determined based on the first depth map at the first moment in the camera coordinate system. The first point cloud data of the target dynamic obstacle at the first moment in the boundary coordinate system, and the second point cloud data of the target dynamic obstacle at the second moment in the world coordinate system, determined according to the second depth map at the second moment in the camera coordinate system; the first point cloud data is used to represent the set of points on the surface of the target dynamic obstacle at the first moment in three-dimensional space; the second point cloud data is used to represent the set of points on the surface of the target dynamic obstacle at the second moment in three-dimensional space; the step of determining the first disparity map at the first moment in the camera coordinate system based on the environmental image pair at the first moment in the camera coordinate system includes: acquiring the first point cloud data of the target dynamic obstacle at the first moment in the camera coordinate system. The first image of the first time-phase environment image pair is input into a semantic segmentation network to obtain the position of the target dynamic obstacle in the first image; the position of the target dynamic obstacle in the second image of the first time-phase environment image pair is determined according to an environment matching algorithm; based on the position of the target dynamic obstacle in the first image and the position of the target dynamic obstacle in the second image at the first time-phase, the disparity value of all pixels of the target dynamic obstacle at the first time-phase is determined; based on the disparity value of all pixels of the target dynamic obstacle, the first disparity map in the camera coordinate system at the first time-phase is determined. Based on the first point cloud data, the centroid position of the target dynamic obstacle at the first moment is determined, and based on the second point cloud data, the centroid position of the target dynamic obstacle at the second moment is determined. Based on the centroid position of the target dynamic obstacle at the first moment and the centroid position of the target dynamic obstacle at the second moment, the motion state of the target dynamic obstacle is determined, and the motion state includes at least one parameter among the moving distance, moving speed and moving direction of the target dynamic obstacle.

2. The method as described in claim 1, characterized in that, The step of determining the first point cloud data of the target dynamic obstacle in the world coordinate system at the first moment based on the first depth map in the first moment in the camera coordinate system includes: Based on the first depth map at the first moment in the camera coordinate system, determine the first point cloud data of the target dynamic obstacle at the first moment in the camera coordinate system; Obtain the mapping relationship between the camera coordinate system and the world coordinate system; Based on the point cloud data of the target dynamic obstacle at the first moment in the camera coordinate system and the mapping relationship, the first point cloud data of the target dynamic obstacle at the first moment in the world coordinate system is determined.

3. The method as described in claim 1, characterized in that, Determining the motion state of the target dynamic obstacle based on its centroid position at the first time moment and its centroid position at the second time moment includes: Based on the centroid position of the target dynamic obstacle at the first moment and the centroid position of the target dynamic obstacle at the second moment, determine the moving distance and direction of movement of the target dynamic obstacle; The movement speed of the target dynamic obstacle is determined based on the moving distance of the target dynamic obstacle and the preset duration; the preset duration is the difference between the second moment and the first moment.

4. The method according to any one of claims 1-3, characterized in that, When multiple dynamic obstacles exist, the first point cloud data consists of multiple sets, and the second point cloud data consists of multiple sets; different point cloud data correspond to different dynamic obstacles; the method further includes: Each second point cloud data is sequentially matched with each first point cloud data to obtain multiple corresponding point cloud data pairs; the point cloud data pairs include first point cloud data at a first moment and second point cloud data at a second moment representing the same target dynamic obstacle.

5. The method according to claim 1, characterized in that, When multiple dynamic obstacles exist, the first point cloud data consists of multiple sets, and the second point cloud data consists of multiple sets; different point cloud data correspond to different dynamic obstacles; the method further includes: Obtain the centroid position of each first point cloud data and the centroid position of each second point cloud data; The centroid position of the first point cloud data and the centroid position of the second point cloud data that are closest to each other are determined as the centroid position of the same target dynamic obstacle.

6. A motion state detection device for dynamic obstacles, characterized in that, Includes a module for performing the method as described in any one of claims 1-5.

7. A self-moving device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1-5.

8. A readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-5.

Images