Map construction method and device, and storage medium

Abstract

A kind of map construction method, comprising: according to the first probability of each pixel point and distance, the second probability that each pixel point belongs to obstacle is obtained, according to second probability and position information, determine the voxel corresponding to each pixel point, according to voxel, generate the three-dimensional probability map of current moment, according to the preset adjustment parameter, update the probability value of each voxel in the three-dimensional probability map of last moment, the probability value of each voxel in the three-dimensional probability map of current moment is multiplied with the probability value of the corresponding voxel in the three-dimensional probability map of last moment after updating, and according to the multiplication result, the probability value of each voxel in the three-dimensional probability map of current moment is updated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of map construction, and in particular to a map construction method and device and a storage medium. BACKGROUND

[0002] The statements herein merely provide background information related to the present application and do not necessarily constitute exemplary technologies.

[0003] Currently, a self-moving device can use a three-dimensional probability map for obstacle avoidance to ensure normal movement and safety of the self-moving device. However, the construction of a general three-dimensional probability map is mainly in the form of Bayesian filtering. Due to sensor noise and environmental complexity, there is a perception error, and the Bayesian filtering form cannot adapt to dynamic changes in the environment, so that the accuracy of the constructed three-dimensional probability map is low, resulting in a risk of collision with obstacles when the self-moving device uses the three-dimensional probability map for obstacle avoidance. Therefore, how to improve the accuracy of the three-dimensional probability map to reduce the risk of collision between the self-moving device and the obstacle is a problem to be solved at present. SUMMARY

[0004] According to various embodiments of the present application, a map construction method, device and storage medium are provided.

[0005] The embodiments of the present application provide a map construction method, comprising:

[0006] Obtaining obstacle information in image data at a current time, the image data being collected by an image collection device on a self-moving device, the obstacle information including a distance of each pixel point belonging to an obstacle to the image collection device, position information of each pixel point, and a first probability of each pixel point belonging to the obstacle;

[0007] According to the first probability and the distance of each pixel point, a second probability of each pixel point belonging to the obstacle is obtained;

[0008] According to the second probability and the position information of each pixel point, a voxel corresponding to each pixel point is determined, and a three-dimensional probability map at the current time is generated according to the voxel, wherein a probability value carried by each voxel in the three-dimensional probability map at the current time is the second probability;

[0009] Obtaining a three-dimensional probability map at a previous time, and updating a probability value of each voxel in the three-dimensional probability map at the previous time according to a preset adjustment parameter;

[0010] multiply the probability value of each voxel in the three-dimensional probability map of the current moment with the probability value of the corresponding voxel in the updated three-dimensional probability map of the previous moment, and update the probability value of each voxel in the three-dimensional probability map of the current moment according to the multiplication result;

[0011] take the updated three-dimensional probability map of the current moment as the target three-dimensional probability map of the current moment.

[0012] The embodiment of the present application further provides a map construction device, the map construction device comprising a processor, a memory, a computer program stored on the memory and executable by the processor, and a data bus used for realizing connection communication between the processor and the memory, wherein the computer program is executed by the processor to realize any one of the map construction methods provided in the specification of the present application.

[0013] The embodiment of the present application further provides a storage medium used for computer readable storage, the storage medium storing one or more programs, the one or more programs being executable by one or more processors to realize any one of the map construction methods provided in the specification of the present application.

[0014] Details of one or more embodiments of the present application are presented in the following drawings and description. Other features, objects and advantages of the present application will become apparent from the specification, drawings and claims. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0016] Figure 1 is a flowchart of a map construction method provided by the embodiment of the present application;

[0017] Figure 2 is a flowchart of a map construction method provided by the embodiment of the present application; Figure 1

[0018] Figure 3 is a flowchart of a map construction method provided by the embodiment of the present application; Figure 1

[0019] Figure 4 is a flowchart of a map construction method provided by the embodiment of the present application; Figure 1

[0020] Figure 5 ​​​is a scene schematic diagram of obstacle avoidance of a self-moving device in the embodiment of the present application.

[0021] Figure 6 is Figure 1 is a sub-step flow schematic diagram of a map construction method in the embodiment of the present application.

[0022] Figure 7 is a structure schematic block diagram of a map construction device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be clearly and completely described in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0024] The flowchart shown in the drawings is only an example and does not necessarily include all the contents and operations / steps, nor does it necessarily execute in the order described. For example, some operations / steps can be further decomposed, combined or partially merged, so the actual execution order may be changed according to the actual situation.

[0025] It should be understood that the terms used in the present application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, unless otherwise clear from the context, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0026] The embodiments of the present application provide a map construction method, device and storage medium. The map construction method can be applied to a self-moving device, including a sweeping machine, a mower, a meal delivery machine, etc. The map construction method can also be applied to a server or a terminal device. The server can be an independent server, a server cluster composed of multiple servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN), and big data and artificial intelligence platforms, etc. Basic cloud computing services. The terminal device can be a remote control device, a mobile phone, a tablet computer, a notebook computer, a desktop computer, a personal digital assistant and a wearable device, etc.

[0027] Some embodiments of the present application will be described in detail below in combination with the drawings. In the case of no conflict, the embodiments described below and the features in the embodiments can be combined with each other.

[0028] Referring to Figure 1 , Figure 1 is a flowchart of a map construction method provided by an embodiment of the present application.

[0029] As Figure 1 shown, the map construction method comprises steps S101 to S106.

[0030] Step S101, obtaining obstacle information in image data at a current time.

[0031] In the embodiment of the present application, the image data is collected by an image collection device on a self-moving device, and the image data at the current time is image data collected by the image collection device on the self-moving device at the current time.

[0032] The pixel points possibly belonging to the obstacle are filtered out from the image data, and the filtering method can adopt RANSAC algorithm (Random Sample Consensus), plane model segmentation algorithm, Euclidean Cluster Extraction, Color-based region growing segmentation, Conditional Euclidean Clustering, etc., which are not limited here.

[0033] Obstacle information of the pixel points belonging to the obstacle is obtained, and the obstacle information comprises distance of each pixel point belonging to the obstacle to the image collection device, position information of each pixel point, and first probability of each pixel point belonging to the obstacle. The image collection device can comprise an RGB camera and / or a depth camera.

[0034] It should be noted that, since the first probability of the pixel point in the obstacle information represents the possibility of the pixel point belonging to the obstacle, for example, when the first probability is less than a set threshold, it means that the possibility of the pixel point belonging to the obstacle is low. That is, there are some pixel points in the obstacle information which actually may not belong to the obstacle, that is, the first probability in the obstacle information is only a primary judgment of the pixel point belonging to the obstacle, and the process of further judging the pixel point as the obstacle needs to be combined with steps S102 to S106.

[0035] In an embodiment, as Figure 2 shown, step S101 comprises sub-step S1011 to sub-step S1012.

[0036] In the substep S1011, when the image data is an RGB image, semantic recognition is performed on each pixel point in the RGB image to obtain a semantic recognition result.

[0037] In the substep S1012, a pixel point with a semantic label of an obstacle label is obtained, and a semantic label probability of the obtained pixel point is taken as the first probability.

[0038] In the embodiment of the application, the semantic recognition result includes a semantic label and a semantic label probability of each pixel point in the RGB image. The semantic label describes the type of the pixel point, and the semantic label probability describes the probability that the pixel point belongs to the type corresponding to the semantic label. The semantic label includes an obstacle label and a non-obstacle label. The obstacle label is used to describe that the type of the pixel point is an obstacle, and the non-obstacle label is used to describe that the type of the pixel point is a non-obstacle.

[0039] In an embodiment, when the image data is an RGB image, the RGB image is input into a preset semantic segmentation model for processing to obtain a semantic segmentation image corresponding to the RGB image. The semantic segmentation image includes a semantic label and a semantic label probability of each pixel point in the RGB image. The preset semantic segmentation model is a pre-trained neural network model, and the neural network model includes but is not limited to a convolutional neural network (CNN), a fully convolutional neural network (FCN), and a deep convolutional neural network (DCNN).

[0040] In an embodiment, as shown in FIG. 1, the step S101 includes a substep S1013 and a substep S1014. Figure 3

[0041] In the substep S1013, when the image data is a depth image, the depth image is converted to obtain point cloud data.

[0042] In the substep S1014, a point cloud belonging to an obstacle and a point cloud probability of each point in the point cloud are extracted from the point cloud data, and the point cloud probability is taken as the first probability.

[0043] ​In the embodiments of the present application, since the points in the point cloud data correspond to the pixel points in the depth image one by one, the corresponding relationship between the points in the point cloud data and the pixel points in the depth image and the point cloud of the obstacle and the point cloud probability of each point in the point cloud can be used to determine all the pixel points corresponding to the point cloud of the obstacle as the pixel points belonging to the obstacle. The point cloud probability of each point in the point cloud is taken as the first probability of the corresponding pixel point belonging to the obstacle, and the point cloud probability represents the probability of the point in the point cloud belonging to the obstacle.

[0044] In an embodiment, the position information of each pixel point in the depth image in the camera coordinate system and the distance between each pixel point in the depth image and the image acquisition device are obtained; the three-dimensional position information of each pixel point in the depth image in the camera coordinate system is determined according to the position information of each pixel point in the depth image in the camera coordinate system and the distance between each pixel point in the depth image and the image acquisition device; the intrinsic matrix and the extrinsic matrix of the image acquisition device are obtained, and the three-dimensional position information of each pixel point in the depth image in the camera coordinate system is converted into the three-dimensional position information in the world coordinate system according to the intrinsic matrix and the extrinsic matrix, to obtain the point cloud data corresponding to the depth image.

[0045] In step S102, the second probability of each pixel point belonging to the obstacle is obtained according to the first probability and the distance of each pixel point.

[0046] In the embodiments of the present application, by comprehensively considering the first probability of the pixel point belonging to the obstacle and the distance of the pixel point to the image acquisition device, the second probability of the pixel point belonging to the obstacle can be more accurately determined, so as to improve the accuracy of the probability of the pixel point belonging to the obstacle.

[0047] In an embodiment, the first probability of the pixel point can include the semantic label probability and / or the point cloud probability of the pixel point. For example, the distance probability of the pixel point is multiplied by the semantic label probability to obtain the second probability of the pixel point belonging to the obstacle; or the distance probability of the pixel point is multiplied by the point cloud probability to obtain the second probability of the pixel point belonging to the obstacle; or the distance probability of the pixel point, the semantic label probability and the point cloud probability are multiplied to obtain the second probability of the pixel point belonging to the obstacle.

[0048] In an embodiment, as shown in FIG. 10, step S102 includes sub-step S1021 to sub-step S1022. Figure 4

[0049] In sub-step S1021, the distance probability of each pixel point is determined according to the distance of each pixel point to the image acquisition device.

[0050] In sub-step S1022, the first probability and the distance probability are multiplied to obtain the second probability of each pixel point belonging to the obstacle.​

[0051] In the embodiment of the present application, the distance probability of a pixel point represents the probability that the corresponding pixel point belongs to an obstacle during the movement of the self-mobile device, that is, during the change of the distance from the acquisition device on the self-mobile device to the obstacle. The distance probability of the pixel point is inversely proportional to the distance from the pixel point to the image acquisition device. That is, the farther the distance from the pixel point to the image acquisition device, the smaller the distance probability of the pixel point, and the closer the distance from the pixel point to the image acquisition device, the larger the distance probability of the pixel point. Since the distance probability of the pixel point is inversely proportional to the distance from the pixel point to the image acquisition device, when the distance probability is used to generate a three-dimensional probability map, the weight of the obstacle point cloud in the three-dimensional probability map can be made to decrease as the distance becomes farther. In this way, the self-mobile device can observe the changes of the obstacle in the three-dimensional probability map, and can then accurately avoid the obstacle, reducing the risk of collision between the self-mobile device and the obstacle.

[0052] For example, Figure 5 As shown in the figure, the mobile device 11 moves from position 21 to position 22, and the mobile device 11 needs to turn to reach position 22. However, there is an obstacle 12 around position 22. Due to the perception error of the mobile device 11, the commonly used Bayesian filter obstacle avoidance method cannot adapt to the dynamic changes of the environment, so the mobile device 11 cannot observe the nearby obstacle 12 when turning. In other words, the mobile device 11 cannot perceive the obstacle 12 at the turn, which can easily cause the mobile device 11 to collide with the obstacle 12. Figure 5 The gray rectangle shown in the figure represents the self-moving device 11 when it collides with obstacle 12. However, the present application solution uses the distance probability obtained from the distance between the image acquisition device and the obstacle, combined with the first probability of the pixel belonging to the obstacle, to further determine the probability that the pixel belongs to the obstacle, thereby reflecting the position change of obstacle 12 relative to self-moving device 11. Using the distance probability to generate a three-dimensional probability map, the weight of the obstacle point cloud in the three-dimensional probability map increases with the closer the distance. In this way, the self-moving device can use the previously acquired three-dimensional probability map to determine the position of obstacle 12 when turning, thereby allowing self-moving device 11 to avoid obstacle 12.

[0053] In one embodiment, the inverse of the distance from the pixel to the image acquisition device is determined as the distance probability of the pixel. In other embodiments, a preset inverse proportionality constant may be obtained and divided by the distance from the pixel to the image acquisition device to obtain the distance probability of the pixel. The preset inverse proportionality constant can be set based on actual conditions and is not specifically limited in this embodiment of the present application. For example, assuming the distance from the pixel to the image acquisition device is d and the preset inverse proportionality constant is k, the distance probability of the pixel can be expressed as k / d.

[0054] In an embodiment, as shown in FIG. 1, step S102 comprises sub-step S1023 to sub-step S1025. Figure 6

[0055] Sub-step S1023, determining a distance probability of each pixel point according to a distance of each pixel point to the image acquisition device;

[0056] Sub-step S1024, obtaining a semantic weight coefficient and a point cloud weight coefficient;

[0057] Sub-step S1025, determining a second probability according to the semantic label probability of the pixel point, the point cloud probability, the distance probability, the semantic weight coefficient and the point cloud weight coefficient.

[0058] In an embodiment of the present application, the semantic weight coefficient is used to describe the accuracy of the semantic label probability of the pixel point. The greater the semantic weight coefficient is, the higher the accuracy of the semantic label probability of the pixel point is. The smaller the semantic weight coefficient is, the lower the accuracy of the semantic label probability of the pixel point is. The point cloud weight coefficient is used to describe the accuracy of the point cloud probability of the pixel point. The greater the point cloud weight coefficient is, the higher the accuracy of the point cloud probability of the pixel point is. The smaller the point cloud weight coefficient is, the lower the accuracy of the point cloud probability of the pixel point is. By comprehensively considering the semantic label probability of the pixel point, the point cloud probability, the distance probability, the semantic weight coefficient and the point cloud weight coefficient, the accuracy of the probability of the pixel point belonging to the obstacle can be improved.

[0059] In an embodiment, an image quality index of the RGB image is obtained, and the image quality index of the RGB image is determined as a first confidence degree of the semantic label probability of the pixel point; wherein the image quality index of the RGB image is used to describe the image quality of the RGB image; the semantic weight coefficient is determined according to the first confidence degree; an image quality index of the depth image is obtained, and the image quality index of the depth image is determined as a second confidence degree of the point cloud probability of the pixel point; wherein the image quality index of the depth image is used to describe the image quality of the depth image; the point cloud weight coefficient is determined according to the second confidence degree. The first confidence degree and the semantic weight coefficient have a positive correlation relationship, and the second confidence degree and the point cloud weight coefficient have a positive correlation relationship.

[0060] ​The manner of acquiring the image quality index can include, but is not limited to, algorithms such as full reference image quality assessment (FR-IQA), reduced reference image quality assessment (RR-IQA) and no reference image quality assessment (NR-IQA), without limitation.

[0061] For example, the first mapping relationship table and the second mapping relationship table are acquired. The first mapping relationship table is queried to acquire the semantic weight coefficient corresponding to the first confidence, and the second mapping relationship table is queried to acquire the point cloud weight coefficient corresponding to the second confidence. The first mapping relationship table includes a mapping relationship between the confidence and the semantic weight coefficient, and the second mapping relationship table includes a mapping relationship between the confidence and the point cloud weight coefficient.

[0062] In an embodiment, the semantic weight coefficient and the point cloud weight coefficient are negatively correlated, that is, the greater the semantic weight coefficient, the smaller the point cloud weight coefficient, and the smaller the semantic weight coefficient, the greater the point cloud weight coefficient. For example, in the case that the accuracy of the semantic label probability of the pixel point is high, and the accuracy of the point cloud probability of the pixel point is low, the semantic weight coefficient is increased, and the point cloud weight coefficient is decreased. In the case that the accuracy of the semantic label probability of the pixel point is low, and the accuracy of the point cloud probability of the pixel point is high, the semantic weight coefficient is decreased, and the point cloud weight coefficient is increased.

[0063] In an embodiment, the semantic label probability, the distance probability and the semantic weight coefficient are multiplied to obtain a first result after multiplication; the point cloud probability, the distance probability and the point cloud weight coefficient are multiplied to obtain a second result after multiplication; and the first result and the second result are added to obtain a second probability. For example, assuming that the point cloud probability is P1, the point cloud weight coefficient is k1, the distance probability is P2, the semantic label probability is P3, the semantic weight coefficient is k2, and P is the second probability, then P=k1×P1×P2+k2×P2×P3.

[0064] In step S103, the second probability and the position information of each pixel point are used to determine the voxel corresponding to each pixel point, and a three-dimensional probability map at the current time is generated based on the voxel.

[0065] In the embodiment, the three-dimensional probability map is a three-dimensional grid map, and in the three-dimensional probability map at the current time, the probability value carried by each voxel is the second probability of the pixel point corresponding to the voxel. Based on the position information of the pixel point, the corresponding voxel of the pixel point in the three-dimensional grid map can be determined, and the second probability value of the pixel point is configured as the probability value carried by the voxel.

[0066] In an embodiment, the position information of the pixel point comprises three-dimensional position information of the pixel point in camera coordinates, and the manner of determining the voxel corresponding to the pixel point according to the second probability of the pixel point and the position information can be: obtaining an intrinsic matrix and an extrinsic matrix of the image acquisition device, and converting the three-dimensional position information of the pixel point in camera coordinates into three-dimensional position information in a world coordinate system according to the intrinsic matrix and the extrinsic matrix; generating the voxel corresponding to the pixel point according to the converted three-dimensional position information and an octree-based three-dimensional map creation tool, that is, the three-dimensional probability map of the present application is a three-dimensional grid map composed of voxels. The position of the voxel in the three-dimensional probability map is represented by a voxel grid index, which is calculated according to the length, width and resolution ratio of the three-dimensional probability map. The probability value of the voxel is configured as the second probability of the pixel point, and the voxel grid stores the second probability of the corresponding pixel point.

[0067] In step S104, the three-dimensional probability map of the previous moment is obtained, and the probability value of each voxel in the three-dimensional probability map of the previous moment is updated according to a preset adjustment parameter.

[0068] In the embodiment of the present application, the three-dimensional probability map of the previous moment and the three-dimensional probability map of the current moment are constructed in the same manner, and the preset adjustment parameter can be set by the user based on the actual situation, which is not limited in the embodiment of the present application.

[0069] In an embodiment, the manner of updating the probability value of each voxel in the three-dimensional probability map of the previous moment according to the preset adjustment parameter can be: multiplying the probability value of each voxel in the three-dimensional probability map of the previous moment by the preset adjustment parameter, and updating the probability value of each voxel in the three-dimensional probability map of the previous moment to the probability value of each voxel obtained by multiplication. For example, assuming that the preset adjustment parameter is A, and the probability value of a voxel B in the three-dimensional probability map of the previous moment is p1, then p2=Axp1, and therefore the probability value of the voxel B is updated to p2.

[0070] In an embodiment, the length of stay of each voxel in the three-dimensional probability map of the previous moment is obtained, which represents the length of time from the generation time of the voxel to the current time; the probability value of each voxel in the three-dimensional probability map of the previous moment is multiplied by the preset adjustment parameter; the probability value of each voxel obtained by multiplication is divided by the length of stay of each corresponding voxel, and the result obtained by division is taken as the updated probability value of each voxel. For example, the preset adjustment parameter is A, the probability value of a voxel B in the three-dimensional probability map of the previous moment is p1, the generation time of the voxel is t1, and the current time is t2, so the updated probability value of the voxel B can be represented as p3=(p1xA) / (t2-t1).

[0071] In step S105, the probability value of each voxel in the three-dimensional probability map at the current time is multiplied by the probability value of the corresponding voxel in the updated three-dimensional probability map at the previous time, and the probability value of each voxel in the three-dimensional probability map at the current time is updated according to the multiplication result.

[0072] For example, the three-dimensional probability map at the current time includes voxels C1, C2, C3, C4, C5 and C6, and the probability values of the voxels C1, C2, C3, C4, C5 and C6 are P C1 , P C2 , P C3 , P C4 , P C5 and P C6 , respectively. The updated three-dimensional probability map at the previous time includes voxels D1, D2, D3, D4, D5 and D6, and the probability values of the voxels D1, D2, D3, D4, D5 and D6 are P D1 , P D2 , P D3 , P D4 , P D5 and P D6 , respectively. Since the voxel C1 corresponds to the voxel D1, the voxel C2 corresponds to the voxel D2, the voxel C3 corresponds to the voxel D3, the voxel C4 corresponds to the voxel D4, the voxel C5 corresponds to the voxel D5, and the voxel C6 corresponds to the voxel D6, the probability values of the voxels C1, C2, C3, C4, C5 and C6 in the updated three-dimensional probability map at the current time are P C1 ×P D1 , P C2 ×P D2 , P C3 ×P D3 , P C4 ×P D4 , P C5 ×P D5 and P C6 ×P D6 , respectively.

[0073] In an embodiment, in the case that there is a target voxel in the three-dimensional probability map at the current time, the probability value of the target voxel is multiplied by a preset probability value, and the probability value of the target voxel in the three-dimensional probability map at the current time is updated according to the multiplication result. The target voxel and the voxel in the updated three-dimensional probability map at the previous time are not corresponding to each other. The preset probability value can be set based on actual conditions, which is not limited in the embodiments of the present application. For example, the preset probability value is 1 or 0.85.

[0074] For example, assuming that the preset probability value is P', the three-dimensional probability map at the current time includes voxels C1, C2, C3, C4, C5 and C6, and the updated three-dimensional probability map at the previous time includes voxels D1, D2, D3 and D4, voxel C1 corresponds to voxel D1, voxel C2 corresponds to voxel D2, voxel C3 corresponds to voxel D3, and voxel C4 corresponds to voxel D4. The updated three-dimensional probability map at the previous time does not include voxels corresponding to voxels C5 and C6. Therefore, the probability values of voxels C1, C2, C3, C4, C5 and C6 in the updated three-dimensional probability map at the current time are P C1 ×P D1 , P C2 ×P D2 , P C3 ×P D3 , P C4 ×P D4 , P C5 ×P' and P C6 ×P', respectively.

[0075] In an embodiment, the distance between each voxel in the three-dimensional probability map at the current time and the mobile device is obtained. When the distance between any voxel and the mobile device is greater than a preset distance threshold, the voxel corresponding to the distance greater than the preset distance threshold is deleted from the three-dimensional probability map at the current time to obtain an updated three-dimensional probability map at the current time. The probability value of each voxel in the updated three-dimensional probability map at the current time is multiplied by the probability value of the corresponding voxel in the updated three-dimensional probability map at the previous time, and the probability value of each voxel in the three-dimensional probability map at the current time is updated according to the multiplication result. By deleting the voxel corresponding to the distance greater than the preset distance threshold in the three-dimensional probability map at the current time, the size of the three-dimensional probability map can be reduced, the three-dimensional probability map can be prevented from being too large and wasting storage space, and the utilization rate of the storage space can be improved.

[0076] The distance between each voxel in the three-dimensional probability map at the current time and the mobile device can be obtained by converting the distance between the pixel corresponding to each voxel in the three-dimensional probability map at the current time and the image acquisition device. The preset distance threshold can be set by the user according to the actual situation, and the embodiments of the present application do not make specific limitations thereto.

[0077] In step S106, the updated three-dimensional probability map at the current time is taken as the target three-dimensional probability map at the current time.

[0078] By taking the updated three-dimensional probability map of the current time as the target three-dimensional probability map of the current time, when determining the three-dimensional probability map of the next time, the three-dimensional probability map can be established and updated in the same way, so that the three-dimensional probability map can change with the movement of the self-moving device, and the self-moving device can know the change of the obstacle based on the changed three-dimensional probability map, thereby safely avoiding the obstacle.

[0079] The embodiment of the present application provides a map construction method and device and a storage medium. The second probability of each pixel point belonging to an obstacle is determined according to the first probability of each pixel point belonging to an obstacle and the distance of each pixel point belonging to an obstacle to an image acquisition device. Since the second probability is related to the distance of the pixel point to the image acquisition device, the current three-dimensional probability map capable of representing the position change of the obstacle can be generated by using the second probability of each pixel point belonging to an obstacle and the position information of each pixel point belonging to an obstacle. After the probability value of each voxel in the three-dimensional probability map of the previous time is updated, the probability value of each voxel in the three-dimensional probability map of the current time is multiplied, so that the three-dimensional probability map of the current time can be updated according to the multiplication result. The updated three-dimensional probability map can more accurately describe the change of the obstacle. When the self-moving device avoids the obstacle, the change of the obstacle can be observed by using the three-dimensional probability map, so that the self-moving device can accurately avoid the obstacle, and the risk of collision between the self-moving device and the obstacle is reduced.

[0080] Please refer to Figure 7 , Figure 7 is a structural schematic block diagram of a map construction device provided by the embodiment of the present application.

[0081] As Figure 7 shown, the map construction device 200 comprises a processor 201 and a memory 202, and the processor 201 and the memory 202 are connected through a bus 203, such as an I2C (Inter-integrated Circuit) bus.

[0082] Specifically, the processor 201 is configured to provide computing and control capabilities to support the operation of the entire map construction device. The processor 201 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0083] Specifically, the memory 202 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a U disk or a mobile hard disk, etc.

[0084] Those skilled in the art can understand that, Figure 7 The structure shown in the figure is only a block diagram of part of the structure related to the embodiment of the present application, and does not constitute a limitation on the map construction device to which the embodiment of the present application is applied. Specifically, the map construction device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0085] The processor 201 is configured to run a computer program stored in the memory 202, and implement any one of the map construction methods provided by the embodiments of the present application when the computer program is executed.

[0086] In an embodiment, the processor 201 is configured to run a computer program stored in the memory 202, and implement any one of the map construction methods provided by the embodiments of the present application when the computer program is executed.

[0087] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the map construction device described above can refer to the corresponding process in the foregoing map construction method embodiments, which will not be described here.

[0088] The embodiments of the present application also provide a storage medium for computer readable storage, the storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement any one of the map construction methods provided by the embodiments of the present application.

[0089] The storage medium can be an internal storage unit of the map construction device, such as a hard disk or a memory of the map construction device. The storage medium can also be an external storage device of the map construction device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like.

[0090] Those of ordinary skill in the art will understand that all or some of the steps in the above disclosed methods, the functional modules / units in the above disclosed systems and devices can be implemented by software, firmware, hardware, or any appropriate combination thereof. In hardware implementations, the division between the functional modules / units referred to in the above description does not necessarily correspond to the division between physical components; for example, one physical component can have multiple functions, or one function or step can be performed by several physical components working in cooperation. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on computer readable media, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those of ordinary skill in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Further, it should be understood by those of ordinary skill in the art that communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The above disclosed methods, systems and devices can be implemented in a variety of ways. Below, some example implementations are described.

[0091] It should be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items, and that the term "comprises" or "comprising" or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0092] The above-mentioned embodiment serial numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A map construction method, comprising: Obtaining obstacle information from image data at a current moment, the image data being acquired by an image acquisition device on a mobile device, the obstacle information including a distance from each pixel belonging to the obstacle to the image acquisition device, position information of each pixel, and a first probability that each pixel belongs to an obstacle; Obtaining a second probability that each pixel point is an obstacle based on the first probability and the distance of each pixel point; Determining a voxel corresponding to each pixel point based on the second probability and position information of each pixel point, and generating a three-dimensional probability map at a current moment based on the voxels, wherein a probability value carried by each voxel in the three-dimensional probability map at the current moment is the second probability; Obtaining a three-dimensional probability map at a previous moment, and updating a probability value of each voxel in the three-dimensional probability map at a previous moment according to a preset adjustment parameter; multiplying the probability value of each voxel in the three-dimensional probability map at the current moment by the probability value of the corresponding voxel in the updated three-dimensional probability map at the previous moment, and updating the probability value of each voxel in the three-dimensional probability map at the current moment according to the multiplication result; The updated three-dimensional probability map at the current moment is used as the target three-dimensional probability map at the current moment.

2. The map construction method according to claim 1, wherein: The obtaining of obstacle information in the image data at the current moment includes: When the image data is an RGB image, semantic recognition is performed on each pixel in the RGB image to obtain a semantic recognition result, wherein the semantic recognition result includes a semantic label and a semantic label probability for each pixel, wherein the semantic label describes the type of the pixel, and the semantic label probability describes the probability that the pixel belongs to the type corresponding to the semantic label; A pixel whose semantic label is an obstacle label is obtained, and the obtained semantic label probability of the pixel is used as the first probability, where the obstacle label is used to describe the type of the pixel as an obstacle.

3. The map construction method according to claim 1, wherein: The obtaining of obstacle information in the image data at the current moment includes: When the image data is a depth image, converting the depth image to obtain point cloud data; A point cloud belonging to an obstacle and a point cloud probability of each point in the point cloud are extracted from the point cloud data, and the point cloud probability is used as the first probability, wherein the point cloud probability represents the probability that a point in the point cloud belongs to an obstacle.

4. The map construction method according to claim 2 or 3, wherein: Obtaining a second probability that each pixel point belongs to an obstacle according to the first probability and the distance of each pixel point includes: Determining a distance probability of each pixel point based on the distance from each pixel point to the image acquisition device, wherein the distance probability represents a probability that the pixel point is an obstacle during a distance change process, and the distance probability is inversely proportional to the distance; The first probability and the distance probability are multiplied to obtain a second probability that each pixel point belongs to an obstacle.

5. The map construction method according to claim 1, wherein: The image data includes an RGB image and a depth image, the first probability includes a semantic label probability of each pixel in the RGB image belonging to an obstacle and a point cloud probability of each pixel in the depth image belonging to an obstacle, and obtaining a second probability that each pixel belongs to an obstacle based on the first probability and the distance of each pixel includes: Determining a distance probability of each pixel point according to the distance between each pixel point and the image acquisition device; Get semantic weight coefficient and point cloud weight coefficient; A second probability is determined according to the semantic label probability of the pixel point, the point cloud probability, the distance probability, the semantic weight coefficient, and the point cloud weight coefficient.

6. The map construction method according to claim 5, wherein: The obtaining of the semantic weight coefficient and the point cloud weight coefficient includes: Obtaining an image quality index of the RGB image, and determining the image quality index of the RGB image as a first confidence level of a semantic label probability of a pixel point; wherein the image quality index of the RGB image is used to describe the image quality of the RGB image; Determining a semantic weight coefficient according to the first confidence level; Obtaining an image quality index of the depth image, and determining the image quality index of the depth image as a second confidence level of the point cloud probability of the pixel point; wherein the image quality index of the depth image is used to describe the image quality of the depth image; A point cloud weight coefficient is determined according to the second confidence level.

7. The map construction method according to claim 5, wherein: The determining the second probability according to the semantic label probability of the pixel point, the point cloud probability, the distance probability, the semantic weight, and the point cloud weight coefficient includes: Multiplying the semantic label probability, the distance probability, and the semantic weight coefficient to obtain a first multiplied result; Multiplying the point cloud probability, the distance probability, and the point cloud weight coefficient to obtain a second multiplied result; The first result and the second result are added to obtain the second probability.

8. The map construction method according to any one of claims 1 to 7, wherein: Before multiplying the probability value of each voxel in the three-dimensional probability map at the current moment by the probability value of the corresponding voxel in the updated three-dimensional probability map at the previous moment, the method further includes: Obtaining the distance between each voxel in the three-dimensional probability map at the current moment and the self-moving device; When the distance between any voxel and the mobile device is greater than a preset distance threshold, the voxel is deleted from the three-dimensional probability map at the current moment to obtain an updated three-dimensional probability map at the current moment.

9. The map construction method according to claim 1, wherein: The updating of the probability value of each voxel in the three-dimensional probability map at the previous moment according to the preset adjustment parameter includes: Obtaining the residence time of each voxel in the three-dimensional probability map at the previous moment, where the residence time represents the time from the moment the voxel was generated to the current moment; Multiplying the probability value of each voxel in the three-dimensional probability map at the previous moment by a preset adjustment parameter; The probability value of each voxel obtained by multiplication is divided by the corresponding residence time of each voxel, and the result of the division is used as the updated probability value of each voxel.

10. A map construction device, comprising a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for implementing communication between the processor and the memory, wherein when the computer program is executed by the processor, the map construction method according to any one of claims 1 to 9 is implemented.

11. A storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the map construction method according to any one of claims 1 to 9.

Images