Updating method and device for noise-filtered image, equipment and moving tool
By detecting and filtering the point cloud data, the memory obstacle set is updated using the noise filtering image of the previous frame and the noise filtering image, the problem of untimely filtering of single-line laser noise is solved and the noise filtering is improved accuracy and stability of robot operation.
Patent Information
- Application Number
- CN202410747451.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-11
- Publication Date
- 2024-11-26
AI Technical Summary
In the prior art, the noise filtering method generated by single-line laser shelling has a high labor cost and the risk of obstacles in the shelling area being unable to detect, and the noise filtering image is not updated in time to affect the accuracy of noise filtering.
By detecting the collected point cloud data, the noise barrier is filtered out using the noise filtering image of the previous frame, the memory obstacle set is updated, and the noise filtering image is updated according to the obstacle elements, ensuring that it is used every time it is filtered out Latest images.
It improves the accuracy of noise filtering and the efficient, intelligent and smooth operation of intelligent robots, ensuring that the robot can drive normally after collision or structural deformation.
Smart Images

Figure CN119027337A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of autonomous driving technology, and in particular to a method, device, equipment and mobile tool for updating a noise-filtered image. Background Art
[0002] In recent years, with the maturity of artificial intelligence technology, robots assisting humans or even replacing humans in some repetitive, dirty, tiring and unsafe work has become a demand for social development. In order to meet this demand, a variety of intelligent service robots have emerged in society, such as cleaning robots, logistics robots, hotel service robots, etc. A good perception sensor plays a vital role in robot obstacle avoidance. For cost and performance considerations, single-line laser has become a common sensor for robots to perceive the surrounding environment. However, once a robot collides or structural deformation occurs during use, the single-line laser will have the problem of shelling. Shelling will cause noise in the single-line laser, causing perception misdetection, which will further cause the machine to run unsmoothly or even stop.
[0003] In the prior art, the method for eliminating the noise generated by single-line laser shell punching is to manually adjust the position of the single-line laser or directly shield the shell punching area. However, these methods have certain disadvantages. Either a large amount of labor costs are required, or there is a risk that the robot cannot detect obstacles in the shell punching area. Based on this, a noise filtering image can be used to filter out the noise generated by shell punching. However, the noise generated during driving is not fixed. If the noise filtering image is not updated in time during driving, it will still affect the accuracy of noise filtering.
[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention
[0005] The present application provides a method, device, equipment and mobile tool for updating a noise-filtered image to solve the technical problem of untimely updating of the noise-filtered image.
[0006] According to one aspect of an embodiment of the present application, the present application provides a method for updating a noise-filtered image, including: performing obstacle detection on the collected point cloud data to obtain a candidate noise obstacle set; filtering out known noise obstacles in the candidate noise obstacle set through a first noise-filtered image to obtain an unfiltered noise obstacle set, wherein the first noise-filtered image is obtained by updating a candidate noise obstacle set of a previous frame of point cloud data after noise obstacles are filtered out; using the unfiltered noise obstacle set to update a first memory obstacle set to obtain a second memory obstacle set; using obstacle elements in the second memory obstacle set to update the first noise-filtered image to obtain a second noise-filtered image, wherein the second noise-filtered image is used to filter out noise obstacles from a candidate noise obstacle set of a next frame.
[0007] Optionally, obstacle detection is performed on the collected point cloud data to obtain a candidate noise point obstacle set, specifically including: acquiring point cloud data in the vehicle coordinate system, and fitting the point cloud data with a ground plane; performing background segmentation according to the fitted ground plane to obtain foreground point cloud data; performing obstacle detection according to the foreground point cloud data to obtain an initial obstacle set; extracting candidate noise point obstacles that meet preset conditions from the initial obstacle set to obtain a candidate noise point obstacle set.
[0008] Optionally, extracting candidate noise obstacles that meet preset conditions from the initial obstacle set to obtain the candidate noise obstacle set includes: selecting a first obstacle whose obstacle width is less than a width threshold and whose obstacle height is less than a height threshold from the initial obstacle set to obtain a first initial obstacle set; selecting a second obstacle whose number of sub-elements is less than a number threshold from the first initial obstacle set to obtain a second initial obstacle set; selecting a third obstacle that can only be detected by a target sensor from the second initial obstacle set to obtain a third initial obstacle set; and selecting candidate noise obstacles from the third initial obstacle set to obtain a candidate noise obstacle set, wherein the sub-elements of the candidate noise obstacles are all located within the target area under the ego-vehicle coordinates.
[0009] Optionally, known noise obstacles in the candidate noise obstacle set are filtered out by using the first noise-filtered image to obtain an unfiltered noise obstacle set, specifically including: acquiring the first noise-filtered image and generating a filtered image coordinate system corresponding to the first noise-filtered image; converting the first sub-element of each candidate noise obstacle in the candidate noise obstacle set to the filtered image coordinate system to obtain a first pixel value corresponding to the first sub-element; comparing the first pixel value with a preset pixel value; if the first pixel value of any first sub-element in the candidate noise obstacle is equal to the preset pixel value, then the candidate noise obstacle is determined to be a known noise obstacle; if the first pixel values of the first sub-elements in the candidate noise obstacle are not equal to the preset pixel value, then the candidate noise obstacle is determined to be an unfiltered noise obstacle; filtering out the known noise obstacles in the candidate noise obstacle set to obtain an unfiltered noise obstacle set including unfiltered noise obstacles.
[0010] Optionally, the first memory obstacle set is updated using the unfiltered noise obstacle set to obtain the second memory obstacle set, specifically including: generating a memory image corresponding to each unfiltered noise obstacle in the unfiltered noise obstacle set; matching the first memory obstacle in the first memory obstacle set with the unfiltered noise obstacle according to the memory image to obtain update data; updating the first memory obstacle according to the update data to obtain the second memory obstacle set, wherein the update data includes the newly added memory obstacles and the number of occurrences of each first memory obstacle.
[0011] Optionally, matching the first memory obstacles in the first memory obstacle set with the unfiltered noise obstacles according to the memory image specifically includes: converting the second sub-element of each first memory obstacle to the memory image coordinate system corresponding to the memory image to obtain the second pixel value of the second sub-element; comparing the second pixel value with the preset pixel value; if the second pixel values of all the second sub-elements in each first memory obstacle are not equal to the preset pixel value, then it is determined that the first memory obstacle and the unfiltered noise obstacle are not the same object, and the unfiltered noise obstacle is determined as a newly added memory obstacle; if the second pixel value of any second sub-element in the first memory obstacle is equal to the preset pixel value, then it is determined that the first memory obstacle and the unfiltered noise obstacle are the same object, and it is determined whether the first memory obstacle and the unfiltered noise obstacle are the same object following the movement of the ego vehicle according to the ego vehicle posture data; if the first memory obstacle and the unfiltered noise obstacle are the same object following the movement of the ego vehicle, the number of occurrences of the first memory obstacle is increased by 1.
[0012] Optionally, the first noise-filtered image is updated using obstacle elements in the second memory obstacle set to obtain the second noise-filtered image, specifically including: obtaining the number of occurrences of each second memory obstacle in the second memory obstacle set; and updating the second memory obstacles whose number of occurrences is greater than a preset number into the first noise-filtered image to obtain the second noise-filtered image.
[0013] According to another aspect of an embodiment of the present application, the present application provides a device for updating a noise-filtered image, including: a detection module, configured to perform obstacle detection on collected point cloud data to obtain a candidate noise obstacle set; a filtering module, configured to filter known noise obstacles in the candidate noise obstacle set through a first noise-filtered image to obtain an unfiltered noise obstacle set, wherein the first noise-filtered image is obtained by updating a candidate noise obstacle set of a previous frame of point cloud data after noise obstacles are filtered out; a first updating module, configured to update a first memory obstacle set using the unfiltered noise obstacle set to obtain a second memory obstacle set; and a second updating module, configured to update the first noise-filtered image using obstacle elements in the second memory obstacle set to obtain a second noise-filtered image, wherein the second noise-filtered image is used to perform noise obstacle filtering on a candidate noise obstacle set of a next frame.
[0014] According to another aspect of an embodiment of the present application, the present application provides an electronic device, including a memory, a processor, a communication interface and a communication bus, wherein the memory stores a computer program that can be run on the processor, the memory and the processor communicate through the communication bus and the communication interface, and the steps of the above method are implemented when the processor executes the computer program.
[0015] According to another aspect of an embodiment of the present application, the present application provides a mobile tool, including the above-mentioned electronic device.
[0016] According to another aspect of an embodiment of the present application, the present application also provides a computer-readable medium having a non-volatile program code executable by a processor, and the program code enables the processor to execute the above method.
[0017] The above technical solution provided by the embodiment of the present application has the following advantages compared with the related art:
[0018] The present application provides a method for updating a noise-filtered image, including: performing obstacle detection on the collected point cloud data to obtain a candidate noise obstacle set; filtering out known noise obstacles in the candidate noise obstacle set through a first noise-filtered image to obtain an unfiltered noise obstacle set, wherein the first noise-filtered image is obtained by updating the candidate noise obstacle set of the previous frame of point cloud data after noise obstacles are filtered out; updating the first memory obstacle set using the unfiltered noise obstacle set to obtain a second memory obstacle set; updating the first noise-filtered image using obstacle elements in the second memory obstacle set to obtain a second noise-filtered image, wherein the second noise-filtered image is used to filter out noise obstacles for the candidate noise obstacle set of the next frame. After filtering out noise using the noise-filtered image of the previous frame, the noise-filtered image is updated according to the filtering result, so that the updated noise-filtered image can be applied to the noise filtering of the next frame of data, and the latest noise-filtered image can be used each time the noise is filtered out, thereby solving the problem of untimely updating of the noise-filtered image. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0021] Figure 1 A flowchart of an optional method for updating a noise-filtered image provided according to an embodiment of the present application;
[0022] Figure 2 A block diagram of an optional device for updating a noise-filtered image according to an embodiment of the present application;
[0023] Figure 3 A schematic diagram of an optional electronic device structure provided in an embodiment of the present application. DETAILED DESCRIPTION
[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0025] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application and have no specific meaning. Therefore, "module" and "component" can be used interchangeably.
[0026] In recent years, with the maturity of artificial intelligence technology, robots assisting humans or even replacing humans in some repetitive, dirty, tiring and unsafe work has become a demand for social development. In order to meet this demand, a variety of intelligent service robots have emerged in society, such as cleaning robots, logistics robots, hotel service robots, etc. A good perception sensor plays a vital role in robot obstacle avoidance. For cost and performance considerations, single-line laser has become a common sensor for robots to perceive the surrounding environment. However, once a robot collides or structural deformation occurs during use, the single-line laser will have the problem of shelling. Shelling will cause noise in the single-line laser, causing perception misdetection, which will further cause the machine to run unsmoothly or even stop.
[0027] In the prior art, the method for eliminating the noise generated by single-line laser shell punching is to manually adjust the position of the single-line laser or directly shield the shell punching area. However, these methods have certain disadvantages. Either a large amount of labor costs are required, or there is a risk that the robot cannot detect obstacles in the shell punching area. Based on this, a noise filtering image can be used to filter out the noise generated by shell punching. However, the noise generated during driving is not fixed. If the noise filtering image is not updated in time during driving, it will still affect the accuracy of noise filtering.
[0028] In order to solve the problem mentioned in the background technology, according to one aspect of the embodiments of the present application, an embodiment of a method for updating a noise-filtered image is provided, such as Figure 1 As shown, including:
[0029] Step 101, performing obstacle detection on the collected point cloud data to obtain a candidate noise obstacle set;
[0030] Step 103, filtering out known noise obstacles in the candidate noise obstacle set through the first noise-filtered image to obtain an unfiltered noise obstacle set, wherein the first noise-filtered image is obtained by updating the candidate noise obstacle set of the previous frame of point cloud data after filtering out noise obstacles;
[0031] Step 105, using the unfiltered noise obstacle set to update the first memory obstacle set, to obtain a second memory obstacle set;
[0032] Step 107: Update the first noise-filtered image using the obstacle elements in the second memory obstacle set to obtain a second noise-filtered image, wherein the second noise-filtered image is used to filter out noise obstacles from the next frame of candidate noise obstacle set.
[0033] In order to ensure the accuracy of noise filtering, the present application provides a method for updating a noise-filtered image, which uses a first noise-filtered image obtained by updating a previous frame of data after noise filtering to perform noise filtering on noise obstacles of a current frame, and then updates the first noise-filtered image according to the filtered unfiltered noise obstacles, and then uses the updated second noise-filtered image to perform noise filtering on the next frame of data, which can ensure that the noise-filtered image used each time is the latest, and can greatly improve the accuracy of noise filtering.
[0034] This application proposes a method for updating a noise-filtered image, which can not only filter out single-line laser noise obstacles but also detect normal obstacles. It can enable the intelligent robot to run normally after a collision or when the shell structure is deformed, thereby increasing the efficiency, intelligence and fluency of the intelligent robot's operation.
[0035] As an optional embodiment, obstacle detection is performed on the collected point cloud data to obtain a candidate noise point obstacle set, which specifically includes: acquiring point cloud data in the vehicle coordinate system, and fitting the point cloud data with a ground plane; performing background segmentation according to the fitted ground plane to obtain foreground point cloud data; performing obstacle detection according to the foreground point cloud data to obtain an initial obstacle set; and extracting candidate noise point obstacles that meet preset conditions from the initial obstacle set to obtain a candidate noise point obstacle set.
[0036] The point cloud data of the present application may be obtained from detection data of one sensor, or may be obtained by integrating detection data of multiple sensors, and this is not limited.
[0037] For example, after the sensor obtains the detection data, the point cloud in the sensor coordinate system is converted to the vehicle coordinate system according to the positional relationship between the calibrated sensor coordinate system and the robot coordinate system. Since the point cloud is structural, the point cloud in the i-th row and j-th column of the robot coordinate system can be recorded as P r(i,j) (X r(i,j) , Y r(i,j) , Z r(i,j) ), where X r(i,j) represents the X coordinate value of the point cloud at the i-th row and j-th column in the robot coordinate system, and Y r(i,j)represents the Y coordinate value of the point cloud at the i-th row and j-th column in the robot coordinate system, and Z r(i,j) Represents the Z coordinate value of the point cloud at the i-th row and j-th column in the robot coordinate system.
[0038] Specifically, after obtaining the point cloud data in the vehicle coordinate system, obstacle detection is performed on the point cloud data in the following manner: the point cloud data is screened for ground plane fitting; the foreground and background are segmented according to the fitted ground plane (the background refers to the points located on the ground plane, and the foreground refers to the points not on the ground plane); obstacle detection is performed on the foreground point cloud data; misdetected obstacles are eliminated, and the initial obstacle set is output.
[0039] As an optional embodiment, extracting candidate noise obstacles that meet preset conditions from the initial obstacle set to obtain the candidate noise obstacle set includes: selecting a first obstacle whose obstacle width is less than a width threshold and whose obstacle height is less than a height threshold from the initial obstacle set to obtain a first initial obstacle set; selecting a second obstacle whose sub-element quantity is less than a quantity threshold from the first initial obstacle set to obtain a second initial obstacle set; selecting a third obstacle that can only be detected by a target sensor from the second initial obstacle set to obtain a third initial obstacle set; selecting candidate noise obstacles from the third initial obstacle set to obtain a candidate noise obstacle set, wherein the sub-elements of the candidate noise obstacles are all located within the target area under the ego-vehicle coordinates.
[0040] Optionally, the width threshold, height threshold and quantity threshold can be set according to actual conditions, and this application does not limit the specific values. The target area range can also be set according to the actual obstacle detection range.
[0041] The target sensor provided in this application is a single-line laser sensor.
[0042] For example, the preset condition is that the initial obstacle needs to satisfy four constraints at the same time, namely: the width of the obstacle element is less than the obstacle width threshold and the height is less than the obstacle height threshold; the number of cell elements (corresponding to the sub-elements of this application) contained in the obstacle element is less than the cell number threshold; the sub-elements in the obstacle element can only be detected by a single-line laser sensor; all cell elements of the obstacle element are within a certain distance range centered on the vehicle.
[0043] For example, traverse the obstacle elements in the initial obstacle set. Denote objs(i) as the i-th element in the target obstacle set. If objs(i) simultaneously satisfies the following four constraints: objs(i).width < WIDTH_THRESH and objs(i).height < HEIGHT_THRESH; objs(i).cells_size < CELLS_SIZE_THRESH; all cell elements of objs(i) are detected only by the single-line laser sensor; all cell elements of objs(i) are within a certain distance range centered on the host vehicle; then the objs(i) belongs to the elements of the noise obstacle set. All obstacle elements in the initial obstacle set that simultaneously satisfy the 4 constraints form the candidate noise obstacle set, denoted as noise_objs. Among them, objs(i).width refers to the width of objs(i), objs(i).height refers to the height of objs(i), objs(i).cells_size refers to the number of cell elements included in objs(i), WIDTH_THRESH refers to the obstacle width threshold, which is 0.1m in this application, HEIGHT_THRESH refers to the obstacle height threshold, which is 0.1m in this application; CELLS_SIZE_THRESH refers to the threshold of the number of cells included in the obstacle, which is 10 in this application.
[0044] As an optional embodiment, filter the known noise obstacles in the candidate noise obstacle set through the first noise-filtered image to obtain the unfiltered noise obstacle set, which specifically includes: obtaining the first noise-filtered image and generating a filtered image coordinate system corresponding to the first noise-filtered image; converting the first sub-elements of each candidate noise obstacle in the candidate noise obstacle set to the filtered image coordinate system to obtain the first pixel value corresponding to the first sub-element; comparing the first pixel value with the preset pixel value; if the first pixel value of any first sub-element in the candidate noise obstacle is equal to the preset pixel value, then determine that the candidate noise obstacle is a known noise obstacle, if the first pixel values of the first sub-elements in the candidate noise obstacle are not equal to the preset pixel value, then determine that the candidate noise obstacle is an unfiltered noise obstacle; filter the known noise obstacles in the candidate noise obstacle set to obtain the unfiltered noise obstacle set including unfiltered noise obstacles.
[0045] The preset pixel value provided in this application is 255.
[0046] An image has a default image coordinate system, usually with the upper left corner as the coordinate origin, the right direction as the X direction, and the down direction as the Y direction.
[0047] For example, noise_objs(i) is denoted as the i-th element in the candidate noise obstacle set, noise_objs_cells(i, j) is denoted as the j-th cell element in noise_objs(i), and noise_objs_cells(i, j) is filtered out in the following manner: noise_objs_cells(i, j) is converted to the filtered image coordinate system, and the corresponding filtered image pixel is denoted as img_filter(u(i, j), v(i, j)); the value of img_filter(u(i, j), v(i, j)) is determined. j), v(i, j)) are 255: If the value of img_filter(u(i, j), v(i, j)) is 255, it means that noise_objs_cells(i, j) needs to be filtered out; otherwise, the next cell element is detected; all cell elements in noise_objs(i) are traversed to perform the above filtering detection. As long as there is a cell element in noise_objs(i) that needs to be filtered out, the noise_objs(i) obstacle needs to be filtered out (that is, the element is deleted from the noise_objs set). Among them, u(i, j) and v(i, j) are the corresponding horizontal and vertical coordinates of the jth cell element of the i-th element in the noise obstacle set when it is converted to the filtered image, and img_filter(u(i, j), v(i, j)) is the pixel value with horizontal coordinate u(i, j) and vertical coordinate v(i, j) in the filtered image.
[0048] Traverse all obstacle elements in the candidate noise obstacle set, and perform the above-mentioned filtering detection operation on each element; all obstacle elements in the candidate noise obstacle set are filtered out, and the remaining obstacle elements after image filtering form a new set, which is recorded as the unfiltered noise obstacle set.
[0049] As an optional embodiment, the first memory obstacle set is updated using the unfiltered noise obstacle set to obtain the second memory obstacle set, specifically including: generating a memory image corresponding to each unfiltered noise obstacle in the unfiltered noise obstacle set; matching the first memory obstacle in the first memory obstacle set with the unfiltered noise obstacle according to the memory image to obtain update data; updating the first memory obstacle according to the update data to obtain the second memory obstacle set, wherein the update data includes the newly added memory obstacles and the number of occurrences of each first memory obstacle.
[0050] For example, clear the pixel values of the memory image, that is, set all pixel values of the memory image to 0, record unfilt_objs(i) as the i-th element in the unfiltered noise obstacle set, record unfilt_objs_cells(i, j) as the j-th cell element in unfilt_objs(i), and perform the following imaging operation on unfilt_objs_cells(i, j): convert unfilt_objs_cells(i, j) to the memory image coordinate system, and record the corresponding pixels as img_mem(u(i, j), v(i, j)), and perform an expansion operation on the memory image with img_mem(u(i, j), v(i, j)) as the center and an R*R rectangular size as the window, that is, all pixel values in the window are set to 255; traverse all cell elements in unfilt_objs(i) to perform the above imaging operation to obtain the memory image corresponding to unfilt_objs(i). Among them, u(i, j) and v(i, j) are the horizontal and vertical coordinates corresponding to the j-th cell of the i-th element in the unfiltered noise obstacle set when it is converted to the memory image, respectively, and img_mem(u(i, j), v(i, j)) is the pixel value with horizontal coordinate u(i, j) and vertical coordinate v(i, j) in the memory image. In this application, the value of R is 5.
[0051] As an optional embodiment, matching the first memory obstacles in the first memory obstacle set with the unfiltered noise obstacles according to the memory image specifically includes: converting the second sub-element of each first memory obstacle to the memory image coordinate system corresponding to the memory image to obtain the second pixel value of the second sub-element; comparing the second pixel value with the preset pixel value; if the second pixel values of all the second sub-elements in each first memory obstacle are not equal to the preset pixel value, it is determined that the first memory obstacle and the unfiltered noise obstacle are not the same object, and the unfiltered noise obstacle is determined as a newly added memory obstacle; if the second pixel value of any second sub-element in the first memory obstacle is equal to the preset pixel value, it is determined that the first memory obstacle and the unfiltered noise obstacle are the same object, and it is determined whether the first memory obstacle and the unfiltered noise obstacle are the same object following the movement of the self-vehicle according to the ego-vehicle posture data; if the first memory obstacle and the unfiltered noise obstacle are the same object following the movement of the self-vehicle, the number of occurrences of the first memory obstacle is increased by 1.
[0052] For example, let mem_objs(k) be the kth element in the memory obstacle set, and perform the memory obstacle information update operation on mem_objs(k), as follows: Let mem_objs_cells(k, l) be the lth cell element in mem_objs(k), and perform the following cell detection operation on mem_objs_cells(k, l): Convert mem_objs_cells(k, l) to the memory image coordinate system, and record the corresponding pixels as img_mem(u(k, l), v(k, l)), and determine whether the value of img_mem(u(k, l), v(k, l)) is 255: If the value of img_mem(u(k, l), v(k, l)) is 255, it means that mem_objs(k) and unfil t_objs(i) is the same object; otherwise, detect the next cell element; traverse all cell elements in mem_objs(k) to perform cell detection operation. As long as there is a cell whose corresponding pixel value in the memory image is 255, it means that mem_objs(k) and unfilt_objs(i) are the same object; if mem_objs(k) and unfilt_objs(i) are the same object, further determine whether they are the same object following the movement of the vehicle. If it is determined that mem_objs(k) and unfilt_objs(i) are the same object following the movement of the vehicle, the number of occurrences of mem_objs(k) is increased by 1, that is, mem_objs(k).count=mem_objs(k).count+1.
[0053] Traverse all elements in the first memory obstacle set to update the memory obstacle information. If all elements are not the same object as unfilt_objs(i), unfilt_objs(i) needs to be added to the new memory obstacle set.
[0054] Specifically, the step of determining whether the two are the same object following the movement of the vehicle includes: calculating whether the movement distance or rotation angle of the vehicle is greater than the corresponding threshold value according to the corresponding posture information of the vehicle, that is, the constraint condition is: Dis(mem_objs(k).loc, sel_objs(i).loc)>DIS_THRESH or Diff_Ang(mem_objs(k).ang, sel_objs(i).ang)>ANGLE_THRESH; as long as the above constraints are met, it means that mem_objs(k) and unfilt_objs(i) are the same object following the movement of the vehicle. Among them, u(k, l) and v(k, l) are respectively the horizontal coordinate and vertical coordinate corresponding to the lth cell of the kth element in the unfiltered noise obstacle set when it is converted to the memory image, img_mem(u(k, l), v(k, l)) is the pixel value with horizontal coordinate u(k, l) and vertical coordinate v(k, l) in the memory image; Dis(mem_objs(k).loc, sel_objs(i).loc) represents the moving distance of the vehicle, and DIS_THRESH is the corresponding distance threshold; Diff_Ang(mem_objs(k).ang, sel_objs(i).ang) represents the rotation angle of the vehicle, and ANGLE_THRESH is the corresponding angle threshold; in this application, the value of DIS_THRESH is 0.1m and the value of ANGLE_THRESH is 2°.
[0055] Traverse all elements in the obstacle set without filtering out noise points, repeat the above operation for each element, and update the memory obstacle set information.
[0056] Specifically, the elements of the obstacle set that have not filtered out the noise points also need to be output as target obstacles to the downstream for obstacle avoidance.
[0057] As an optional embodiment, the first noise-filtered image is updated using obstacle elements in the second memory obstacle set to obtain the second noise-filtered image, specifically including: obtaining the number of occurrences of each second memory obstacle in the second memory obstacle set; and updating the second memory obstacles whose number of occurrences is greater than a preset number into the first noise-filtered image to obtain the second noise-filtered image.
[0058] For example, let mem_objs(k) be the kth element in the memory obstacle set, and determine whether the number of occurrences of mem_objs(k) is greater than the number threshold COUNT_THRESH: if mem_objs(k).count>COUNT_THRESH, then use all cell elements of mem_objs(k) to update the first denoised image; otherwise, continue with the next memory obstacle to perform the above number determination.
[0059] Specifically, the step of updating the second memory obstacle to the first noise-filtered image includes: recording mem_objs(k) as the kth element in the memory obstacle set, recording mem_objs_cells(k, l) as the lth cell element in mem_objs(k), performing the following imaging operation on mem_objs_cells(k, l): converting mem_objs_cells(k, l) to the noise-filtered image coordinate system, recording the corresponding pixels as img_filter(u(k, l), v(k, l)), and performing an expansion operation on the noise-filtered image with img_filter(u(k, l), v(k, l)) as the center and a rectangular window size of R1*R1, that is, setting all pixel values in the window to 255; traversing all cell elements in mem_objs(k) to perform the above imaging operation. Among them, u(k, l) and v(k, l) are the horizontal coordinate and vertical coordinate corresponding to the lth cell of the kth element in the memory obstacle set when it is converted to the denoised image, respectively, and img_filter(u(k, l), v(k, l)) is the pixel value with horizontal coordinate u(k, l) and vertical coordinate v(k, l) in the denoised image; in this example, R1=5.
[0060] All elements in the second memory obstacle set are traversed, and the noise-filtered image information is updated for each element to obtain a new second noise-filtered image, which is used to filter out noise obstacles in the next frame.
[0061] This application first screens the noise obstacle set, then uses the noise filtered image to filter the noise obstacle set, then uses the unfiltered noise obstacle set to update the noise memory obstacle set, and then uses the noise memory obstacle set to update the noise filtered image, so that the noise filtered images of each frame have a very large correlation and can be updated in time.
[0062] The present application provides a method for updating a noise-filtered image, including: performing obstacle detection on the collected point cloud data to obtain a candidate noise obstacle set; filtering out known noise obstacles in the candidate noise obstacle set through a first noise-filtered image to obtain an unfiltered noise obstacle set, wherein the first noise-filtered image is obtained by updating the candidate noise obstacle set of the previous frame of point cloud data after noise obstacles are filtered out; updating the first memory obstacle set using the unfiltered noise obstacle set to obtain a second memory obstacle set; updating the first noise-filtered image using obstacle elements in the second memory obstacle set to obtain a second noise-filtered image, wherein the second noise-filtered image is used to filter out noise obstacles for the candidate noise obstacle set of the next frame. After filtering out noise using the noise-filtered image of the previous frame, the noise-filtered image is updated according to the filtering result, so that the updated noise-filtered image can be applied to the noise filtering of the next frame of data, and the latest noise-filtered image can be used each time the noise is filtered out, thereby solving the problem of untimely updating of the noise-filtered image.
[0063] According to another aspect of the embodiment of the present application, the present application provides a device for updating a noise-filtered image, such as Figure 2 As shown, including:
[0064] The detection module 202 is used to perform obstacle detection on the collected point cloud data to obtain a candidate noise obstacle set;
[0065] The filtering module 204 is used to filter out known noise obstacles in the candidate noise obstacle set through a first noise-filtered image to obtain an unfiltered noise obstacle set, wherein the first noise-filtered image is obtained by updating the candidate noise obstacle set of a previous frame of point cloud data after noise obstacles are filtered out;
[0066] A first updating module 206, configured to update the first memory obstacle set using the unfiltered noise obstacle set to obtain a second memory obstacle set;
[0067] The second updating module 208 is used to update the first noise-filtered image using the obstacle elements in the second memory obstacle set to obtain a second noise-filtered image, wherein the second noise-filtered image is used to filter out noise obstacles from the next frame of candidate noise obstacle set.
[0068] It should be noted that the detection module 202 in this embodiment can be used to execute step 101 in the embodiment of the present application, the filtering module 204 in this embodiment can be used to execute step 103 in the embodiment of the present application, the first update module 206 in this embodiment can be used to execute step 105 in the embodiment of the present application, and the second update module 208 in this embodiment can be used to execute step 107 in the embodiment of the present application.
[0069] The detection module 202 is also used to obtain point cloud data in the vehicle coordinate system and fit the ground plane to the point cloud data; perform background segmentation based on the fitted ground plane to obtain foreground point cloud data; perform obstacle detection based on the foreground point cloud data to obtain an initial obstacle set; extract candidate noise point obstacles that meet preset conditions from the initial obstacle set to obtain a candidate noise point obstacle set.
[0070] Optionally, the detection module 202 is further configured to select a first obstacle whose obstacle width is less than a width threshold and whose obstacle height is less than a height threshold from the initial obstacle set to obtain a first initial obstacle set; select a second obstacle whose number of sub-elements is less than a number threshold from the first initial obstacle set to obtain a second initial obstacle set; select a third obstacle that can only be detected by the target sensor from the second initial obstacle set to obtain a third initial obstacle set; and select candidate noise obstacles from the third initial obstacle set to obtain a candidate noise obstacle set, wherein the sub-elements of the candidate noise obstacles are all located within the target area under the ego-vehicle coordinates.
[0071] Optionally, the filtering module 204 is further used to obtain a first noise-filtered image and generate a filtered image coordinate system corresponding to the first noise-filtered image; convert the first sub-element of each candidate noise obstacle in the candidate noise obstacle set to the filtered image coordinate system to obtain a first pixel value corresponding to the first sub-element; compare the first pixel value with a preset pixel value; if the first pixel value of any first sub-element in the candidate noise obstacle is equal to the preset pixel value, the candidate noise obstacle is determined to be a known noise obstacle; if the first pixel values of the first sub-elements in the candidate noise obstacle are not equal to the preset pixel value, the candidate noise obstacle is determined to be an unfiltered noise obstacle; filter out the known noise obstacles in the candidate noise obstacle set to obtain an unfiltered noise obstacle set including unfiltered noise obstacles.
[0072] Optionally, the first updating module 206 includes:
[0073] A generating submodule, for generating a memory image corresponding to each unfiltered noise obstacle in the unfiltered noise obstacle set;
[0074] A matching submodule, used for matching the first memory obstacle in the first memory obstacle set with the unfiltered noise obstacle according to the memory image to obtain updated data;
[0075] The updating submodule is used to update the first memory obstacles according to the update data to obtain a second memory obstacle set, wherein the update data includes the newly added memory obstacles and the number of occurrences of each first memory obstacle.
[0076] Optionally, the matching submodule is further used to convert the second sub-element of each first memory obstacle to a memory image coordinate system corresponding to the memory image to obtain a second pixel value of the second sub-element; compare the second pixel value with a preset pixel value; if the second pixel values of all the second sub-elements in each first memory obstacle are not equal to the preset pixel value, it is determined that the first memory obstacle and the unfiltered noise obstacle are not the same object, and the unfiltered noise obstacle is determined as a newly added memory obstacle; if the second pixel value of any second sub-element in the first memory obstacle is equal to the preset pixel value, it is determined that the first memory obstacle and the unfiltered noise obstacle are the same object, and it is determined whether the first memory obstacle and the unfiltered noise obstacle are the same object following the movement of the self-vehicle according to the self-vehicle posture data; if the first memory obstacle and the unfiltered noise obstacle are the same object following the movement of the self-vehicle, the number of occurrences of the first memory obstacle is increased by 1.
[0077] Optionally, the second updating module 208 is further configured to obtain the number of occurrences of each second memory obstacle in the second memory obstacle set; and update the second memory obstacles whose number of occurrences is greater than a preset number into the first noise-filtered image to obtain a second noise-filtered image.
[0078] It should be noted here that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiments.
[0079] According to another aspect of the embodiments of the present application, the present application provides an electronic device, such as Figure 3 As shown, it includes a memory 301, a processor 303, a communication interface 305 and a communication bus 307. The memory 301 stores a computer program that can be run on the processor 303. The memory 301 and the processor 303 communicate through the communication interface 305 and the communication bus 307. When the processor 303 executes the computer program, the steps of the above method are implemented.
[0080] The memory and processor in the above electronic device communicate via a communication bus and a communication interface. The communication bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus may be divided into an address bus, a data bus, a control bus, etc.
[0081] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0082] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0083] According to another aspect of the embodiments of the present application, a mobile tool is provided, including the above-mentioned electronic device.
[0084] The mobile tool of the present application can be a vehicle or robot with an autonomous driving function, such as a vacuum truck, a floor scrubber, a sprinkler truck, a sewage treatment truck, a sanitation truck, a lawn mower, a golf cart, an entertainment vehicle, an amusement park autonomous driving device, a balance car, a sweeping robot, a food delivery robot, a shopping guide robot, etc.
[0085] In some possible implementations, the electronic device provided in the present application is an autonomous driving domain controller, and the mobile tool provided in the present application is equipped with an autonomous driving domain controller and various sensors for sensing environmental information and the mobile tool's own information (including but not limited to lidar, camera, millimeter wave radar, ultrasonic radar, inertial measurement unit, wheel speed meter, etc.). When the mobile tool is in working state, the autonomous driving domain controller is communicated with the various sensors loaded on the mobile tool, and path planning is performed for the mobile tool based on the environmental information collected by these sensors and the mobile tool's own information, so as to achieve the purpose of controlling the mobile tool to drive automatically.
[0086] According to another aspect of the embodiments of the present application, a computer-readable medium having a non-volatile program code executable by a processor is provided.
[0087] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.
[0088] When the embodiments of the present application are specifically implemented, reference may be made to the above-mentioned embodiments, which have corresponding technical effects.
[0089] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.
[0090] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.
[0091] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0092] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0093] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0094] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0095] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0096] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk. It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.
[0097] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.
Claims
1. A method for updating a noise-filtered image, characterized in that: include: Perform obstacle detection on the collected point cloud data to obtain a candidate noise obstacle set; Filtering known noise obstacles in the candidate noise obstacle set through a first noise-filtered image to obtain an unfiltered noise obstacle set, wherein the first noise-filtered image is obtained by updating the candidate noise obstacle set of a previous frame of the point cloud data after filtering out noise obstacles; Using the unfiltered noise obstacle set to update the first memory obstacle set, to obtain a second memory obstacle set; The first noise-filtered image is updated using obstacle elements in the second memory obstacle set to obtain a second noise-filtered image, wherein the second noise-filtered image is used to filter out noise point obstacles from the next frame of candidate noise point obstacle set.
2. The method according to claim 1, characterized in that The obstacle detection is performed on the collected point cloud data to obtain a candidate noise obstacle set, which specifically includes: Acquire the point cloud data in the vehicle coordinate system, and perform ground plane fitting on the point cloud data; Perform background segmentation based on the fitted ground plane to obtain foreground point cloud data; Perform obstacle detection according to the foreground point cloud data to obtain an initial obstacle set; Candidate noise point obstacles meeting preset conditions are extracted from the initial obstacle set to obtain the candidate noise point obstacle set.
3. The method according to claim 2, characterized in that The extracting candidate noise point obstacles meeting preset conditions from the initial obstacle set to obtain the candidate noise point obstacle set comprises: Selecting a first obstacle whose obstacle width is less than a width threshold and whose obstacle height is less than a height threshold from the initial obstacle set, to obtain a first initial obstacle set; Selecting a second obstacle whose number of sub-elements is less than a number threshold from the first initial obstacle set to obtain a second initial obstacle set; Selecting a third obstacle that can only be detected by the target sensor from the second initial obstacle set to obtain a third initial obstacle set; The candidate noise point obstacles are selected from the third initial obstacle set to obtain the candidate noise point obstacle set, wherein the sub-elements of the candidate noise point obstacles are all located within the target area under the ego-vehicle coordinates.
4. The method according to claim 1, characterized in that The filtering out known noise point obstacles in the candidate noise point obstacle set by using the first noise filtering image to obtain an unfiltered noise point obstacle set specifically includes: Acquire the first noise-filtered image, and generate a filtered image coordinate system corresponding to the first noise-filtered image; Converting the first sub-element of each candidate noise obstacle in the candidate noise obstacle set to the filtered image coordinate system to obtain a first pixel value corresponding to the first sub-element; Comparing the first pixel value with a preset pixel value; If the first pixel value of any of the first sub-elements in the candidate noise obstacle is equal to the preset pixel value, the candidate noise obstacle is determined to be the known noise obstacle; if the first pixel values of the first sub-elements in the candidate noise obstacle are not equal to the preset pixel value, the candidate noise obstacle is determined to be the unfiltered noise obstacle; The known noise point obstacles in the candidate noise point obstacle set are filtered out to obtain the unfiltered noise point obstacle set including the unfiltered noise point obstacles.
5. The method according to claim 1, characterized in that: The updating of the first memory obstacle set by using the unfiltered noise obstacle set to obtain the second memory obstacle set specifically includes: Generate a memory image corresponding to each unfiltered noise point obstacle in the unfiltered noise point obstacle set; Matching the first memory obstacle in the first memory obstacle set with the unfiltered noise obstacle according to the memory image to obtain update data; The first memory obstacles are updated according to the update data to obtain the second memory obstacle set, wherein the update data includes the newly added memory obstacles and the number of occurrences of each of the first memory obstacles.
6. The method according to claim 5, characterized in that The matching the first memory obstacle in the first memory obstacle set with the unfiltered noise obstacle according to the memory image specifically includes: Converting the second sub-element of each of the first memory obstacles to a memory image coordinate system corresponding to the memory image to obtain a second pixel value of the second sub-element; Comparing the second pixel value with a preset pixel value; If the second pixel values of all the second sub-elements in each of the first memory obstacles are not equal to the preset pixel value, it is determined that the first memory obstacle and the unfiltered noise obstacle are not the same object, and the unfiltered noise obstacle is determined as the newly added memory obstacle; if the second pixel value of any of the second sub-elements in the first memory obstacle is equal to the preset pixel value, it is determined that the first memory obstacle and the unfiltered noise obstacle are the same object, and it is determined whether the first memory obstacle and the unfiltered noise obstacle are the same object following the movement of the ego vehicle according to the ego vehicle posture data; If the first memorized obstacle and the unfiltered noise obstacle are the same object moving with the vehicle, the number of occurrences of the first memorized obstacle is increased by 1.
7. The method according to claim 1, characterized in that The updating of the first noise-filtered image by using the obstacle elements in the second memory obstacle set to obtain the second noise-filtered image specifically includes: Obtaining the number of occurrences of each second memory obstacle in the second memory obstacle set; The second memory obstacle whose occurrence times are greater than a preset number is updated into the first noise-filtered image to obtain the second noise-filtered image.
8. A device for updating a noise-filtered image, characterized in that: include: The detection module is used to detect obstacles on the collected point cloud data and obtain a set of candidate noise obstacle points; a filtering module, configured to filter out known noise obstacles in the candidate noise obstacle set through a first noise-filtered image to obtain an unfiltered noise obstacle set, wherein the first noise-filtered image is obtained by updating the candidate noise obstacle set of a previous frame of the point cloud data after noise obstacles are filtered out; A first updating module, configured to update the first memory obstacle set by using the unfiltered noise obstacle set to obtain a second memory obstacle set; The second updating module is used to update the first noise-filtered image using the obstacle elements in the second memory obstacle set to obtain a second noise-filtered image, wherein the second noise-filtered image is used to filter out noise obstacles from the next frame of candidate noise obstacle set.
9. An electronic device, comprising a memory, a processor, a communication interface and a communication bus, wherein the memory stores a computer program that can be run on the processor, and the memory and the processor communicate through the communication bus and the communication interface, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A mobile tool, characterized in that: Comprising the electronic device as claimed in claim 9.
Citation Information
Patent Citations
Multi-barrier detection and tracking method based on laser radar depth image
CN107330925A
Method for improving cleaning efficiency and cleaning robot
CN112666946A
Dynamic obstacle avoidance method and device and mobile robot
CN114764252A
Robot, local map updating method and device thereof and storage medium
CN115014322A
Obstacle detection method and device, electronic equipment and storage medium
CN116500641A
