Obstacle detection method and device, storage medium and computer equipment

Abstract

The invention discloses an obstacle detection method and device, a storage medium and computer equipment. Comprising the steps of obtaining a first position coordinate of an obstacle in a three-dimensional coordinate system of a target object according to to-be-measured point cloud data, scanned by a laser radar, of a space where the target object is located; according to a preset conversion relation, converting the first position coordinate in the three-dimensional coordinate system into a station center coordinate system to obtain a second position coordinate of the obstacle; generating a thermal image according to the laser intensity values corresponding to the second position coordinates and the first position coordinates; updating the laser intensity value of the boundary point of the obstacle in the thermal image to obtain a target intensity value; according to a second position coordinate and a target intensity value in the thermal image, performing correction processing on the to-be-measured point cloud data, and determining target point cloud data; and inputting the target point cloud data into the obstacle prediction model to obtain target obstacle information. Consequently, the front and back frame information is used for suppression and complementation, and the obstacle detection precision is improved.

Description

Technical Field

[0001] The present application relates to the field of laser radar technology, and in particular to an obstacle detection method, device, storage medium and computer equipment. Background Art

[0002] LiDAR (laser radar) sensors provide high-resolution laser detection data for autonomous driving by emitting thousands of laser signals, receiving laser signals reflected by objects back to the sensor, calculating the time required for the signals to return, and determining the distance of the objects.

[0003] In related technologies, due to the small size and low frequency of static obstacles (cones, warning signs, etc.), traditional lidar perception solutions fail to detect static obstacles in vehicle cruise assisted driving, resulting in no effective warnings and braking of the vehicle and causing panic and safety risks to the driver. Summary of the invention

[0004] In view of this, the present application provides an obstacle detection method, device, storage medium and computer equipment. Based on the characteristic that the position of static obstacles does not change, a mapping method is used to suppress false positives (FP) and complement false negatives (FN), thereby achieving the purpose of correcting point cloud data and further improving the obstacle recognition accuracy based on point cloud data.

[0005] According to one aspect of the present application, there is provided an obstacle detection method, comprising:

[0006] According to the point cloud data to be measured in the space where the target object is located scanned by the laser radar, the first position coordinates of the obstacle in the three-dimensional coordinate system of the target object are obtained, wherein the points in the point cloud data to be measured correspond to the laser intensity values;

[0007] According to the preset conversion relationship, the first position coordinates in the three-dimensional coordinate system are converted to the station center coordinate system to obtain the second position coordinates of the obstacle;

[0008] generating a thermal image according to the laser intensity values ​​corresponding to the second position coordinates and the first position coordinates;

[0009] The laser intensity value of the boundary point of the obstacle in the thermal image is updated to obtain the target intensity value;

[0010] According to the second position coordinates and the target intensity value in the thermal image, the point cloud data to be measured is corrected and processed to determine the target point cloud data;

[0011] The target point cloud data is input into the obstacle prediction model to obtain the target obstacle information.

[0012] Optionally, the point cloud data to be measured is corrected according to the second position coordinates and the target intensity value in the thermal image, specifically including:

[0013] According to a preset conversion relationship, the second position coordinates in the thermal image are converted into a three-dimensional coordinate system to obtain the third position coordinates of the obstacle in the point cloud data to be measured;

[0014] Performing obstacle supplementation processing on the point cloud data to be measured according to the third position coordinates to obtain first point cloud data;

[0015] Updating the laser intensity value of the boundary point of the obstacle in the first point cloud data to the target intensity value;

[0016] Identify boundary points in the first point cloud data according to preset identification parameters and target strength values;

[0017] The boundary points of the first point cloud data are edited according to preset conditions to generate target point cloud data.

[0018] Optionally, obstacle supplementation processing is performed on the point cloud data to be measured according to the third position coordinates, specifically including:

[0019] Determine the difference between the third position coordinates corresponding to the point cloud data to be measured and the first position coordinates;

[0020] If the difference between the third position coordinates corresponding to the point cloud data to be measured and the first position coordinates is greater than or equal to the preset difference, an obstacle detection frame is added to the point cloud data to be measured according to the third position coordinates corresponding to the difference greater than or equal to the preset difference;

[0021] The obstacle detection frame includes boundary points of the obstacle.

[0022] Optionally, the preset recognition parameters include a preset precision rate, and the boundary points of the first point cloud data are edited according to the preset conditions, specifically including:

[0023] Count the first times that the boundary point in the first point cloud data of the current frame appears in the first point cloud data of the continuous frame interval to which the current frame belongs;

[0024] If the first number is less than or equal to the first preset number, the boundary point in the first point cloud data of the current frame is deleted.

[0025] Optionally, the preset recognition parameters include a preset recall rate, and the boundary points of the first point cloud data are edited according to the preset conditions, specifically including:

[0026] Determine an estimated point in the first point cloud data of the current frame whose target intensity value is within a preset intensity range;

[0027] Counting the second number of times that the target intensity value of the estimated point in the first point cloud data of the continuous frame interval to which the current frame belongs is greater than the maximum value in the preset intensity range;

[0028] If the second number is greater than or equal to the second preset number, the estimated point is determined as a boundary point in the first point cloud data of the current frame.

[0029] Optionally, updating the laser intensity value corresponding to the obstacle in the thermal image includes:

[0030] Determine the direction angle between the centroid points of adjacent obstacles in the thermal image according to the second position coordinates;

[0031] Clustering the direction angles to determine the layout trajectory of obstacles in the thermal image;

[0032] Calculate the average value of the laser intensity values ​​at the boundary points of all obstacles on the layout trajectory;

[0033] The average value is taken as the target intensity value of the boundary points of the obstacles on the layout trajectory.

[0034] Optionally, the obstacle detection method further includes:

[0035] Determine the arrangement rules of adjacent obstacles in the layout trajectory;

[0036] The laser intensity values ​​of the points in the thermal image that meet the arrangement rules are configured as target intensity values ​​to supplement the boundary points of the obstacles.

[0037] Optionally, the obstacle detection method further includes:

[0038] If the loss value of any obstacle detection frame in the target point cloud data exceeds the first loss value range, any obstacle detection frame in the target point cloud data is deleted.

[0039] Optionally, the obstacle detection method further includes:

[0040] If the loss value of any boundary point in the target point cloud data exceeds the second loss value range, any boundary point in the target point cloud data is deleted.

[0041] Optionally, the obstacle detection method further includes:

[0042] If the size of the obstacle detection frame in the target point cloud data does not match the preset frame size corresponding to the obstacle, delete the obstacle detection frame.

[0043] Optionally, obtaining the first position coordinates of the obstacle in the three-dimensional coordinate system of the target object according to the point cloud data to be measured in the space where the target object is located scanned by the laser radar includes:

[0044] Inputting the point cloud data to be measured into the obstacle prediction model to obtain predicted obstacle information, wherein the predicted obstacle information includes the first position coordinates of the obstacle in the point cloud data to be measured;

[0045] If the point cloud data to be measured includes multiple frames of point cloud data, the first position coordinates corresponding to the multiple frames of point cloud data are fused.

[0046] According to another aspect of the present application, there is provided an obstacle detection device, comprising:

[0047] A detection module is used to obtain the first position coordinates of the obstacle in the three-dimensional coordinate system of the target object according to the point cloud data to be measured in the space where the target object is located scanned by the laser radar, wherein the points in the point cloud data to be measured correspond to the laser intensity values;

[0048] A mapping module, for converting the first position coordinates in the three-dimensional coordinate system to the station center coordinate system according to a preset conversion relationship to obtain the second position coordinates of the obstacle; and for generating a thermal image according to the laser intensity values ​​corresponding to the second position coordinates and the first position coordinates;

[0049] An updating module is used to update the laser intensity value of the boundary point of the obstacle in the thermal image to obtain a target intensity value;

[0050] A correction module is used to correct the point cloud data to be measured according to the second position coordinates and the target intensity value in the thermal image to determine the target point cloud data;

[0051] The detection module is also used to input the target point cloud data into the obstacle prediction model to obtain the target obstacle information.

[0052] Optionally, the mapping module is further used to convert the second position coordinates in the thermal image into a three-dimensional coordinate system according to a preset conversion relationship to obtain the third position coordinates of the obstacle in the point cloud data to be measured;

[0053] The correction module is specifically used to perform obstacle supplement processing on the point cloud data to be measured according to the third position coordinates to obtain the first point cloud data; and update the laser intensity value of the boundary point of the obstacle in the first point cloud data to the target intensity value;

[0054] The obstacle detection device further includes: an identification module, configured to identify boundary points in the first point cloud data according to preset identification parameters and target strength values;

[0055] The correction module is specifically used to edit the boundary points of the first point cloud data according to preset conditions to generate target point cloud data.

[0056] Optionally, the correction module is specifically used to determine the difference between the third position coordinates corresponding to the point cloud data to be measured and the first position coordinates; if the difference between the third position coordinates corresponding to the point cloud data to be measured and the first position coordinates is greater than or equal to a preset difference, an obstacle detection frame is added to the point cloud data to be measured according to the third position coordinates corresponding to the difference greater than or equal to the preset difference; wherein the obstacle detection frame contains the boundary points of the obstacle.

[0057] Optionally, the preset recognition parameters include a preset precision rate;

[0058] The correction module is specifically used to count the first time that the boundary point in the first point cloud data of the current frame appears in the first point cloud data of the continuous frame interval to which the current frame belongs; if the first time number is less than or equal to the first preset number, the boundary point in the first point cloud data of the current frame is deleted.

[0059] Optionally, the preset recognition parameters include a preset recall rate;

[0060] The correction module is specifically used to determine an estimated point in the first point cloud data of the current frame whose target intensity value is within a preset intensity range; to count the second number of times that the target intensity value of the estimated point in the first point cloud data of the continuous frame interval to which the current frame belongs is greater than the maximum value in the preset intensity range; if the second number is greater than or equal to the second preset number, the estimated point is determined as a boundary point in the first point cloud data of the current frame.

[0061] Optionally, the update module is specifically used to determine the direction angle between the center of gravity points of adjacent obstacles in the thermal image according to the second position coordinates; perform clustering processing on the direction angles to determine the layout trajectory of the obstacles in the thermal image; calculate the average value of the laser intensity values ​​of the boundary points of all obstacles on the layout trajectory; and use the average value as the target intensity value of the boundary points of the obstacles on the layout trajectory.

[0062] Optionally, the correction module is further used to determine the arrangement rules of adjacent obstacles in the layout trajectory; configure the laser intensity values ​​of points in the thermal image that meet the arrangement rules as target intensity values ​​to supplement the boundary points of the obstacles.

[0063] Optionally, the correction module is also used to delete any obstacle detection box in the target point cloud data if the loss value of any obstacle detection box in the target point cloud data exceeds the first loss value range.

[0064] Optionally, the correction module is further used to delete any boundary point in the target point cloud data if the loss value of any boundary point in the target point cloud data exceeds the second loss value range.

[0065] Optionally, the correction module is further used to delete the obstacle detection frame if the size of the obstacle detection frame in the target point cloud data does not match the preset frame size corresponding to the obstacle.

[0066] Optionally, the detection module is specifically used to input the point cloud data to be measured into the obstacle prediction model to obtain predicted obstacle information, wherein the predicted obstacle information includes the first position coordinates of the obstacle in the point cloud data to be measured;

[0067] The obstacle detection device also includes: a fusion module, which is used to fuse the first position coordinates corresponding to the multiple frames of point cloud data if the point cloud data to be measured includes multiple frames of point cloud data.

[0068] According to another aspect of the present application, a computer-readable storage medium is provided, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the above obstacle detection method are implemented.

[0069] According to another aspect of the present application, a computer device is provided, comprising at least one processor coupled to a memory, the memory storing a computer program running on the processor, and the processor implementing the steps of the above obstacle detection method when executing the program.

[0070] By means of the above technical solution, a laser radar is used to scan the space where the target object is located to obtain at least one frame of point cloud data to be measured in the space. The obstacle information of the obstacles appearing in the point cloud data to be measured is identified, and the first position coordinates of the target object in the obstacle information in the three-dimensional coordinate system are converted to the station center coordinate system through conversion processing, so as to obtain the second position coordinates of all obstacles in at least one frame of point cloud data to be measured in the station center coordinate system. According to the laser intensity values ​​corresponding to the second position coordinates and the first position coordinates, a thermal image containing all obstacles that can be detected in at least one frame of point cloud data to be measured is generated, so that all obstacles are superimposed on the same coordinate system, and a unified reference standard is formed for multiple obstacles. Furthermore, due to the distance difference when the laser radar scans obstacles at different positions, the laser intensity value of the boundary points of the obstacles in the thermal image is first updated to make the target intensity value of the boundary points of the obstacles in the thermal image uniform, and then the point cloud data to be tested where the obstacle appears is corrected according to the second position coordinates and target intensity value of the obstacle in the thermal image, so as to complete and delete the obstacles in each frame of the point cloud data to be tested, and finally the corrected target point cloud data is used to identify the obstacles again to obtain the target obstacle information. On the one hand, based on the characteristic that the position of static obstacles remains unchanged, the thermal image of possible obstacles in the viewing space is constructed by coordinate transformation and local mapping, and the obstacle features in the thermal image are transferred to each frame of point cloud data, so that the information of the previous and next frames can be very effectively used for FP suppression and FN completion, which solves the problem that the laser radar sensor is easily affected by the environment and missed detection in obstacle detection and measurement, improves the detection accuracy and detection range of obstacle identification, and achieves a high-precision 3D prediction effect of static obstacles. On the other hand, static obstacles can be mapped by aggregating obstacles on multi-frame point cloud data into the same coordinate system, which can obtain 3D true values ​​at longer distances and increase the detection distance of the lidar, making it easier to detect objects at longer distances.

[0071] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0073] Figure 1 A schematic diagram of the process of an obstacle detection method provided in an embodiment of the present application is shown;

[0074] Figure 2The structure block diagram of the obstacle detection device provided in the embodiment of the present application is shown. DETAILED DESCRIPTION

[0075] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0076] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0077] Those skilled in the art will appreciate that, unless expressly stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, or there may be intermediate elements. In addition, the "connection" or "connection" used herein may include wireless connection or wireless fusion. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0078] Now, exemplary embodiments according to the present application will be described in more detail with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in a variety of different forms and should not be interpreted as being limited to the embodiments set forth herein. It should be understood that these embodiments are provided to make the disclosure of the present application thorough and complete, and to fully convey the concepts of these exemplary embodiments to those of ordinary skill in the art.

[0079] In this embodiment, an obstacle detection method is provided. Figure 1 As shown, the method includes:

[0080] Step 101, obtaining the first position coordinates of the obstacle in the three-dimensional coordinate system of the target object according to the point cloud data to be measured in the space where the target object is located scanned by the laser radar.

[0081] Among them, the points in the point cloud data to be measured correspond to laser intensity values.

[0082] It is understandable that the target object can be a vehicle, a human body, an animal, or other subject equipped with a laser radar. The point cloud data to be measured can be point cloud data generated based on the laser signal of the target object returning in real time, or it can be historical point cloud data. The embodiment of the present application does not specifically limit the target object and the point cloud data to be measured.

[0083] In this embodiment, a laser radar is used to scan the space where the target object is located to obtain at least one frame of point cloud data to be measured in the space. The point cloud data to be measured is initially inspected to identify the first position coordinates of obstacles that may appear in each frame of point cloud data to be measured. In order to facilitate the subsequent placement of all initially inspected obstacles in a unified coordinate system based on the first position coordinates, local mapping of the target object is achieved, so that obstacles on multiple frames of point cloud data can be aggregated in the same coordinate system, thereby obtaining 3D true values ​​at a longer distance, increasing the detection distance of the laser radar, and facilitating the detection of objects at a longer distance.

[0084] In a specific application scenario, step 101 specifically includes the following steps:

[0085] Step 101 - 1 , input the point cloud data to be measured into the obstacle prediction model to obtain predicted obstacle information.

[0086] The predicted obstacle information includes the first position coordinates of the obstacle in the point cloud data to be measured, and may also include information such as the obstacle detection frame and the number of obstacles.

[0087] Specifically, the obstacle prediction model is based on the obstacle recognition algorithm commonly used in the prior art. For example, the obstacle prediction model can be a PointRCNN model. The Region Proposal method is used through the PointRCNN model to select candidate regions to extract obstacle detection frames. The region corresponding to the obstacle detection frame is input into CNN for training, obstacle features are extracted, and finally the precise position of the obstacle is obtained through classification and boundary regression.

[0088] Step 101 - 2 : If the point cloud data to be measured includes multiple frames of point cloud data, a fusion process is performed on the first position coordinates corresponding to the multiple frames of point cloud data.

[0089] In this embodiment, static obstacle prediction is performed on each frame of the point cloud data to be tested based on the obstacle prediction model to obtain the predicted obstacle information corresponding to each frame. Taking into account the possibility of repeated scanning of obstacles during the data acquisition process, the first position coordinates corresponding to the multi-frame point cloud data are fused based on the model essemble (hybrid fusion) model to obtain the prediction results (predicted obstacle information) of static obstacles in continuous frames, and remove the prediction results of obstacles that may be repeated in continuous frames. In this way, while achieving the initial inspection of obstacles, the recognition error caused by the same obstacle in different frames of point cloud data is reduced, and the recognition accuracy of multi-frame point cloud data is improved.

[0090] Step 102: According to a preset conversion relationship, the first position coordinates in the three-dimensional coordinate system are converted to the station center coordinate system to obtain the second position coordinates of the obstacle.

[0091] Among them, the local Cartesian coordinates coordinate system is also called the site coordinate system, the east-north-sky coordinate system ENU. The station-centered coordinate system is a coordinate system with the measuring station as the origin, that is, the prepared base is used to locate the point and observe and measure. The preset conversion relationship can be determined by the relative angle between the three-dimensional coordinate system of the target object and the station-centered coordinate system.

[0092] Step 103: Generate a thermal image according to the laser intensity values ​​corresponding to the second position coordinates and the first position coordinates.

[0093] In this embodiment, the first position coordinates of the target object in the obstacle information in the three-dimensional coordinate system are converted to the station center coordinate system through conversion processing, and the second position coordinates of all obstacles in at least one frame of the point cloud data to be measured in the station center coordinate system are obtained. According to the laser intensity values ​​corresponding to the second position coordinates and the first position coordinates, a thermal image containing all obstacles that can be detected in at least one frame of the point cloud data to be measured is generated, so that all obstacles are superimposed in the same coordinate system, and a unified reference standard is formed for multiple obstacles.

[0094] Taking the cone barrel in the autonomous driving scenario as an example, the static obstacle target is converted to the station center coordinate system through the vehicle coordinate system. Since the point cloud data carries the laser intensity value (RB) of the lidar, after conversion to the station center coordinate system, the laser intensity value is assigned to the corresponding second position coordinate position to form a RB superimposed thermal image of at least one frame of point cloud data, which contains all possible obstacles in at least one frame of point cloud data.

[0095] Step 104 , updating the laser intensity value of the boundary point of the obstacle in the thermal image to obtain a target intensity value.

[0096] In this embodiment, due to the difference in distance when the laser radar scans obstacles at different positions, the laser intensity values ​​of the boundary points of the obstacles in the thermal image are first updated to unify the target intensity values ​​of the boundary points of the obstacles in the thermal image, so as to facilitate the subsequent correction processing of multiple frames of point cloud data based on the thermal image at the same time, thereby further reducing the recognition error.

[0097] In a specific application scenario, step 104 specifically includes the following steps:

[0098] Step 104 - 1 , determining the direction angle between the center points of adjacent obstacles in the thermal image according to the second position coordinates.

[0099] The direction angle refers to the angle not greater than 90° between the north or south direction and the target direction, with the position of the target as the center. The target direction is the direction in which the target is pointing to another obstacle. In this embodiment, the target is one of two adjacent obstacles. It should be noted that when calculating the direction angle of the center of gravity of all adjacent obstacles in the thermal image, in order to ensure the statistical standard of the direction angle, the same conditions are used as much as possible when setting the target of the adjacent obstacles. For example, for all adjacent obstacles, the obstacle target located in the negative direction of the X-axis of the two adjacent obstacles is used.

[0100] Step 104 - 2 , clustering the direction angles to determine the layout trajectory of the obstacles in the thermal image.

[0101] Step 104 - 3 , calculating the average value of the laser intensity values ​​of the boundary points of all obstacles on the layout trajectory.

[0102] Step 104 - 4 , taking the average value as the target strength value of the boundary point of the obstacle on the layout trajectory.

[0103] In this embodiment, the direction angles between the centroids of all adjacent obstacles in the thermal image are clustered, so that the adjacent obstacles with similar direction angles are divided into a set, and then the obstacles in the set are connected in sequence to form a layout track of the obstacles. The average value of the laser intensity values ​​of the boundary points of all obstacles on the layout track is used as the target intensity value of the obstacles in the set, thereby achieving the unification of the target intensity values ​​of the boundary points of the obstacles in the thermal image, correcting the laser intensity value error caused by the difference in scanning distance, so as to facilitate the subsequent correction processing of multiple frames of point cloud data based on the thermal image at the same time, and further reducing the recognition error.

[0104] For example, the on-board laser radar scans the cones on the lane, and connects the cones near the same horizontal position in the thermal image to the layout track. The cones on the layout track can be roughly regarded as a row of cones arranged beside the lane. The laser intensity value score of each layout track is updated to the average score of the layout track, so that the same type of cones are divided by the layout track, and the laser intensity value is updated and corrected.

[0105] Furthermore, in some possible embodiments, after step 104, the method further includes: determining an arrangement rule of adjacent obstacles in the layout trajectory; and configuring the laser intensity values ​​of points in the thermal image that meet the arrangement rule as target intensity values ​​to supplement the boundary points of the obstacles.

[0106] The arrangement rule is used to indicate the arrangement pattern between multiple obstacles, for example, multiple obstacles are arranged at intervals or horizontally.

[0107] In this embodiment, after the layout track is generated and the target intensity value is configured, the arrangement rule of adjacent obstacles in the layout track is determined according to the coordinate positions between multiple adjacent obstacles. According to the arrangement rule, the laser intensity value of the non-boundary point of the thermal image that meets the arrangement rule is configured as the target intensity value, so that the point is used as the boundary point of the new obstacle to supplement the obstacles that may be missed during the initial inspection and extend the obstacles at the beginning and end of the layout track. Even if the laser radar does not perceive the obstacle, it can be supplemented and corrected through the arrangement rule, thereby achieving the purpose of obstacle correction, solving the problem that the laser radar sensor is easily affected by the environment and missed detection in obstacle detection measurement, and improving the detection accuracy and detection range of obstacle recognition.

[0108] Step 105 , correcting the point cloud data to be measured according to the second position coordinates and the target intensity value in the thermal image, and determining the target point cloud data.

[0109] In a specific application scenario, step 105 specifically includes the following steps:

[0110] Step 105 - 1 , according to a preset conversion relationship, the second position coordinates in the thermal image are converted into a three-dimensional coordinate system to obtain the third position coordinates of the obstacle in the point cloud data to be measured.

[0111] Step 105 - 2 , performing obstacle supplementation processing on the point cloud data to be measured according to the third position coordinates to obtain first point cloud data.

[0112] In this embodiment, according to the preset conversion relationship between the three-dimensional coordinate system of the target object and the station center coordinate system, the obstacles in the thermal image are redistributed to each frame of the point cloud data to be tested. Since the thermal image collects all obstacles that may be detected, after the secondary conversion of the coordinate system, the point cloud data of continuous frames can be suppressed and completed by the obstacles in the overall space, thereby achieving the effect of correcting the point cloud data to be tested.

[0113] Specifically, step 105-2, that is, performing obstacle supplementation processing on the point cloud data to be measured according to the third position coordinates, specifically includes:

[0114] Step 105-2-a, determining the difference between the third position coordinates corresponding to the point cloud data to be measured and the first position coordinates.

[0115] Step 105-2-b, if the difference between the third position coordinates corresponding to the point cloud data to be measured and the first position coordinates is greater than or equal to the preset difference, an obstacle detection box is added to the point cloud data to be measured according to the third position coordinates corresponding to the difference greater than or equal to the preset difference.

[0116] The obstacle detection frame includes boundary points of the obstacle.

[0117] In this embodiment, when the PointRCNN model is used as the obstacle prediction model, the obstacle detection frame will be identified during the initial inspection. Therefore, in the process of redistributing the obstacles in the thermal image to each frame of the point cloud data to be tested, the difference between the third position coordinates and the first position coordinates based on the same frame of the point cloud data to be tested is determined. If the difference is greater than or equal to the preset difference, it means that the obstacle is not detected at the position of the third position coordinate in the original point cloud data to be tested, then the obstacle detection frame is supplemented in the original point cloud data to be tested according to the third position coordinate, and the first point cloud data after the obstacle supplement processing is obtained. On the contrary, it means that the obstacle has been detected in the point cloud data to be tested, and the obstacle detection frame that has been detected in the point cloud data to be tested is retained. Therefore, based on the characteristic that the position of the static obstacle remains unchanged, the thermal image of possible obstacles in the viewing space is constructed by coordinate transformation and local mapping. When the obstacle features in the thermal image are decentralized to each frame of the point cloud data, the information of the previous and next frames is used to suppress FP and complete FN, thereby improving the detection accuracy and detection range of obstacle recognition.

[0118] Step 105 - 3 , updating the laser intensity value of the boundary point of the obstacle in the first point cloud data to the target intensity value.

[0119] Step 105 - 4 , identifying boundary points in the first point cloud data according to preset identification parameters and target strength values.

[0120] The preset recognition parameters include a preset precision rate and / or a preset recall rate.

[0121] Step 105 - 5 , editing the boundary points of the first point cloud data according to preset conditions to generate target point cloud data.

[0122] In this embodiment, the laser intensity value of the boundary point of the obstacle in the first point cloud data corresponding to the layout trajectory is updated to the target intensity value corresponding to the layout trajectory, so as to unify the laser intensity values ​​of all obstacles on the same layout trajectory in the converted first point cloud data. Then, the boundary point recognition of the first point cloud data is performed according to the preset recognition parameters and the target intensity value, and the recognized boundary points in the first point cloud data of the continuous frames are edited according to the preset conditions to generate the target point cloud data. Thus, the FP suppression and FN completion are performed by using the information of the previous and next frames, which solves the problem that the laser radar sensor is easily affected by the environment and missed detection in obstacle detection and measurement, and realizes the 3D prediction of static obstacles with high accuracy.

[0123] Specifically, step 105-5, that is, editing the boundary points of the first point cloud data according to the preset conditions, includes the following methods:

[0124] Method 1: When the preset recognition parameters include a preset precision rate, the first time that the boundary point in the first point cloud data of the current frame appears in the first point cloud data of the continuous frame interval to which the current frame belongs is counted; if the first time number is less than or equal to the first preset number, the boundary point in the first point cloud data of the current frame is deleted.

[0125] In this embodiment, the preset precision rate and target intensity value are used as recognition parameters. If the current frame is detected, but the boundary points detected in the first point cloud data sequence of the continuous frame interval to which the current frame belongs do not exceed the first preset number, it means that the boundary points in the first point cloud data of the current frame may be false detections, and they are deleted.

[0126] Method 2, when the preset recognition parameters include a preset recall rate, determine an estimated point in the first point cloud data of the current frame whose target intensity value is within a preset intensity range; count the second number of times that the target intensity value of the estimated point in the first point cloud data of the continuous frame interval to which the current frame belongs is greater than the maximum value in the preset intensity range; if the second number is greater than or equal to the second preset number, determine the estimated point as a boundary point in the first point cloud data of the current frame.

[0127] The preset intensity range may be reasonably set according to the release threshold of the point cloud data. For example, if the preset intensity range is set to 0.2-0.5, the maximum value in the preset intensity range is 0.5.

[0128] In this embodiment, a preset recall rate and a target intensity value are used as recognition parameters. If the current frame detects an estimated point within a preset intensity range, and at the same time, in a continuous frame interval to which the current frame belongs, there are more than a second preset number of times that the intensity at this point is greater than the maximum value in the preset intensity range, indicating that the estimated point may be missed in the first point cloud data of the current frame, then the estimated point in the current frame is determined as a boundary point in the first point cloud data of the current frame, thereby completing the boundary point.

[0129] It is understandable that the first preset number and the second preset number can be reasonably set according to the detection accuracy, and the first preset number and the second preset number can be the same or different. The continuous frame interval is a plurality of frames before and after the current frame.

[0130] Step 106: input the target point cloud data into the obstacle prediction model to obtain target obstacle information.

[0131] In this embodiment, the point cloud data to be measured where the obstacle appears is corrected according to the second position coordinates and target intensity value of the obstacle in the thermal image, so as to complete and delete the obstacle in each frame of the point cloud data to be measured, and finally the obstacle is identified again using the corrected target point cloud data to obtain the target obstacle information. Based on the characteristic that the position of static obstacles remains unchanged, a thermal image of possible obstacles in the viewing space is constructed by coordinate transformation and local mapping, and the obstacle features in the thermal image are transferred to each frame of point cloud data, so that the information of the previous and next frames can be very effectively used for FP suppression and FN completion, solving the problem that the laser radar sensor is easily affected by the environment and missed detection in obstacle detection and measurement, improving the detection accuracy and detection range of obstacle identification, and achieving a high-precision 3D prediction effect of static obstacles.

[0132] For example, the traditional laser radar perception solution, that is, directly inputting the point cloud data to be measured into the obstacle prediction model to obtain the obstacle information, and the various indicators of the target obstacle information obtained by the obstacle detection method proposed in this application are shown in Table 1. As can be seen from Table 1, the obstacle detection method proposed in this application improves the precision by 1% to 3%, the recall by 18% to 24%, and the error by more than 90% within the range of 30m to 60m, and obtains a high-precision 3D prediction of static obstacles.

[0133] Table 1

[0134]

[0135]

[0136] Further, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, before step 106, the method further includes: filtering the target point cloud data to remove the influence of environmental factors such as non-obstacles on the target point cloud data, and further improve the obstacle detection accuracy.

[0137] Specifically, the target point cloud data is filtered, including but not limited to the following methods:

[0138] Method 1: If the loss value of any obstacle detection frame in the target point cloud data exceeds the first loss value range, any obstacle detection frame in the target point cloud data is deleted.

[0139] Method 2: If the loss value of any boundary point in the target point cloud data exceeds the second loss value range, any boundary point in the target point cloud data is deleted.

[0140] Method three: If the size of the obstacle detection frame in the target point cloud data does not match the preset frame size corresponding to the obstacle, the obstacle detection frame is deleted.

[0141] Furthermore, if Figure 2 As shown, as a specific implementation of the above obstacle detection method, an embodiment of the present application provides an obstacle detection device 200, which includes: a detection module 201, a mapping module 202, an updating module 203 and a correction module 204.

[0142] The detection module 201 is used to obtain the first position coordinates of the obstacle in the three-dimensional coordinate system of the target object according to the point cloud data to be measured in the space where the target object is located scanned by the laser radar, wherein the points in the point cloud data to be measured correspond to the laser intensity values;

[0143] A mapping module 202 is used to convert the first position coordinates in the three-dimensional coordinate system into the station center coordinate system according to a preset conversion relationship to obtain the second position coordinates of the obstacle; and generate a thermal image according to the laser intensity values ​​corresponding to the second position coordinates and the first position coordinates;

[0144] An updating module 203 is used to update the laser intensity value of the boundary point of the obstacle in the thermal image to obtain a target intensity value;

[0145] The correction module 204 is used to correct the point cloud data to be measured according to the second position coordinates and the target intensity value in the thermal image to determine the target point cloud data;

[0146] The detection module 201 is also used to input the target point cloud data into the obstacle prediction model to obtain target obstacle information.

[0147] In this embodiment, a laser radar is used to scan the space where the target object is located to obtain at least one frame of point cloud data to be measured in the space. Obstacle information of obstacles appearing in the point cloud data to be measured is identified, and the first position coordinates of the target object in the obstacle information in the three-dimensional coordinate system are converted to the station center coordinate system through conversion processing to obtain the second position coordinates of all obstacles in at least one frame of point cloud data to be measured in the station center coordinate system. According to the laser intensity values ​​corresponding to the second position coordinates and the first position coordinates, a thermal image containing all obstacles that can be detected in at least one frame of point cloud data to be measured is generated, so that all obstacles are superimposed on the same coordinate system to form a unified reference standard for multiple obstacles. Furthermore, due to the distance difference when the laser radar scans obstacles at different positions, the laser intensity value of the boundary points of the obstacles in the thermal image is first updated to make the target intensity value of the boundary points of the obstacles in the thermal image uniform, and then the point cloud data to be tested where the obstacle appears is corrected according to the second position coordinates and target intensity value of the obstacle in the thermal image, so as to complete and delete the obstacles in each frame of the point cloud data to be tested, and finally the corrected target point cloud data is used to identify the obstacles again to obtain the target obstacle information. On the one hand, based on the characteristic that the position of static obstacles remains unchanged, the thermal image of possible obstacles in the viewing space is constructed by coordinate transformation and local mapping, and the obstacle features in the thermal image are transferred to each frame of point cloud data, so that the information of the previous and next frames can be very effectively used for FP suppression and FN completion, which solves the problem that the laser radar sensor is easily affected by the environment and missed detection in obstacle detection and measurement, improves the detection accuracy and detection range of obstacle identification, and achieves a high-precision 3D prediction effect of static obstacles. On the other hand, static obstacles can be mapped by aggregating obstacles on multi-frame point cloud data into the same coordinate system, which can obtain 3D true values ​​at longer distances and increase the detection distance of the lidar, making it easier to detect objects at longer distances.

[0148] Furthermore, the mapping module 202 is also used to convert the second position coordinates in the thermal image into a three-dimensional coordinate system according to a preset conversion relationship, so as to obtain the third position coordinates of the obstacle in the point cloud data to be measured; the correction module 204 is specifically used to perform obstacle supplementary processing on the point cloud data to be measured according to the third position coordinates to obtain the first point cloud data; and update the laser intensity value of the boundary point of the obstacle in the first point cloud data to the target intensity value; the obstacle detection device 200 also includes: an identification module (not shown in the figure), the identification module is used to identify the boundary points in the first point cloud data according to preset identification parameters and target intensity values; the correction module 204 is specifically used to edit the boundary points of the first point cloud data according to preset conditions to generate target point cloud data.

[0149] Furthermore, the correction module 204 is specifically used to determine the difference between the third position coordinates corresponding to the point cloud data to be measured and the first position coordinates; if the difference between the third position coordinates corresponding to the point cloud data to be measured and the first position coordinates is greater than or equal to the preset difference, an obstacle detection frame is added to the point cloud data to be measured according to the third position coordinates corresponding to the difference greater than or equal to the preset difference; wherein the obstacle detection frame contains the boundary points of the obstacle.

[0150] Furthermore, the preset recognition parameters include a preset precision rate; the correction module 204 is specifically used to count the first time that the boundary point in the first point cloud data of the current frame appears in the first point cloud data of the continuous frame interval to which the current frame belongs; if the first number is less than or equal to the first preset number, the boundary point in the first point cloud data of the current frame is deleted.

[0151] Furthermore, the preset recognition parameters include a preset recall rate; a correction module 204 is specifically used to determine an estimated point in the first point cloud data of the current frame whose target intensity value is within a preset intensity range; counting the second number of times that the target intensity value of the estimated point in the first point cloud data of the continuous frame interval to which the current frame belongs is greater than the maximum value in the preset intensity range; if the second number is greater than or equal to the second preset number, the estimated point is determined as a boundary point in the first point cloud data of the current frame.

[0152] Furthermore, the update module 203 is specifically used to determine the direction angle between the center of gravity points of adjacent obstacles in the thermal image according to the second position coordinates; perform clustering processing on the direction angles to determine the layout trajectory of the obstacles in the thermal image; calculate the average value of the laser intensity values ​​of the boundary points of all obstacles on the layout trajectory; and use the average value as the target intensity value of the boundary points of the obstacles on the layout trajectory.

[0153] Furthermore, the correction module 204 is also used to determine the arrangement rule of adjacent obstacles in the layout trajectory; configure the laser intensity value of the point in the thermal image that meets the arrangement rule as the target intensity value to supplement the boundary point of the obstacle.

[0154] Furthermore, the correction module 204 is further configured to delete any obstacle detection frame in the target point cloud data if the loss value of any obstacle detection frame in the target point cloud data exceeds the first loss value range.

[0155] Furthermore, the correction module 204 is further configured to delete any boundary point in the target point cloud data if the loss value of any boundary point in the target point cloud data exceeds the second loss value range.

[0156] Furthermore, the correction module 204 is further configured to delete the obstacle detection frame if the size of the obstacle detection frame in the target point cloud data does not match the preset frame size corresponding to the obstacle.

[0157] Furthermore, the detection module 201 is specifically used to input the point cloud data to be measured into the obstacle prediction model to obtain predicted obstacle information, and the predicted obstacle information includes the first position coordinates of the obstacle in the point cloud data to be measured; the obstacle detection device 200 also includes: a fusion module (not shown in the figure), the fusion module is used to, if the point cloud data to be measured includes multiple frames of point cloud data, fuse the first position coordinates corresponding to the multiple frames of point cloud data.

[0158] For the specific definition of the obstacle detection device, please refer to the definition of the obstacle detection method above, which will not be repeated here. Each module in the above obstacle detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0159] Based on the above Figure 1 The method shown in the embodiment of the present application accordingly provides a computer-readable storage medium on which a computer program is stored, which, when executed by a processor, implements the above-mentioned Figure 1 The obstacle detection method shown.

[0160] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0161] Based on the above Figure 1 The method shown, and Figure 2 In order to achieve the above-mentioned purpose, the embodiment of the present application further provides a computer device, which can be a personal computer, a server, a network device, etc. The computer device includes at least one processor; the processor is coupled to a memory, the memory stores a computer program running on the processor, and the processor is used to execute the computer program to achieve the above-mentioned Figure 1 The obstacle detection method shown.

[0162] Optionally, the computer device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Bluetooth interface, a WI-FI interface), etc.

[0163] Those skilled in the art will appreciate that the computer device structure provided in this embodiment does not limit the computer device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0164] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages and saves the hardware and software resources of the computer device, and supports the operation of information processing programs and other software and / or programs. The network communication module is used to realize communication between the components inside the storage medium, and communication with other hardware and software in the physical device.

[0165] Through the description of the above implementation methods, technical personnel in this field can clearly understand that the present application can be implemented by means of software plus necessary general hardware platforms, and can also be implemented by hardware according to the point cloud data of the target object in the space scanned by the laser radar, to obtain the first position coordinates of the obstacle in the three-dimensional coordinate system of the target object, wherein the points in the point cloud data to be measured correspond to laser intensity values; according to a preset conversion relationship, the first position coordinates in the three-dimensional coordinate system are converted to the station center coordinate system to obtain the second position coordinates of the obstacle; according to the laser intensity values ​​corresponding to the second position coordinates and the first position coordinates, a thermal image is generated; the laser intensity values ​​of the boundary points of the obstacle in the thermal image are updated to obtain the target intensity values; according to the second position coordinates and the target intensity values ​​in the thermal image, the point cloud data to be measured are corrected to determine the target point cloud data; the target point cloud data is input into the obstacle prediction model to obtain the target obstacle information. In the embodiment of the present application, on the one hand, based on the characteristic that the position of static obstacles does not change, a thermal image of possible obstacles in the viewing space is constructed by coordinate transformation and local mapping. The obstacle features in the thermal image are transferred to each frame of point cloud data, so that the information of the previous and next frames can be very effectively used for FP suppression and FN completion, which solves the problem that the obstacle detection and measurement of the laser radar sensor is easily affected by the environment and missed detection, improves the detection accuracy and detection range of obstacle identification, and achieves a high-precision 3D prediction effect of static obstacles. On the other hand, static obstacles can use mapping to aggregate obstacles on multiple frames of point cloud data in the same coordinate system, which can obtain 3D true values ​​at a longer distance, increase the detection distance of the laser radar, and facilitate the detection of objects at a longer distance.

[0166] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.

[0167] The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of this application, but this application is not limited to them, and any changes that can be thought of by technicians in this field should fall within the scope of protection of this application.

Claims

1. An obstacle detection method, characterized in that: The method comprises: According to the point cloud data to be measured in the space where the target object is located scanned by the laser radar, the first position coordinates of the obstacle in the three-dimensional coordinate system of the target object are obtained, wherein the points in the point cloud data to be measured correspond to laser intensity values; According to a preset conversion relationship, the first position coordinates in the three-dimensional coordinate system are converted to the station center coordinate system to obtain the second position coordinates of the obstacle; generating a thermal image according to the laser intensity values ​​corresponding to the second position coordinates and the first position coordinates; Updating the laser intensity value of the boundary point of the obstacle in the thermal image to obtain a target intensity value; According to the second position coordinates and the target intensity value in the thermal image, the point cloud data to be measured is corrected to determine the target point cloud data; The target point cloud data is input into an obstacle prediction model to obtain target obstacle information.

2. The obstacle detection method according to claim 1, characterized in that: The correcting process of the point cloud data to be measured according to the second position coordinates and the target intensity value in the thermal image specifically includes: According to a preset conversion relationship, the second position coordinates in the thermal image are converted into the three-dimensional coordinate system to obtain the third position coordinates of the obstacle in the point cloud data to be measured; Perform obstacle supplementation processing on the point cloud data to be measured according to the third position coordinates to obtain first point cloud data; Updating the laser intensity value of the boundary point of the obstacle in the first point cloud data to the target intensity value; Identify boundary points in the first point cloud data according to preset identification parameters and the target intensity value; The boundary points of the first point cloud data are edited according to preset conditions to generate the target point cloud data.

3. The obstacle detection method according to claim 2, characterized in that: The obstacle supplementation processing of the point cloud data to be measured according to the third position coordinates specifically includes: Determine the difference between the third position coordinates corresponding to the point cloud data to be measured and the first position coordinates; If the difference between the third position coordinates corresponding to the point cloud data to be measured and the first position coordinates is greater than or equal to a preset difference, adding an obstacle detection frame to the point cloud data to be measured according to the third position coordinates corresponding to the difference greater than or equal to the preset difference; The obstacle detection frame includes boundary points of the obstacle.

4. The obstacle detection method according to claim 2, characterized in that: The preset recognition parameters include a preset precision rate, and the editing process of the boundary points of the first point cloud data according to the preset conditions specifically includes: Counting the number of first times that a boundary point in the first point cloud data of the current frame appears in the first point cloud data of the continuous frame interval to which the current frame belongs; If the first number is less than or equal to a first preset number, the boundary point in the first point cloud data of the current frame is deleted.

5. The obstacle detection method according to claim 2, characterized in that: The preset recognition parameters include a preset recall rate, and the editing process of the boundary points of the first point cloud data according to the preset conditions specifically includes: Determine an estimated point in the first point cloud data of the current frame where the target intensity value is within a preset intensity range; Counting the second number of times that the target intensity value of the estimated point in the first point cloud data of the continuous frame interval to which the current frame belongs is greater than the maximum value in the preset intensity range; If the second number is greater than or equal to a second preset number, the estimated point is determined as the boundary point in the first point cloud data of the current frame.

6. The obstacle detection method according to claim 1, characterized in that: The updating process of the laser intensity value corresponding to the obstacle in the thermal image specifically includes: Determine the direction angle between the centroid points of adjacent obstacles in the thermal image according to the second position coordinates; Performing clustering processing on the direction angles to determine the layout trajectory of obstacles in the thermal image; Calculating the average value of the laser intensity values ​​of the boundary points of all obstacles on the layout trajectory; The average value is used as the target strength value of the boundary point of the obstacle on the layout track.

7. The obstacle detection method according to claim 6, characterized in that: The method further comprises: Determining an arrangement rule of adjacent obstacles in the layout trajectory; The laser intensity values ​​of the points in the thermal image that meet the arrangement rule are configured as the target intensity values ​​to supplement the boundary points of the obstacles.

8. The obstacle detection method according to any one of claims 1 to 7, characterized in that: The method further comprises: If the loss value of any obstacle detection frame in the target point cloud data exceeds the first loss value range, deleting any obstacle detection frame in the target point cloud data; and / or, If the loss value of any boundary point in the target point cloud data exceeds the second loss value range, deleting any boundary point in the target point cloud data; and / or, If the size of the obstacle detection frame in the target point cloud data does not match the preset frame size corresponding to the obstacle, the obstacle detection frame is deleted.

9. The obstacle detection method according to any one of claims 1 to 7, characterized in that: The step of obtaining the first position coordinates of the obstacle in the three-dimensional coordinate system of the target object according to the point cloud data to be measured in the space where the target object is located scanned by the laser radar comprises: Inputting the point cloud data to be measured into the obstacle prediction model to obtain predicted obstacle information, wherein the predicted obstacle information includes the first position coordinates of the obstacle in the point cloud data to be measured; If the point cloud data to be measured includes multiple frames of point cloud data, the first position coordinates corresponding to the multiple frames of point cloud data are fused.

10. An obstacle detection device, characterized in that: The device comprises: A detection module, used for obtaining the first position coordinates of the obstacle in the three-dimensional coordinate system of the target object according to the point cloud data to be measured in the space where the target object is located scanned by the laser radar, wherein the points in the point cloud data to be measured correspond to the laser intensity values; A mapping module, used to convert the first position coordinates in the three-dimensional coordinate system into the station center coordinate system according to a preset conversion relationship to obtain the second position coordinates of the obstacle; and generating a thermal image according to the laser intensity values ​​corresponding to the second position coordinates and the first position coordinates; An updating module, used for updating the laser intensity value of the boundary point of the obstacle in the thermal image to obtain a target intensity value; A correction module, used for correcting the point cloud data to be measured according to the second position coordinates and the target intensity value in the thermal image to determine the target point cloud data; The detection module is also used to input the target point cloud data into an obstacle prediction model to obtain target obstacle information.

11. A computer-readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by a processor, the steps of the obstacle detection method according to any one of claims 1 to 9 are implemented.

12. A computer device comprising at least one processor, the processor being coupled to a memory, the memory storing a computer program running on the processor, characterized in that: When the processor executes the program, the obstacle detection method according to any one of claims 1 to 9 is implemented.

Images