Transparent object recognition method and apparatus, and computing device

Abstract

Disclosed are a transparent object recognition method and apparatus, and a computing device, relating to the field of computers. The transparent object recognition method comprises: acquiring an obstacle in a scanning range; forming an included angle between a light beam emitted by a processing device and a tangent plane of a reference point on the obstacle; receiving a detection result for the obstacle in the process of changing from a first included angle to a second included angle; and determining a transparent object from the obstacle on the basis of the detection result for the obstacle, wherein the first included angle is different from the second included angle, and the reference point is the intersection of the light beam on the obstacle.

Description

Transparent object recognition method, device and computing equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on September 25, 2023, with application number 202311250353.2 and application name “Transparent Object Identification Method, Device and Computing Equipment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computers, and in particular to a method, apparatus, and computing device for identifying transparent objects. Background Art

[0003] Transparent object detection refers to the identification of transparent objects in the environment. Due to the high light transmittance of transparent objects and the refraction and reflection of light in transparent objects, transparent objects are difficult to be captured by sensors.

[0004] Typically, deep learning models are used to identify transparent objects in an environment. For example, when using a deep learning model, it is necessary to collect a large number of images containing transparent objects (such as glass), manually annotate them, and then train the model to obtain a trained model that can identify glass in the image. However, the use of deep learning models requires a large amount of manual annotation of the images, and the constructed deep learning model is relatively complex, resulting in low efficiency in identifying transparent objects using deep learning models.

[0005] Summary of the Invention

[0006] The present application provides a transparent object recognition method, apparatus, and computing device, which solve the problem of low efficiency in identifying transparent objects in an environment.

[0007] In a first aspect, the present application provides a method for identifying transparent objects. The method can be applied to a computer system or a processing device that supports the computer system in implementing the method. For example, the processing device can be a self-driving car, a robot, or a drone. The method can include: acquiring an obstacle within a scanning range, forming an angle between a light beam emitted by the processing device and a tangent plane at a reference point on the obstacle, and receiving obstacle detection results during the process of changing from a first angle to a second angle. Based on the obstacle detection results, a transparent object is determined from the obstacle. The first angle is different from the second angle, and the reference point is the intersection point of the light beams on the obstacle.

[0008] In this application, the process of changing the angle from the first angle to the second angle represents a continuous change in the position of the processing device relative to the obstacle. Detection results at different angles are obtained at different positions. These detection results at different angles indicate the optical properties of the obstacle. Therefore, transparent objects within the obstacle can be identified based on the optical properties of the obstacle, improving recognition accuracy. Furthermore, transparent objects within the obstacle can be identified simply by obtaining detection results at different angles at different positions, improving the efficiency of transparent object identification.

[0009] Exemplarily, the light beam emitted by the above-mentioned processing device is a laser beam emitted by a laser radar in the processing device.

[0010] In one possible implementation, detecting obstacles within the scanning range includes: a processing device emitting a light beam in all directions, receiving reflections of the light beam from obstacles, and obtaining a detection result. The detection result is then compared with a target threshold range. If the detection result is outside the target range, an obstacle is determined to be present in the scanning area.

[0011] In one possible implementation, the detection result indicates a set of point cloud intensities at multiple different angles. The point cloud intensity set includes multiple point cloud intensities, each of which indicates the reflection intensity of a light beam at an angle between a first angle and a second angle at a reference point of an obstacle.

[0012] In this application, the processing device obtains the point cloud intensity of the same reference point on the obstacle at different angles, and can fully obtain the optical properties of the obstacle. Based on the optical properties of the obstacle, it can accurately determine whether the obstacle is a transparent object, thereby improving the accuracy of transparent object recognition.

[0013] In one possible implementation, receiving obstacle detection results includes: the processing device moves according to a movement strategy to sequentially reach multiple target locations. At each target location, a laser radar transmits a light beam to a reference point on the transparent object, receives the light beam reflected from the reference point on the transparent object, and obtains the point cloud intensity at each target location at the angle between the light beam emitted by the processing device and the reference point on the transparent object.

[0014] The determination of the target position during the movement according to the movement strategy may be determined according to a granularity set by the user, or the target position may be pre-set on the movement trajectory indicated by the movement strategy.

[0015] Two possible examples are provided below for the above movement strategy.

[0016] Example 1: The processing device moves around an obstacle. During the movement of the processing device around the obstacle, the angle between the light beam emitted by the processing device and the tangent plane of the reference point changes from a first angle to a second angle.

[0017] Exemplarily, the moving of the processing device around the obstacle includes: the processing device moving clockwise around the obstacle, or the processing device moving counterclockwise around the obstacle.

[0018] In this application, the processing device moves around the obstacle to obtain point cloud intensities at different locations on the obstacle's reference point. This means obtaining point cloud intensities at different angles relative to the reference point, thereby fully determining the obstacle's optical properties. The processing device then identifies transparent objects within the obstacle based on these optical properties, improving the accuracy of transparent object recognition. Furthermore, transparent objects within the obstacle can be identified simply by obtaining detection results at different angles and at different locations, improving the efficiency of transparent object recognition.

[0019] Example 2: The processing device moves in a straight line. During the linear movement, the angle between the light beam emitted by the processing device and the tangent plane of the reference point changes from a first angle to a second angle. That is, during the linear movement, the linear movement direction does not intersect with obstacles.

[0020] In the present application, the processing device moves in a straight line, and the direction of the straight line movement does not intersect with the obstacle to obtain the point cloud intensity of the reference point at different angles. This can reduce the energy consumption of the processing device in obtaining the point cloud intensity of the reference point at different angles, and the processing device can improve the accuracy and efficiency of determining transparent objects in obstacles based on the point cloud intensity at different angles.

[0021] In one possible implementation, transparent objects are determined from obstacles based on the detection results of the obstacles, including: obtaining the similarity between the point cloud intensity set of the obstacle and the point cloud intensity set of the non-transparent object, and determining obstacles with a similarity less than or equal to a first threshold as transparent objects.

[0022] In this application, the processing device determines the transparent objects in the obstacles based on the similarity between the point cloud intensity set of the obstacles and the point cloud intensity of the non-transparent objects, and distinguishes between transparent objects and non-transparent objects based on the different optical properties between the materials, thereby improving the accuracy of identifying transparent objects.

[0023] In a possible implementation, a set of point cloud intensities at multiple different angles is a point cloud intensity curve, and the point cloud intensity curve is used to indicate a corresponding relationship between point cloud intensities and angles.

[0024] In this application, the similarity between point cloud intensity sets is converted into the similarity between point cloud intensity curves. Since the curve is insensitive to small changes, determining the similarity based on the point cloud intensity curve has better robustness and can produce consistent results even when the data changes slightly, thereby improving the accuracy of identifying transparent objects.

[0025] In one possible implementation, the similarity of a point cloud intensity set is used to indicate the similarity of properties of a point cloud intensity curve, including one or more of peak value, average width, and average gradient. Average width indicates the difference between the horizontal coordinates of two points in the point cloud intensity curve whose vertical coordinates are half the peak value, with the vertical coordinate indicating the point cloud intensity and the horizontal coordinate indicating the angle. Average gradient indicates the average value of the gradients of multiple points in the point cloud intensity curve.

[0026] In this application, the peak value, average width, and average gradient characteristics of the point cloud intensity curve characterize the shape and structure of the entire curve. Therefore, when similarity calculation is performed based on the properties of the point cloud intensity curve, it will be more robust when dealing with noise or local interference, making the results more consistent and improving the stability and accuracy of identifying transparent objects.

[0027] In one possible implementation, the transparent object recognition method further includes receiving a user-inputted set of point cloud intensities of multiple transparent materials. For the point cloud intensity sets of the multiple transparent materials, obtaining similarities between the point cloud intensity set of the transparent object and the point cloud intensity set of each transparent material. The transparent material with a similarity greater than or equal to a second threshold is determined as the material of the transparent object.

[0028] In the present application, since the point cloud intensity set of the transparent object at different incident angles is similar to the point cloud intensity set of one of the multiple transparent materials at different angles, which is greater than or equal to the threshold, the optical properties (point cloud intensity at different incident angles) of the transparent object and the transparent material are relatively consistent. In addition, it is determined that the material of the transparent object is consistent with the transparent material, and the material of the transparent object is determined using prior data (the point cloud intensity set of multiple transparent materials at different angles). While improving the accuracy of transparent object recognition, it also improves the processing efficiency of transparent object recognition. In addition, compared to the deep learning model that can only identify transparent objects, the present application can also determine the material of the transparent object based on the prior data, thereby providing data support for obstacle avoidance processing equipment and improving the success rate of obstacle avoidance.

[0029] For example, the point cloud intensity set of the transparent object is clustered with the point cloud intensity sets of multiple transparent materials, and the transparent material corresponding to the point cloud intensity set in the same cluster as the point cloud intensity set of the transparent object is used as the material of the transparent object. The similarity of the point cloud intensity sets in the same cluster is greater than or equal to a second threshold.

[0030] In one possible implementation, the transparent object identification method further includes: obtaining point cloud distance distribution data, generating an occupancy grid map based on the point cloud distance distribution data, and merging the occupancy grid map with the transparent object map to obtain a semantic map. The point cloud distance distribution data indicates the distance between any two locations within the scanning range, and the occupancy grid map indicates whether a grid in the map corresponding to the scanning range is occupied by an obstacle, with one grid corresponding to one or more pixels. The transparent object map indicates whether a grid in the map corresponding to the scanning range is occupied by a transparent object; the semantic map indicates whether a grid in the map corresponding to the scanning range is occupied by an obstacle and / or a transparent object, with the grid markings indicating the material of the transparent object.

[0031] In this application, because the processing device can accurately identify transparent objects and their materials within the scanning area, the semantic map constructed based on these objects and their materials has a high degree of accuracy. Furthermore, when the processing device navigates based on this semantic map, it can avoid both transparent and non-transparent objects, improving the safety of the processing device during movement.

[0032] In a possible implementation, the transparent object recognition method further includes: navigating using a navigation strategy based on the semantic map, wherein the navigation strategy is used to instruct: circumventing occupied grids in the semantic map.

[0033] In the present application, since the semantic map records the location of transparent objects, navigation based on the semantic map can accurately avoid transparent objects in the scanning area, thereby improving the safety of the processing equipment during movement.

[0034] For example, since the semantic map also indicates the materials of different transparent objects, corresponding detour strategies can be adopted for different transparent object materials to further ensure the safety of the processing equipment during movement.

[0035] In a possible implementation, the transparent object recognition method further includes: displaying a semantic map.

[0036] In this application, for users, the user can adjust the processing device to meet user needs based on the visualized semantic map, which is conducive to more accurate recognition of transparent objects or mapping process of the processing device.

[0037] In the second aspect, the present application also provides a transparent object recognition device, which is applied to a processing device, and the transparent object recognition device includes various modules for executing the transparent object recognition method in the first aspect or any optional method of the first aspect. For example, the transparent object recognition device includes: a first acquisition module, a receiving module and a first determination module. The first acquisition module is used to acquire obstacles within the scanning range. The receiving module is used to process the angle formed between the light beam emitted by the device and the tangent plane of the reference point on the obstacle, and receive the detection result of the obstacle during the process of changing from the first angle to the second angle; the first angle is different from the second angle, and the reference point is the intersection point of the light beam on the obstacle. The first determination module is used to determine the transparent object from the obstacle based on the detection result of the obstacle.

[0038] In one possible implementation, a light beam emitted by the processing device forms an angle with a tangent plane of a reference point on the obstacle, and the change from a first angle to a second angle is used to indicate that: during the process of the processing device moving around the obstacle, the angle between the light beam emitted by the processing device and the tangent plane of the reference point changes from the first angle to the second angle.

[0039] In a possible implementation, the moving of the processing device around the obstacle includes: the processing device moving clockwise around the obstacle, or the processing device moving counterclockwise around the obstacle.

[0040] In one possible implementation, a light beam emitted by the processing device forms an angle with a tangent plane of a reference point on the obstacle, and the change from a first angle to a second angle is used to indicate that, during the process of the processing device moving along a straight line, the angle between the light beam emitted by the processing device and the tangent plane of the reference point changes from the first angle to the second angle.

[0041] In one possible implementation, the detection result is used to indicate a set of point cloud intensities at multiple different angles. The point cloud intensity set includes multiple point cloud intensities, each of which indicates the reflection intensity of a light beam at an angle between a first angle and a second angle at a reference point of an obstacle.

[0042] In a possible implementation, the first determination module is specifically configured to obtain a similarity between a point cloud intensity set of an obstacle and a point cloud intensity set of a non-transparent object, and determine an obstacle having a similarity less than or equal to a first threshold as a transparent object.

[0043] In a possible implementation, a set of point cloud intensities at multiple different angles is a point cloud intensity curve, and the point cloud intensity curve is used to indicate a corresponding relationship between point cloud intensities and angles.

[0044] In one possible implementation, the similarity of a point cloud intensity set is used to indicate the similarity of properties of a point cloud intensity curve. The properties of the point cloud intensity curve include one or more of peak value, average width, and average gradient. The average width indicates the difference between the horizontal coordinates of two points in the point cloud intensity curve whose vertical coordinates are half the peak value, with the vertical coordinate indicating the point cloud intensity and the horizontal coordinate indicating the angle. The average gradient indicates the average value of the gradients of multiple points in the point cloud intensity curve.

[0045] In one possible implementation, the transparent object recognition device further includes: a similarity acquisition module and a second determination module. The similarity acquisition module is configured to obtain, from point cloud intensity sets for multiple transparent materials, similarities between the point cloud intensity set of the transparent object and the point cloud intensity set of each transparent material. The second determination module is configured to determine, among the multiple similarities, the transparent material with a value greater than or equal to a second threshold as the material of the transparent object.

[0046] In one possible implementation, the transparent object recognition device further includes: a second acquisition module, a generation module, and a merging module. The second acquisition module is used to acquire point cloud distance distribution data, and the point cloud distance distribution data is used to indicate the distance between any two positions within the scanning range. The generation module is used to generate an occupied grid map based on the point cloud distance distribution data; the occupied grid map is used to indicate whether the grid in the map corresponding to the scanning range is occupied by obstacles, and one grid corresponds to one or more pixels. The merging module is used to merge the occupied grid map with the transparent object map to obtain a semantic map. The transparent object map is used to indicate whether the grid in the map corresponding to the scanning range is occupied by transparent objects; the semantic map is used to indicate whether the grid in the map corresponding to the scanning range is occupied by obstacles and / or transparent objects, and the grid mark is used to indicate the material of the transparent object.

[0047] In a possible implementation, the transparent object recognition device further includes a navigation module configured to perform navigation based on the semantic map using a navigation strategy, wherein the navigation strategy is configured to instruct the user to navigate around occupied grids in the semantic map.

[0048] In a possible implementation, the transparent object recognition device further includes: a display module. The display module is configured to display the semantic map.

[0049] In a third aspect, the present application also provides a chip comprising: an interface circuit and a power supply circuit; the interface circuit is used to acquire obstacles within a scanning range, and the control circuit is used to execute the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0050] In a fourth aspect, the present application provides a computing device. The computing device includes a memory and a processor. The memory is configured to store computer instructions. When the processor executes the computer instructions, it implements the method described in the first aspect or any possible implementation of the first aspect. The computing device may be a self-driving car, a robot, or a drone.

[0051] In one possible implementation, the computing device also includes a lidar and a motion component.

[0052] In a fifth aspect, the present application further provides a computer-readable storage medium having a computer program or instructions stored therein, which, when executed by a computing device, implements the method of the first aspect or any optional implementation of the first aspect.

[0053] In a sixth aspect, the present application further provides a computer program product, which includes a computer program or instructions, and when executed by a computing device, implements the method in the first aspect or any optional implementation of the first aspect.

[0054] The beneficial effects of the above second to sixth aspects can be referred to the description of the first aspect or any implementation method of the first aspect, and will not be repeated here.

[0055] Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a schematic diagram of a real image and an annotated mask;

[0057] Figure 2 is a schematic diagram of the structure of a deep learning network model;

[0058] Figure 3 is a schematic diagram of laser radar identifying transparent objects;

[0059] FIG4 is an application scenario diagram 1 of a computer system provided by the present application;

[0060] FIG5 is a second application scenario diagram of a computer system provided by the present application;

[0061] FIG6 is a flow chart of a transparent object recognition method provided by the present application;

[0062] FIG7 is a schematic diagram of the positions of obstacles and processing equipment provided by this application;

[0063] FIG8 is a schematic diagram of the movement trajectory of the processing device provided in this application;

[0064] FIG9 is a schematic diagram of a point cloud intensity curve of an obstacle provided by this application;

[0065] FIG10 is a flow chart of the mapping method provided in this application;

[0066] FIG11 is a structural schematic diagram 1 of a transparent object recognition device provided by the present application;

[0067] FIG12 is a second structural diagram of a transparent object recognition device provided by the present application;

[0068] FIG13 is a schematic diagram of the structure of a computing device provided in this application. DETAILED DESCRIPTION

[0069] To facilitate understanding, the technical terms involved in this application are first introduced.

[0070] Point cloud intensity, also known as radar intensity, is a measurement of the intensity of a laser radar pulse echo at a specific point. This value reflects the reflection intensity (reflectivity) or echo strength of the object (a specific point or point cloud) scanned by the laser radar pulse. A point cloud is a three-dimensional spatial data set composed of a large number of discrete points. Each point has its coordinate information in three-dimensional space, which can be used to determine the position information between each point in the point cloud.

[0071] Gmapping and Cartographer are both lidar mapping and positioning algorithms that enable devices to locate and build maps in the environment in real time to ultimately obtain a map.

[0072] Due to the high light transmittance of transparent objects and the refraction and reflection characteristics of light within them, optical sensors (cameras, radars) have limited field of view when perceiving transparent objects, making it difficult to identify them. Consequently, in scenarios such as autonomous driving and robotic operations, where transparent objects are abundant (e.g., in supermarkets and office areas), accidents and collisions are highly likely to occur, posing a serious safety hazard.

[0073] To identify transparent objects in the environment, two possible processing solutions are provided below.

[0074] Solution 1: Camera glass detection solution based on deep learning.

[0075] First, users need to collect a large number of images containing glass and manually annotate the glass masks. Figure 1 shows a schematic diagram of real images and annotated masks. Figure 1a shows various types of real images (glass images) in a supermarket scenario, and Figure 1b shows the corresponding masks for various types of glass images. The white area in the mask represents glass, and the black area represents other objects outside the glass.

[0076] Furthermore, the above-mentioned pictures containing glass and the corresponding masks are used to train a deep learning model so that the model has the ability to recognize and segment glass.

[0077] Figure 2 shows a schematic diagram of the structure of a deep learning network model. Figure 2 shows a GDNet (Glass Detection Dataset and Design a Glass Detection Network). The GDNet architecture includes a multi-level feature extractor, a large contextual feature integration (LCFI) module, convolutional layers, normalization layers, activation function layers for activation processing, and spatially separable convolutional layers. Inputting an image of glass into the GDNet shown in Figure 2 yields the region where the glass is located.

[0078] However, when constructing the training set (including glass images and corresponding masks), a large number of images must be manually acquired, which is costly and inefficient. Furthermore, constructing the GDNet structure shown in Figure 2 is complex, costly, and inefficient.

[0079] Solution 2: Glass detection solution based on lidar.

[0080] 1) As shown in Figure 3, Figure 3 is a schematic diagram of laser radar identifying transparent objects. When the laser radar is vertically directed at the glass, the intensity of the point cloud obtained is the largest, and then when the device moves horizontally along the glass, the intensity of the point cloud remains at the largest (i.e., the device moves from R t Moving to R t+1 , the corresponding device emits a beam of light that illuminates the point on the object from O t ` , move to O t+1 ` ). When the above situation occurs, the object is considered to be glass.

[0081] Because the aforementioned devices cannot determine the orientation of objects, manual control of the device's movement is required to identify whether the object is glass. Furthermore, if a dynamically moving object appears in the scene (such as an electric door covered in advertisements at a supermarket entrance), the object will continuously move with the device and could be mistaken for a transparent object.

[0082] 2) Multiple lidar parameters of a single laser beam irradiated by the lidar on a transparent object are input into a deep learning model to determine whether the transparent object is glass.

[0083] For example, the rotation angle, pitch angle, length, and intensity of the laser beam are input into a glass recognition deep neural network to identify the probability of a radar point being glass. Furthermore, within a spatial voxel map containing the glass probability and glass normal vector, a ray casting method is used to simulate the rotation angle and pitch angle of the laser beam emitted by the lidar when the robot is in its initial position. These are then input into a deep neural network for optical properties to obtain the probability of laser beam transmission. The radar points generated by the reflected laser beam are added to the point cloud to generate a simulated point cloud set. Based on real 3D radar data and the simulated point cloud set, a normal distribution transformation is used to calculate the robot's absolute position within the spatial voxel map containing glass probability and glass direction information.

[0084] Because a deep neural network uses only the parameters of a single laser beam as input, determining whether an object is glass has limitations, resulting in low recognition accuracy. Furthermore, the use of deep neural networks requires collecting large amounts of training data and has a complex network structure, making training a deep neural network expensive and inefficient.

[0085] In summary, the above processing scheme has low overall efficiency when identifying transparent objects, and has low accuracy in judging the material of transparent objects.

[0086] Based on this, the present application provides a transparent object identification method. This transparent object identification method is applied to a processing device and includes: obtaining an obstacle within a scanning range, forming an angle between a light beam emitted by the processing device and a tangent plane at a reference point on the obstacle, and receiving obstacle detection results during the process of changing from a first angle to a second angle. Based on the obstacle detection results, a transparent object is determined within the obstacle. The first angle is different from the second angle, and the reference point is the intersection point of the light beams on the obstacle.

[0087] In this application, the process of changing the angle from the first to the second angle indicates that the position of the processing device relative to the obstacle is constantly changing. Detection results at different angles are obtained at different positions. These detection results at different angles indicate the optical properties of the obstacle. Therefore, transparent objects within the obstacle can be identified based on the optical properties of the obstacle, improving recognition accuracy. Furthermore, transparent objects within the obstacle can be identified simply by obtaining detection results at different angles at different positions, improving the efficiency of transparent object identification.

[0088] Next, the transparent object recognition method provided by the present application is described in detail with reference to the accompanying drawings. The transparent object recognition method provided by the present application can be applied to the scene shown in FIG4 or FIG5.

[0089] In the first scenario, as shown in Figure 4, which is an application scenario diagram of a computer system provided by this application, computing device 410 can scan objects within an area, which can be referred to as a scanning range. After computing device 410 identifies obstacle 420 within its scanning range, computing device 410 can move around obstacle 420 to obtain detection results at different angles between the light beam emitted by computing device 410 and the tangent plane of a reference point on obstacle 420. Furthermore, based on the detection results at different angles, transparent objects within obstacle 420 can be determined.

[0090] The reference point can be a position arbitrarily selected by computing device 410 on obstacle 420, or a position specified by the user on obstacle 420. Furthermore, while moving, computing device 410 continuously emits a light beam, directing the light beam toward the reference point on obstacle 420. Alternatively, while moving, computing device 410 emits a light beam each time it reaches a target position, directing the light beam toward the reference point on obstacle 420. In other words, after determining the reference point on obstacle 420, the reference point is the intersection of the light beams emitted by computing device 410 on the obstacle.

[0091] For example, a user may specify a location on a display interface displayed by computing device 410. This display interface indicates obstacles within the scanning area. The user may then trigger an operation on the display interface, and computing device 410 determines the user-specified location on the obstacle based on the user's trigger operation. The trigger operation may be a click or a slide. The light beam emitted by computing device 410 may be a laser beam emitted by a lidar deployed in computing device 410.

[0092] The tangent plane to a reference point on an obstacle represents the curve passing through the reference point on the obstacle surface. If tangent lines exist, they lie in the same plane, and this plane is the tangent plane to the reference point on the obstacle. The tangent line is a straight line that just touches the reference point on the curve. In other words, when the tangent line passes through the reference point on the curve, the direction of the tangent line is the same as the direction of the reference point on the curve.

[0093] In a possible example, the tangent plane may also be referred to as a tangent plane or a local plane. The tangent plane or the local plane may represent a curve passing through a reference point on the obstacle surface, and if a tangent exists, the plane where the tangent lies.

[0094] In one possible scenario, computing device 410 may move clockwise around obstacle 420 .

[0095] Exemplarily, the computing device 410 moves clockwise, that is, passes through point ① and point ② in sequence, and obtains the detection results of the computing device 410 between point ① and point ② at the angle between the light beam emitted by the computing device 410 and the tangent plane of the reference point on the obstacle 420.

[0096] In another possible scenario, computing device 410 may move counterclockwise around obstacle 420 .

[0097] Exemplarily, the computing device 410 moves counterclockwise, that is, passes through point ② and point ① in sequence, and obtains the detection results between point ② and point ① at the angle between the light beam emitted by the computing device 410 and the tangent plane of the reference point on the obstacle 420.

[0098] It is worth noting that, as shown in Figure 4, if the reference point of the obstacle 420 is located on the right side of the cube-shaped obstacle 420, the tangent plane of the reference point is the right side. The movement of the computing device 410 around the obstacle 420 can be: the computing device 410 moves around the center of the obstacle 420, or the computing device 410 moves around the reference point. When the computing device 410 rotates around the obstacle 420, it only needs to rotate 180° around the tangent plane of the reference point, and this application does not limit the rotation angle. In other embodiments of the present application, the rotation angle may be 90° or 120°, etc. The size of the scanning range is determined according to the processing power of the computing device 410, specifically, according to the power, receiving sensitivity, etc. of the laser radar in the computing device 410.

[0099] In this scenario, the computing device 410 may also create a map based on obstacles 420 within the scanning range and perform navigation based on the obtained map.

[0100] In the second scenario, as shown in Figure 5, which illustrates the second application scenario of a computer system provided by this application, computing device 410 identifies obstacle 420 within its scanning range and then moves in a straight line, with the direction of linear movement not intersecting the reference point on obstacle 420 or obstacle 420 itself. This allows for detection results to be obtained at different angles between the light beam emitted by computing device 410 and the tangent plane of the reference point on obstacle 420. Furthermore, based on the detection results at different angles, the presence of a transparent object within obstacle 420 is determined.

[0101] As shown in Figure 5, computing device 410 can move in a straight line from point ① to point ②, or vice versa. During the movement of computing device 410, the angle between the emitted light beam and the tangent plane of the reference point on obstacle 420 continuously changes, thereby obtaining detection results of obstacle 420 at different angles.

[0102] This application provides a method for identifying transparent objects, which can be applied in the application scenarios shown in Figures 4 or 5. The following description uses the application scenario shown in Figure 4 as an example. As shown in Figure 6, Figure 6 is a flow chart of a transparent object identification method provided in this application. This transparent object identification method can be executed by a processing device 610. The processing device 610 may include the computing device 410 shown in Figure 4, which is equipped with a laser radar. As shown in Figure 6, the transparent object identification method may include the following steps S610 to S630.

[0103] S610 : The processing device 610 obtains obstacles 420 within a scanning range.

[0104] In one possible implementation, the processing device 610 obtains the obstacle 420 within the scanning range, including: the processing device 610 emits a light beam to obtain a detection result of the object within the scanning range, and determines the obstacle 420 within the scanning range according to the detection result of the object.

[0105] In one possible example, during normal movement, processing device 610 continuously emits light beams in all directions around it. If an object is within the scanning range, processing device 610 receives the object's reflection of the light beam, i.e., a detection result. Processing device 610 compares this detection result with a target threshold range. If the detection result is outside the target threshold range, the object is determined to be an obstacle 420 (suspected transparent object). If the detection result is within the target threshold range, the object is determined to be a non-transparent object.

[0106] If the detection result is at the two end values ​​of the target threshold range, the object is determined to be a suspected transparent object or a non-transparent object, which is not limited in this application.

[0107] It's worth noting that the scanning range illustrated in Figure 6 represents only a portion of the complete scanning range of processing device 610. In practice, the complete scanning range of processing device 610 is defined as an area radiating from the location of processing device 610. The extent of this area is determined by factors such as the power of the LiDAR in processing device 610 and the degree of obstruction from surrounding obstacles.

[0108] As shown in Figure 7, Figure 7 is a schematic diagram of the positions of obstacles and processing devices provided by this application. Figure 7a shows that when processing device 610 is in one position, the light beam emitted by processing device 610 forms multiple angles with the surface of an object. The detection intensities corresponding to these multiple angles are the detection intensities of the object. Processing device 610 sequentially compares the detection intensities corresponding to these multiple angles with the target threshold range to determine whether the object is an obstacle.

[0109] In one possible scenario, the processing device 610 may determine the detection strength of the object at multiple locations, and determine whether the object is an obstacle based on the detection strength at the multiple locations.

[0110] Exemplarily, the light beam emitted by the processing device 610 is a laser beam, and the detection intensity is the point cloud intensity.

[0111] In the process of identifying obstacles within the scanning range, the processing device 610 will also determine the relative position of the obstacle and the processing device 610 based on the point cloud intensity of the obstacle. The relative position may include the distance and angle between the obstacle and the processing device 610.

[0112] Since the point cloud intensity indicates the coordinate position of each point in the point cloud, which is determined based on the processing device 610 , the processing device 610 can determine the relative position of the obstacle and the processing device 610 based on the coordinate position of each point.

[0113] Illustratively, the coordinates of each point in the point cloud are determined with the processing device 610 that emits the light beam as the origin.

[0114] S620: The light beam emitted by the processing device 610 forms an angle with the tangent plane of the reference point on the obstacle 420. During the process of changing from the first angle to the second angle, a detection result of the obstacle 420 is received.

[0115] The first angle is different from the second angle, and the reference point is the intersection point of the light beams on the obstacle.

[0116] The difference between the first angle and the second angle can be 180°. In other embodiments of the present application, the difference can be 90° or 120°, and this application is not limited thereto. The reference point on the obstacle can be any point on the outer surface of the obstacle where the light beam emitted by the processing device 610 hits, or the reference point can be a position specified by the user on the outer surface of the obstacle.

[0117] In one possible implementation, the processing device 610 moves according to the relative position of the processing device 610 and the obstacle 420, and the moving direction does not intersect with the obstacle 420, so that the angle between the light beam emitted by the processing device 610 and the tangent plane of the reference point on the obstacle 420 changes from the first angle to the second angle, and the detection result of the obstacle 420 is received.

[0118] As shown in FIG6 , the processing device 610 moves from point a to point c via point b. During the process of moving from point a to point c, the angle between the light beam emitted by the processing device and the tangent plane of the reference point on the obstacle 420 continuously changes, that is, during the process of changing from the first angle to the second angle, the processing device 610 continuously receives the detection results of the obstacle 420.

[0119] Similarly, as shown in FIG7(b), obstacle 420 is a regular cubic structure. Therefore, the tangent plane of the reference point on the obstacle is the plane where the reference point lies. As processing device 610 passes through multiple points, the angle between the light beam emitted by processing device 610 and the tangent plane of the reference point on obstacle 420 continuously changes. For example, angle a changes to angle b, and this angle continuously increases as processing device 610 continues to move.

[0120] As shown in c in FIG. 7 , obstacle 420 is an arc-shaped structure. Based on the reference point on obstacle 420, a tangent plane to the reference point on obstacle 420 is determined. As processing device 610 moves around obstacle 420, the angle between the light beam emitted by processing device 610 and the tangent plane to the reference point on obstacle 420 continuously changes. For example, the angle continuously changes starting from angle c.

[0121] It is worth noting that, since the obstacle 420 shown in FIG. 7 c is an arc-shaped structure, the tangent plane of the reference point on the obstacle 420 is perpendicular to the line connecting the center of the arc-shaped structure and the reference point.

[0122] The detection result indicates a set of point cloud intensities at multiple different angles. The point cloud intensity set includes multiple point cloud intensities, each of which indicates the reflection intensity of the light beam at an angle between a first angle and a second angle at a reference point of obstacle 420 .

[0123] For example, as processing device 610 moves and the angle continuously changes from the first angle to the second angle, processing device 610 continuously emits light beams, obtaining the point cloud intensity at the reference point on obstacle 420 at each angle. That is, the point cloud intensity set includes multiple sets of angle and point cloud intensity data, such as (20°, 60°).

[0124] In the present application, the processing device 610 obtains the point cloud intensity of the same reference point on the obstacle 420 at different angles, and can fully obtain the optical properties of the obstacle 420. Based on the optical properties of the obstacle 420, it can accurately determine whether the obstacle 420 is a transparent object, thereby improving the accuracy of transparent object recognition.

[0125] When the processing device is at the starting position (point a), the angle between the light beam emitted by processing device 610 and the tangent plane of the reference point on obstacle 420 is the first angle. When the processing device is at the ending position (point b), the angle between the light beam emitted by processing device 610 and the tangent plane of the reference point on obstacle 420 is the second angle. The aforementioned starting and ending positions are relative to a single obstacle. Different obstacles may correspond to different starting and ending positions.

[0126] In one possible example, processing device 610 moves according to a movement strategy and sequentially reaches multiple target locations. At each target location, processing device 610 emits a light beam to a reference point on obstacle 420, receives the light beam reflected by the reference point on obstacle 420, and obtains the point cloud intensity of the light beam emitted by processing device 610 at the reference point below obstacle 420 at each target location.

[0127] The target location of the processing device 610 during movement according to the movement strategy can be determined based on a granularity set by the user. For example, the target location is the location reached by the processing device 610 every 10 cm it moves according to the movement strategy. Alternatively, the target location can be pre-set along the movement trajectory indicated by the movement strategy, which is not limited in this application.

[0128] In one possible scenario, the plane in which the processing device 610 forms a trajectory according to the movement strategy may be parallel to the horizontal plane, or may be at a certain angle to the horizontal plane (for example, when the obstacle is on a slope, the processing device 610 may move up and down the slope, and thus the movement trajectory of the processing device 610 may be at a certain angle to the horizontal plane).

[0129] With respect to the above-mentioned movement strategy, a possible embodiment is provided below.

[0130] In a possible embodiment, when the processing device 610 moves around the obstacle 420 , the angle formed by the light beam emitted by the processing device 610 and the tangent plane of the reference point changes from a first angle to a second angle.

[0131] As shown in Figure 4 , processing device 610 moves around obstacle 420. With each movement of processing device 610, the angle between the light beam emitted by processing device 610 and the tangent plane of the reference point on obstacle 420 changes. When processing device 610 moves from point ① to point ②, the angle between the light beam emitted by processing device 610 and the tangent plane of the reference point changes from a first angle to a second angle.

[0132] In a possible scenario, the processing device 610 may move around the obstacle 420 in a clockwise direction or in a counterclockwise direction, which is not limited in this application.

[0133] For example, the shape of the trajectory of the processing device 610 moving around the obstacle 420 may be a curve or a multi-fold line, wherein the opening of the curve or the multi-fold line faces the obstacle 420 .

[0134] As shown in Figure 8, Figure 8 is a schematic diagram of the movement trajectory of the processing equipment provided by this application. In Figure 8, trajectory 1 is an arc, i.e., a curve, trajectory 2 is an irregular curve, and trajectory 3 is a multi-fold line. The openings of trajectory 1, trajectory 2, and trajectory 3 all face obstacle 420.

[0135] In this application, processing device 610 moves around obstacle 420 to obtain point cloud intensities at reference points on obstacle 420 at different locations. Specifically, it obtains point cloud intensities at different angles to the reference points, thereby fully determining the optical properties of obstacle 420. Processing device 610 determines transparent objects within obstacle 420 based on these optical properties, thereby improving the accuracy of transparent object recognition.

[0136] S630 : The processing device 610 determines a transparent object from the obstacles 420 according to the detection result of the obstacles 420 .

[0137] In one possible implementation, processing device 610 determines transparent objects from obstacle 420 based on the detection result of obstacle 420, including obtaining a similarity between a point cloud intensity set of obstacle 420 and a point cloud intensity set of a non-transparent object. Obstacles 420 with a similarity less than or equal to a first threshold are determined to be transparent objects.

[0138] For example, the processing device 610 may use Euclidean distance, Manhattan distance, Chebyshev distance, etc. to determine the similarity of the point cloud intensity sets.

[0139] The processing device 610 uses Euclidean distance, Manhattan distance, and Chebyshev distance to determine the distance between data, and uses the distance to measure the similarity between data.

[0140] Taking the Euclidean distance, among the Euclidean distance, Manhattan distance, and Chebyshev distance, used by processing device 610 as an example, processing device 610 uses the Euclidean distance to calculate the distance between the point cloud intensity set of obstacle 420 and the point cloud intensity set of non-transparent objects at the same angle. Based on the distance, processing device 610 then obtains the similarity of the point cloud intensities at the same angle. Processing device 610 performs a weighted summation of the similarities of the corresponding intensities at each angle to obtain the similarity between the point cloud intensity sets.

[0141] In the present application, the processing device 610 determines the transparent objects in the obstacle 420 based on the similarity between the point cloud intensity set of the obstacle 420 and the point cloud intensity of the non-transparent object, thereby distinguishing between transparent objects and non-transparent objects based on the different optical properties between the materials, thereby improving the accuracy of identifying transparent objects.

[0142] In another embodiment of the present application, the transparent object recognition method is applied to the application scenario shown in Figure 5 as an example. The content of this embodiment is different from that shown in Figure 6 above only in that the movement strategy adopted by the processing device 610 in this embodiment is linear movement.

[0143] In a possible implementation, when the processing device 610 moves along a straight line, the angle formed by the light beam emitted by the processing device and the tangent plane of the reference point changes from a first angle to a second angle.

[0144] As shown in Figure 5, processing device 610 moves in a straight line, and the direction of linear movement does not intersect obstacle 420. This ensures that the angle between the light beam emitted by processing device 610 and the tangent plane at the reference point on obstacle 420 changes with each distance the processing device 610 moves. When processing device 610 moves from point ① to point ②, the angle between the light beam emitted by processing device 610 and the tangent plane at the reference point changes from a first angle to a second angle.

[0145] In one possible scenario, when the processing device 610 moves in a straight line, the direction of the straight line movement does not intersect with the reference point on the obstacle 420 .

[0146] In the present application, the processing device 610 moves in a straight line, and the direction of the straight line movement does not intersect with the obstacle 420, to obtain the point cloud intensity of the reference point at different angles. This can reduce the energy consumption of the processing device 610 in obtaining the point cloud intensity of the reference point at different angles, and the processing device 610 can improve the accuracy of determining transparent objects in the obstacle 420 based on the point cloud intensity at different angles.

[0147] In a possible embodiment, the set of point cloud intensities at different angles is a point cloud intensity curve, which is used to indicate the corresponding relationship between point cloud intensities and angles.

[0148] When the processing device 610 determines a transparent object from the obstacles 420 based on the detection result of the obstacles 420 , the similarity between the point cloud intensity curve of the obstacle 420 and the point cloud intensity curve of the non-transparent object may be used.

[0149] As shown in FIG9 , FIG9 is a schematic diagram of the point cloud intensity curve of the obstacle provided by the present application. FIG9 a shows the point cloud intensity curve of a transparent object and the point cloud intensity curve of a non-transparent object. The point cloud intensity of a transparent object changes parabolically with changes in the angle. When the light beam of the processing device 610 and the tangent plane of the reference point are at 90°, the point cloud intensity reaches a peak. The point cloud intensity of a non-transparent object does not fluctuate much with changes in the angle, and fluctuates within a certain range.

[0150] The processing device 610 may use Fréchet distance, dynamic time warping (DTW), etc. to determine the similarity between the point cloud intensity curve of the obstacle 420 and the point cloud intensity curve of the non-transparent object.

[0151] The Fréchet distance is defined as the maximum value obtained by selecting a point on one curve and a point on the other curve, then connecting the two points to form a connecting line segment. The position and length of the connecting line segment are constrained by the order of the points on the two curves. The Fréchet distance is the maximum value that minimizes the connecting line segment obtained by this process. The smaller the Fréchet distance, the more similar the two curves are.

[0152] The basic idea of ​​DTW is to align two time series (curves) to minimize the distance between them while satisfying strict monotonicity and smoothness constraints. Specifically, DTW calculates the distance matrix between the two series and uses dynamic programming to find the optimal path, providing a measure of the similarity between the two series.

[0153] In the present application, the processing device 610 converts the similarity between point cloud intensity sets into the similarity between point cloud intensity curves. Since the curve is more sensitive to slight changes, determining the similarity based on the point cloud intensity curve has better robustness and can produce consistent results even when the data changes slightly, thereby improving the accuracy of identifying transparent objects.

[0154] In a possible implementation, the similarity of the point cloud intensity sets is used to indicate the similarity of properties of the point cloud intensity curves, where the properties of the point cloud intensity curves include one or more of peak value, average width, and average gradient.

[0155] The average width indicates the difference between the horizontal coordinates of two points in the point cloud intensity curve whose vertical coordinates are half the peak value. The vertical coordinate indicates the point cloud intensity, and the horizontal coordinate indicates the angle. The average gradient indicates the average value of the gradients of multiple points in the point cloud intensity curve.

[0156] Figure 9(b) shows three properties of the point cloud intensity curve (peak value, average width, and average gradient). The processing device 610 determines the average gradient of the point cloud intensity curve by selecting multiple points from the point cloud intensity curve based on a sampling granularity determined by the user, and then determining the average value of the gradients of these multiple points. For example, the sampling granularity may be sampling every 1°, which is not limited in this application. In other embodiments of the present application, the sampling granularity may also be sampling every 2°.

[0157] Exemplarily, the processing device 610 compares the attributes of the point cloud intensity curve of the obstacle 420 with the attributes of the point cloud intensity curve of the non-transparent object one by one to determine the similarity.

[0158] For example, processing device 610 compares peak value a in the attribute of the point cloud intensity curve of obstacle 420 with peak value b in the attribute of the point cloud intensity curve of the non-transparent object. If peak value a satisfies the condition that it is greater than or equal to peak value b plus a first value, then the similarity between peak value a and peak value b is determined to be less than or equal to a threshold value. This first value can be 10, which can be set according to user needs and is not limited by this application.

[0159] Processing device 610 compares the average width a in the attribute of the point cloud intensity curve of obstacle 420 with the average width b in the attribute of the point cloud intensity curve of the non-transparent object. If average width a is less than or equal to average width b minus a second value, then the similarity between average width a and average width b is determined to be less than or equal to a threshold. This second value can be 5 and can be set according to user needs and is not limited by this application.

[0160] Processing device 610 compares the average gradient a in the attribute of the point cloud intensity curve of obstacle 420 with the average gradient b in the attribute of the point cloud intensity curve of the non-transparent object. If average gradient a satisfies the condition that it is greater than or equal to average gradient b plus a third value, then the similarity between average gradient a and average gradient b is determined to be less than or equal to a threshold. This third value can be 1 and can be set according to user needs and is not limited by this application.

[0161] In summary, the processing device 610 performs weighted summation based on the similarities among the peak values, average widths, and average gradients to obtain the similarity between the point cloud intensity set of the obstacle 420 and the point cloud intensity set of the non-transparent object.

[0162] Exemplarily, if the similarity between the peak value a and the peak value b is less than or equal to the threshold, the processing device 610 assigns a value of 0 to the peak value of the point cloud curve of the obstacle. If the similarity between the average width a and the average width b is less than or equal to the threshold, the processing device 610 assigns a value of 0 to the average width of the point cloud curve of the obstacle. If the similarity between the average gradient a and the average gradient b is less than or equal to the threshold, the processing device 610 assigns a value of 0 to the average gradient of the point cloud curve of the obstacle. The processing device 610 determines that the similarity between the attributes of the point cloud intensity curve of the obstacle and the attributes of the point cloud intensity curve of the non-transparent object is 0 based on the weights of the peak value, average width, and average gradient (0.4 / 0.2 / 0.4). When the similarity between the aforementioned attributes is less than or equal to a fourth value, it is determined that the obstacle is a non-transparent object or a transparent object. The fourth value can be 0.2, and the fourth value can be set according to user needs, and is not limited in this application.

[0163] In the present application, the peak value, average width, and average gradient characteristics of the point cloud intensity curve characterize the shape and structure of the entire curve. Therefore, when the processing device 610 performs similarity calculations based on the properties of the point cloud intensity curve, it will be more robust when processing noise or local interference, making the results more consistent and improving the stability and accuracy of identifying transparent objects.

[0164] To solve the problem that the above neural network model can only identify transparent objects but cannot confirm the material of the transparent objects, a possible embodiment is provided below to identify the material of the transparent objects.

[0165] The processing device 610 obtains point cloud intensity sets of multiple transparent materials input by the user. The processing device 610 obtains similarities between the point cloud intensity set of the transparent object and the point cloud intensity set of each transparent material, and selects the transparent material with a similarity greater than or equal to a second threshold as the material of the transparent object.

[0166] In one possible scenario, the point cloud intensity set of the plurality of transparent materials input by the user obtained by the processing device 610 may be a point cloud intensity curve or an attribute of the point cloud intensity curve.

[0167] In a possible example, the processing device 610 may determine the similarity using a neural network model, DTW, etc. The neural network model may be a recurrent neural network (RNN), a temporal convolutional neural network (TCN), etc.

[0168] The material of the transparent object can be transparent plastic, glass, frosted glass, water tank, etc.

[0169] In the present application, since the point cloud intensity set of the transparent object at different incident angles is similar to the point cloud intensity set of one of the multiple transparent materials at different angles, which is greater than or equal to the threshold, the optical properties (point cloud intensity at different incident angles) of the transparent object and the transparent material are relatively consistent. In addition, it is determined that the material of the transparent object is consistent with the transparent material, and the material of the transparent object is determined using prior data (the point cloud intensity set of multiple transparent materials at different angles). While improving the accuracy of transparent object recognition, it also improves the processing efficiency of transparent object recognition. In addition, compared to the deep learning model that can only identify transparent objects, the present application can also determine the material of the transparent object based on the prior data, thereby providing data support for obstacle avoidance processing equipment and improving the success rate of obstacle avoidance.

[0170] Regarding the content of determining similarity in the above embodiment, two possible implementations are provided below.

[0171] In a first possible implementation manner, the processing device 610 calculates the similarity between the point cloud intensity curve of the transparent object and the point cloud intensity curve of each transparent material.

[0172] For example, the processing device 610 may use the above-mentioned neural network model to determine the similarity between the point cloud intensity curve of the transparent object and the point cloud intensity curve of each material.

[0173] In a second possible implementation manner, the processing device 610 calculates the similarity between the attributes of the point cloud intensity curve of the transparent object and the attributes of the point cloud intensity curve of each material.

[0174] For example, the processing device 610 may cluster the properties of the transparent object's point cloud intensity curve with the properties of the point cloud intensity curves of multiple transparent materials to obtain a clustering result. The clustering result is obtained based on the similarity between the properties of the transparent object's point cloud intensity curve and the properties of the point cloud intensity curves of the multiple transparent materials. In the clustering result, the similarity of the properties of the point cloud intensity curves in the same cluster is greater than or equal to a threshold.

[0175] For example, the processing device 610 uses the peak value, average width, and average gradient of the point cloud intensity curve to represent the characteristics of the point cloud intensity curve, showing the three-dimensional properties of the curve. The processing device 610 performs aggregation according to the characteristics of the point cloud intensity curve, which can improve the accuracy of the clustering results.

[0176] The processing device 610 may cluster the attributes of the above-mentioned point cloud intensity curve using K-means, hierarchical clustering, or density clustering algorithms.

[0177] In one possible embodiment, the present application creates a map based on the transparent objects and their materials identified in the scanned area by the processing device 610. As shown in FIG10 , FIG10 is a flow chart of the mapping method provided by the present application. The mapping method can be executed by the processing device 610 and may include the following steps S1010 to S1030.

[0178] S1010 : The processing device 610 obtains point cloud distance distribution data.

[0179] The point cloud distance distribution data is used to indicate the distance between any two positions within the scanning range.

[0180] For example, the processing device 610 may obtain point cloud distance distribution data within the scanning range during normal movement, movement around the obstacle 420 , or movement along a straight line.

[0181] The description of the point cloud distance distribution data obtained by the processing device 610 within the scanning range can refer to the content of the processing device 610 obtaining the detection result of the object in S610 above, which is not repeated here.

[0182] In a possible example, the processing device 610 emits a light beam to illuminate an object while moving, receives the light beam reflected by the object, and obtains point cloud distance distribution data while obtaining detection results.

[0183] S1020: The processing device 610 generates an occupancy grid map based on the point cloud distance distribution data.

[0184] The occupied grid map is used to indicate whether a grid in the map corresponding to the scanning range is occupied by an obstacle 420 , and one grid corresponds to one or more pixel points.

[0185] In one possible implementation, processing device 610 determines the area occupied by a grid cell in the grid map. Based on the area of ​​the grid cell, processing device 610 converts the point cloud in the point cloud distance distribution data into a point in the grid map, and calculates the distribution of point cloud distances within each grid cell. Processing device 610 generates an occupancy grid map based on the distribution of point cloud distances within each grid cell.

[0186] For example, the processing device 610 determines how many small squares (grids) to divide the scan area into, that is, determines the area occupied by each grid. If the grid area is small, the resulting occupancy grid map will have greater accuracy. If the grid area is large, the occupancy grid map will be generated faster. A small square may correspond to one or more pixels, and the area occupied by the small square corresponds to the area in the real environment.

[0187] Processing device 610 uses a transformation matrix to convert the distances between points in the point cloud distance distribution data into distances between points in the coordinate system of the grid map. For each grid, the distance values ​​of all points falling within that grid are counted, and the distribution of these distance values ​​is calculated. Processing device 610 can determine whether a grid is occupied or unoccupied based on a distance threshold.

[0188] Generally, grids with distance values ​​less than the threshold are considered to be occupied by objects, and grids with distance values ​​greater than the threshold are considered to be idle areas.

[0189] The above-mentioned occupied grid may be identified by a binary number, wherein 1 indicates that the grid is an area occupied by an object, and 0 indicates that the grid is an idle area.

[0190] To visualize the occupancy grid map, different colors can be used to represent occupied areas or free areas. For example, black indicates that the grid is occupied by objects, and white indicates that the grid is free.

[0191] In one possible implementation, the processing device 610 may use Gmapping or Cartographer to generate an occupancy grid map.

[0192] S1030: The processing device 610 merges the occupancy grid map and the transparent object map to obtain a semantic map.

[0193] The transparent object map is used to indicate whether the grid in the map corresponding to the scanning range is occupied by transparent objects. The semantic map is used to indicate whether the grid in the map corresponding to the scanning range is occupied by obstacles and / or transparent objects.

[0194] For example, the marks in the grid are used to indicate the material of the transparent object.

[0195] The processing device 610 constructs a transparent object map based on the transparent objects within the scanning range and the point cloud intensity distribution data. The grid marks in the transparent object map are used to indicate the material of the transparent objects, that is, different marks are used to indicate different materials.

[0196] Processing device 610 can fill the grid with different colors to indicate the material of the object occupying the grid. For example, red represents glass, and blue represents frosted glass. The materials represented by these colors are merely examples and should not be construed as limiting the present application. In other embodiments of the present application, red can also represent transparent plastic.

[0197] For the content of the transparent object map constructed by the processing device 610, reference may be made to the description of constructing the occupancy grid map in S1020, which will not be described in detail here.

[0198] In one possible implementation, the processing device 610 merges the occupied grid map with the transparent object map to obtain a semantic map, including: the processing device 610 merges the grids in the occupied grid map with the corresponding grids in the transparent object map to obtain a semantic map.

[0199] Exemplarily, the processing device 610 merges the grids occupying corresponding positions in the grid map and the transparent object map to obtain a semantic map.

[0200] For example, if a grid in the occupied grid map is an area occupied by an object, and the grid at the corresponding position in the transparent object map is an area occupied by glass, the processing device 610 determines that the grid at that position in the semantic map is an area occupied by glass.

[0201] If a grid in the occupied grid map is black and the grid at the corresponding position in the transparent object map is red, the processing device 610 determines that the grid at that position in the semantic map is red, indicating that the grid is occupied by glass.

[0202] In this application, because processing device 610 can accurately identify transparent objects and their materials within the scanned area, the semantic map constructed based on these objects and their materials has a high degree of accuracy. Furthermore, when processing device 610 navigates based on this semantic map, it can avoid both transparent and non-transparent objects, improving the safety of processing device 610 during movement.

[0203] In one possible embodiment, the processing device 610 may visualize the semantic map.

[0204] In one possible implementation, the processing device 610 displays the semantic map on the front end. The front end here may refer to a display connected to the processing device 610, or a display screen provided by the processing device 610, etc., which is not limited in this application.

[0205] In this application, for the user, the user adjusts the processing device 610 according to the visualized semantic map to meet the user's needs, which is conducive to more accurate recognition of transparent objects or mapping by the processing device 610.

[0206] In one possible embodiment, the processing device 610 performs navigation according to an occupancy grid map or a semantic map.

[0207] For example, the processing device 610 may use a navigation strategy to perform navigation according to the occupancy of grids in the grid map or the semantic map.

[0208] The navigation strategy is used to indicate: to detour around occupied cells in the semantic map.

[0209] In one possible scenario, the processing device 610 uses different detour methods according to the material of the transparent object indicated by the occupied grid in the semantic map.

[0210] For example, because glass has a higher risk factor, the expansion coefficient for glass is set to a larger value. That is, the grid area corresponding to glass in the semantic map is expanded by a multiple of the expansion coefficient, so that the processing device 610 will detour further when passing by glass. Because plastic has a lower risk factor, the expansion coefficient for transparent plastic is set to a smaller value. That is, the grid area corresponding to transparent plastic in the semantic map is expanded by a multiple of the expansion coefficient, so that the processing device 610 can move close to the transparent plastic when passing by it.

[0211] In this application, because the semantic map records the location of transparent objects, processing device 610 can navigate based on the semantic map and accurately avoid transparent objects in the scanning area, thereby improving the safety of processing device 610 during movement. Furthermore, the semantic map also indicates the material of transparent objects, allowing processing device 610 to adopt corresponding detour strategies for different transparent object materials, further ensuring the safety of processing device 610 during movement.

[0212] In one possible example, the processing device 610 may further display the movement path of the processing device 610 on the front end. For example, the movement path of the processing device 610 may be additionally displayed on the semantic map. This facilitates the user to adjust the processing device 610 to meet user needs, so that the movement path of the processing device 610 better meets user needs.

[0213] It is understood that in order to implement the functions in the above embodiments, the computing device includes hardware structures and / or software modules corresponding to the execution of each function. It should be readily apparent to those skilled in the art that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in hardware or in a manner driven by computer software depends on the specific application scenario and design constraints of the technical solution.

[0214] The transparent object recognition method provided in accordance with this embodiment is described in detail above with reference to FIG. 4 to FIG. 10 . The transparent object recognition device provided in accordance with this embodiment will be described below with reference to FIG. 11 .

[0215] Figure 11 is a schematic diagram of the structure of a transparent object recognition device provided in this application. This transparent object recognition device can be used to implement the functions of the computing device in the above-mentioned method embodiment, thereby also achieving the beneficial effects of the above-mentioned method embodiment. In this embodiment, the transparent object recognition device can be a module (such as a chip) of the computing device.

[0216] As shown in Figure 11, a transparent object recognition device 1100 includes a first acquisition module 1101, a receiving module 1102, and a first determination module 1103. The transparent object recognition device 1100 is used to implement the functions of the method embodiments shown in Figures 4 to 10 above.

[0217] The first acquisition module 1101 is used to acquire obstacles within the scanning range.

[0218] Receiving module 1102 is used to process the angle formed by the light beam emitted by the device and the tangent plane of the reference point on the obstacle, and receive the detection results of the obstacle during the process of changing from the first angle to the second angle. The first angle is different from the second angle, and the reference point is the intersection point of the light beam on the obstacle.

[0219] The first determining module 1103 is configured to determine a transparent object from obstacles based on the obstacle detection result.

[0220] To further implement the functions of the method embodiments shown in Figures 4 to 10 above, the present application also provides a transparent object recognition device, as shown in Figure 12. Figure 12 is a second structural schematic diagram of a transparent object recognition device provided by the present application. The transparent object recognition device 1100 further includes: a similarity acquisition module 1104, a second determination module 1105, a second acquisition module 1106, a generation module 1107, a merging module 1108, a navigation module 1109, and a display module 1110.

[0221] The similarity acquisition module 1104 is configured to acquire, for the point cloud intensity sets of multiple transparent materials, the similarity between the point cloud intensity set of the transparent object and the point cloud intensity set of each transparent material.

[0222] The second determining module 1105 is configured to use the transparent materials with a similarity greater than or equal to a second threshold among the plurality of materials as the material of the transparent object.

[0223] The second acquisition module 1106 is used to acquire point cloud distance distribution data, where the point cloud distance distribution data is used to indicate the distance between any two positions within the scanning range.

[0224] The generating module 1107 is used to generate an occupancy grid map according to the point cloud distance distribution data. The occupancy grid map is used to indicate whether a grid in the map corresponding to the scanning range is occupied by an obstacle. One grid corresponds to one or more pixels.

[0225] Merging module 1108 is configured to merge the occupied grid map with the transparent object map to generate a semantic map. The transparent object map indicates whether grids in the map corresponding to the scanning range are occupied by transparent objects. The semantic map indicates whether grids in the map corresponding to the scanning range are occupied by obstacles and / or transparent objects. Grid markings indicate the material of the transparent objects.

[0226] The navigation module 1109 is used to perform navigation according to the semantic map using a navigation strategy; the navigation strategy is used to indicate: to detour around the occupied grids in the semantic map.

[0227] The display module 1110 is configured to display the semantic map.

[0228] It is worth noting that the processing device 610 of the aforementioned embodiment may correspond to the transparent object recognition device 1100, and may correspond to the corresponding subject corresponding to Figures 4 to 10 for executing the method according to the embodiments of the present application, and the operations and / or functions of each module in the transparent object recognition device 1100 are respectively for implementing the corresponding processes of each method of the corresponding embodiments in Figures 4 to 10. For the sake of brevity, they will not be repeated here.

[0229] In addition, the apparatus shown in FIG11 and FIG12 can also be implemented by a communication device, where the communication device may refer to the computing device in the aforementioned embodiment, or, when the communication device is a chip or chip system applied to a computing device, the apparatus can also be implemented by the chip or chip system.

[0230] When the transparent object recognition device is implemented via hardware, the hardware can be implemented via a processor or a chip. The following description uses a chip as an example. The chip includes a processor for implementing the functions of processing device 610 in the above method. In one possible design, the chip also includes a power supply circuit for powering the processor. The chip can be directly composed of the chip itself or include the chip and other discrete components.

[0231] An embodiment of the present application provides a computing device, as shown in Figure 13, which is a schematic diagram of the structure of a computing device provided by the present application. The computing device can be applied to the above-mentioned computer system.

[0232] The computing device 1300 can specifically be a mobile phone, a tablet computer, a television (also known as a smart TV, a smart screen or a large-screen device), a laptop computer, an ultra-mobile personal computer (UMPC), a handheld computer, a netbook, a personal digital assistant (PDA), a wearable electronic device (for example, a smart watch, a smart bracelet, smart glasses), a vehicle-mounted device, a virtual reality device, a server, and other electronic devices with computing capabilities, as well as cars, robots, drones, and other simultaneous localization and mapping (SLAM) devices.

[0233] As shown in Figure 13, computing device 1300 includes a bus 1302, a processor 1304, a memory 1306, and a communication interface 1308. Processor 1304, memory 1306, and communication interface 1308 communicate with each other via bus 1302. Computing device 1300 can be a server or a terminal device. It is worth noting that this application does not limit the number of processors and memories in the processing device.

[0234] Bus 1302 may be, but is not limited to, a Peripheral Component Interconnect Express (PCIe) bus, a Universal Serial Bus (USB), an Inter-Integrated Circuit (I2C), an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (UBus or UB), a Compute Express Link (CXL), a Cache Coherent Interconnect for Accelerators (CCIX), a Controller Area Network (CAN), or the like. Bus 1302 may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, FIG13 shows only one line, but this does not indicate that there is only one bus or only one type of bus. Bus 1302 may include a path for transmitting information between various components of computing device 1300 (e.g., memory 1306, processor 1304, and communication interface 1308).

[0235] The processor 1304 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0236] The memory 1306 may include volatile memory, such as random access memory (RAM). The memory 1306 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0237] Memory 1306 stores executable program code, which processor 1304 executes to implement the functions of the first acquisition module, the receiving module, and the first determination module, thereby implementing the transparent object recognition method. In other words, memory 1306 stores instructions for executing the transparent object recognition method.

[0238] The communication interface 1308 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1300 and other devices or a communication network.

[0239] In a possible embodiment, the computing device 1300 may further include a laser radar or a motion component. The processor 1304 , the memory 1306 , the communication interface 1308 , the laser radar, and the motion component are connected via a bus 1302 .

[0240] LiDAR is a sensor used to measure and perceive the distance and geometry of its surroundings. LiDARs include: Rotating LiDAR: Rotating LiDARs acquire point cloud data of the environment through rotational scanning. Rotating LiDARs typically consist of a rotating laser beam and a receiver, enabling panoramic horizontal scanning. Rotating LiDARs offer a wide coverage area and high-density point cloud output, making them suitable for tasks such as 3D reconstruction, mapping, and obstacle detection.

[0241] Solid-state LiDAR: Solid-state LiDAR uses solid-state optoelectronic devices (such as solid-state lasers and solid-state receivers) for laser transmission and reception. Compared to rotating LiDAR, solid-state LiDAR requires no mechanical rotation and offers smaller size, higher reliability, and lower power consumption. However, due to the limited transmission and reception angles, solid-state LiDAR has a relatively small data collection range.

[0242] 3D time-of-flight lidar (TOF LiDAR): 3D TOF LiDAR uses the round-trip time of a light pulse to measure the distance between an object and the radar. It does this by sending a short pulse of light and measuring the time it takes for the light to return to the radar. 3D TOF LiDAR offers high-speed ranging and high accuracy, making it suitable for applications in fast-paced perception and dynamic environments.

[0243] Modulated continuous wave LiDAR: Modulated continuous wave LiDAR uses a modulated continuous wave beam for ranging. Modulated continuous wave LiDAR encodes distance information by varying the frequency or amplitude of light, which is decoded by a receiver to achieve measurement. Modulated continuous wave LiDAR is suitable for long-distance measurement and detection of high-speed moving objects.

[0244] The motion component may include a motor, a wheel set, or a track, etc., to enable the computing device 1300 to move.

[0245] The present application also provides a computer program product containing instructions. This computer program product can be software or a program product containing instructions that can be executed on a processing device or stored on any usable medium. When executed on at least one processing device, this computer program product causes the at least one processing device to perform the transparent object recognition method described above.

[0246] The present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a processing device, or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the processing device to execute the transparent object recognition method.

[0247] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the process or function described in the embodiments of the present application is performed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instruction can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired or wireless means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, a hard disk, or a tape; it can also be an optical medium, such as a digital video disc (DVD); it can also be a semiconductor medium, such as a solid state drive (SSD).

[0248] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A transparent object recognition method, characterized in that: The method is applied to a processing device, and the method comprises: Obtain obstacles within the scanning range; The light beam emitted by the processing device forms an angle with a tangent plane of a reference point on the obstacle, and during the process of changing from a first angle to a second angle, receiving a detection result of the obstacle, the first angle is different from the second angle, and the reference point is an intersection point of the light beam on the obstacle; According to the detection result of the obstacle, a transparent object is determined from the obstacles.

2. The method according to claim 1, characterized in that The light beam emitted by the processing device forms an angle with the tangent plane of the reference point on the obstacle, and the change from the first angle to the second angle is used to indicate that: during the process of the processing device moving around the obstacle, the angle between the light beam emitted by the processing device and the tangent plane of the reference point changes from the first angle to the second angle.

3. The method according to claim 2, characterized in that The processing device moving around the obstacle includes: the processing device moving clockwise around the obstacle, or the processing device moving counterclockwise around the obstacle.

4. The method according to claim 1, characterized in that: The light beam emitted by the processing device forms an angle with the tangent plane of the reference point on the obstacle, and the change from the first angle to the second angle is used to indicate that: during the process of the processing device moving along a straight line, the angle between the light beam emitted by the processing device and the tangent plane of the reference point changes from the first angle to the second angle.

5. The method according to any one of claims 1 to 4, characterized in that The detection result is used to indicate: a point cloud intensity set at multiple different angles; The point cloud intensity set includes multiple point cloud intensities, and any one of the multiple point cloud intensities is used to indicate: the reflection intensity of the light beam at an angle at the reference point of the obstacle, and the angle is between the first angle and the second angle.

6. The method according to claim 5, characterized in that The step of determining the transparent object from the obstacles according to the detection result of the obstacles comprises: Obtaining the similarity between the point cloud intensity set of the obstacle and the point cloud intensity set of the non-transparent object; Obstacles with a similarity less than or equal to a first threshold are determined as the transparent objects.

7. The method according to claim 5 or 6, characterized in that: The point cloud intensity set under the multiple different angles is a point cloud intensity curve, and the point cloud intensity curve is used to indicate: the corresponding relationship between the point cloud intensity and the angle.

8. The method according to claim 7, characterized in that The similarity of the point cloud intensity set is used to indicate: the similarity of the attributes of the point cloud intensity curve; The attributes of the point cloud intensity curve include: one or more of a peak value, an average width, and an average gradient; The average width is used to indicate: the difference between the horizontal coordinates of two points in the point cloud intensity curve whose vertical coordinates are half of the peak value, the vertical coordinate is used to indicate the point cloud intensity, and the horizontal coordinate is used to indicate the angle; The average gradient is used to indicate: an average value of gradients of multiple points in the point cloud intensity curve.

9. The method according to any one of claims 5 to 8, characterized in that The method further comprises: For point cloud intensity sets of multiple transparent materials, obtaining the similarity between the point cloud intensity set of the transparent object and the point cloud intensity set of each transparent material; The transparent materials with a plurality of similarities greater than or equal to a second threshold are used as the material of the transparent object.

10. The method according to claim 9, characterized in that The method further comprises: Acquire point cloud distance distribution data; the point cloud distance distribution data is used to indicate the distance between any two positions within the scanning range; Generate an occupied grid map according to the point cloud distance distribution data, wherein the occupied grid map is used to indicate whether a grid in the map corresponding to the scanning range is occupied by an obstacle, and one grid corresponds to one or more pixel points; The occupied grid map is merged with the transparent object map to obtain a semantic map, wherein the transparent object map is used to indicate whether the grid in the map corresponding to the scanning range is occupied by the transparent object; the semantic map is used to indicate whether the grid in the map corresponding to the scanning range is occupied by the obstacle and / or the transparent object, and the mark of the grid is used to indicate the material of the transparent object.

11. The method according to claim 10, characterized in that The method further comprises: Navigation is performed using a navigation strategy according to the semantic map; the navigation strategy is used to instruct: circumventing occupied grids in the semantic map.

12. The method according to claim 10, characterized in that The method further comprises: The semantic map is displayed.

13. A transparent object recognition device, characterized in that: The device is applied to a processing device, and comprises: a first acquisition module, a receiving module and a first determination module; A first acquisition module is used to acquire obstacles within the scanning range; a receiving module, configured to receive a detection result of the obstacle when an angle is formed between the light beam emitted by the processing device and a tangent plane of a reference point on the obstacle, and when the angle changes from a first angle to a second angle, the first angle is different from the second angle, and the reference point is an intersection point of the light beam on the obstacle; The first determination module is used to determine the transparent object from the obstacles according to the detection result of the obstacles.

14. The device according to claim 13, characterized in that The light beam emitted by the processing device forms an angle with the tangent plane of the reference point on the obstacle, and the change from the first angle to the second angle is used to indicate that: during the process of the processing device moving around the obstacle, the angle between the light beam emitted by the processing device and the tangent plane of the reference point changes from the first angle to the second angle.

15. The device according to claim 14, characterized in that The processing device moving around the obstacle includes: the processing device moving clockwise around the obstacle, or the processing device moving counterclockwise around the obstacle.

16. The device according to claim 13, characterized in that The light beam emitted by the processing device forms an angle with the tangent plane of the reference point on the obstacle, and the change from the first angle to the second angle is used to indicate that: during the process of the processing device moving along a straight line, the angle between the light beam emitted by the processing device and the tangent plane of the reference point changes from the first angle to the second angle.

17. The device according to any one of claims 13 to 16, characterized in that The detection result is used to indicate: a set of point cloud intensities at multiple different angles; the point cloud intensity set includes multiple point cloud intensities, and any one of the multiple point cloud intensities is used to indicate: the reflection intensity of the reference point of the obstacle to a light beam at an angle, and the angle is between the first angle and the second angle.

18. The device according to claim 17, characterized in that The first determination module is specifically used to obtain the similarity between the point cloud intensity set of the obstacle and the point cloud intensity set of the non-transparent object, and determine the obstacle with a similarity less than or equal to a first threshold as the transparent object.

19. The device according to claim 17 or 18, characterized in that The point cloud intensity set under the multiple different angles is a point cloud intensity curve, and the point cloud intensity curve is used to indicate: the corresponding relationship between the point cloud intensity and the angle.

20. The device according to claim 19, characterized in that The similarity of the point cloud intensity set is used to indicate: the similarity of the attributes of the point cloud intensity curve; The attributes of the point cloud intensity curve include: one or more of a peak value, an average width, and an average gradient; The average width is used to indicate: the difference between the horizontal coordinates of two points in the point cloud intensity curve whose vertical coordinates are half of the peak value, the vertical coordinate is used to indicate the point cloud intensity, and the horizontal coordinate is used to indicate the angle; The average gradient is used to indicate: an average value of gradients of multiple points in the point cloud intensity curve.

21. The device according to any one of claims 17 to 20, characterized in that The transparent object recognition device further includes: a similarity acquisition module and a second determination module; The similarity acquisition module is used to indicate the point cloud intensity sets for multiple transparent materials, and to acquire the similarity between the point cloud intensity set of the transparent object and the point cloud intensity set of each transparent material; The second determination module is used to use the transparent materials with a similarity greater than or equal to a second threshold among the multiple similarities as the material of the transparent object.

22. The device according to claim 21, characterized in that The transparent object recognition device further includes: a second acquisition module, a generation module and a merging module; The second acquisition module is used to acquire point cloud distance distribution data; the point cloud distance distribution data is used to indicate the distance between any two positions within the scanning range; The generating module is used to generate an occupied grid map according to the point cloud distance distribution data, wherein the occupied grid map is used to indicate whether a grid in the map corresponding to the scanning range is occupied by an obstacle, and one grid corresponds to one or more pixel points; The merging module is used to merge the occupied grid map with the transparent object map to obtain a semantic map, wherein the transparent object map is used to indicate whether the grid in the map corresponding to the scanning range is occupied by the transparent object; the semantic map is used to indicate whether the grid in the map corresponding to the scanning range is occupied by the obstacle and / or the transparent object, and the mark of the grid is used to indicate the material of the transparent object.

23. The device according to claim 22, characterized in that The transparent object recognition device further includes: a navigation module; The navigation module is used to navigate according to the semantic map using a navigation strategy; the navigation strategy is used to instruct: to detour the occupied grids in the semantic map.

24. The device according to claim 22, characterized in that The transparent object recognition device further includes: a display module; The display module is used to display the semantic map.

25. A computing device, characterized in that including a processor and a memory; The processor is configured to execute instructions stored in the memory so that the computing device performs the method according to any one of claims 1 to 12.

Images