Window shielding object detection method and device of equipment, electronic equipment and storage medium

Abstract

The invention discloses a window shelter detection method and device for equipment, electronic equipment and a storage medium, and the method comprises the steps: obtaining a point cloud data frame collected by target equipment, the point cloud data frame comprising detection data of a plurality of detection points; according to the detection data of the multiple detection points, a first detection point in the multiple detection points is determined, and the first detection point is a detection point corresponding to a shielding object attached to the window of the target equipment; according to the position information of the first detection point, shielding object detection processing is carried out on the window, the window shielding object detection result of the target equipment is obtained, and whether shielding objects are attached to the window of the target equipment or not can be accurately detected.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to a method, device, electronic device and storage medium for detecting window obstructions of a device, wherein the storage medium is a computer-readable storage medium. Background Art

[0002] At present, the shape, size, distance and other information of surrounding objects can be detected through devices such as laser radar to achieve perception of the surrounding environment. However, since the window of the device is exposed to the environment, there may be obstructions attached to the window, blocking the path of the target device's laser emission and receiving echo data, affecting the target device's ability to perceive surrounding obstacles. Therefore, it is very important to detect whether there are obstructions on the window. Summary of the invention

[0003] Embodiments of the present application provide a device window obstruction detection method, apparatus, electronic device, and computer-readable storage medium, which can detect whether there is an obstruction attached to the window of the device.

[0004] In order to achieve the above-mentioned object, according to a first aspect of the present application, a method for detecting a window obstruction of a device is provided, comprising:

[0005] Acquire a point cloud data frame collected by a target device, wherein the point cloud data frame includes detection data of multiple detection points;

[0006] Determine, according to the detection data of the multiple detection points, a first detection point among the multiple detection points, where the first detection point is a detection point corresponding to the obstruction attached to the window of the target device;

[0007] According to the position information of the first detection point, the window is subjected to obstruction detection processing to obtain the window obstruction detection result of the target device.

[0008] According to a second aspect of the present application, a device for detecting window obstructions of a device is provided, comprising:

[0009] An acquisition unit, used to acquire a point cloud data frame collected by a target device, wherein the point cloud data frame includes detection data of multiple detection points;

[0010] a determining unit, configured to determine, according to the detection data of the multiple detection points, a first detection point among the multiple detection points, wherein the first detection point is a detection point corresponding to an obstruction attached to a window of the target device;

[0011] The detection unit is used to perform obstruction detection processing on the window according to the position information of the first detection point to obtain the window obstruction detection result of the target device.

[0012] According to a third aspect of the present application, an electronic device is provided, comprising a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the window obstruction detection method of any device provided in the embodiments of the present application.

[0013] According to a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store a computer program, and the computer program is loaded by a processor to execute the window obstruction detection method of any device provided in the embodiments of the present application.

[0014] The embodiment of the present application obtains a point cloud data frame collected by a target device, where the point cloud data frame includes detection data of multiple detection points; based on the detection data of the multiple detection points, a first detection point among the multiple detection points is determined, where the first detection point is a detection point corresponding to an obstruction attached to a window of the target device; based on the position information of the first detection point, an obstruction detection process is performed on the window to obtain a window obstruction detection result of the target device, so as to accurately detect whether there is an obstruction attached to the window of the target device. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without creative work.

[0016] Figure 1 is a flow chart of a method for detecting window obstructions in a device provided in an embodiment of the present application;

[0017] Figure 2 is a flowchart of obstruction detection provided by an embodiment of the present application;

[0018] Figure 3 is another obstruction detection flow chart provided in an embodiment of the present application;

[0019] Figure 4 is a schematic diagram of an obstruction detection system provided in an embodiment of the present application;

[0020] Figure 5 is a schematic diagram of a window obstruction detection device of a device provided in an embodiment of the present application;

[0021] Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0023] The embodiments of the present application provide a device window obstruction detection method, device, electronic device and computer readable storage medium. The device window obstruction detection device can be integrated in an electronic device.

[0024] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0025] This embodiment will be described from the perspective of a window obstruction detection device of the device. The window obstruction detection device of the device can be specifically integrated in an electronic device, which can be a car or the like. The target device can be integrated in the electronic device, for example, the target device can be a laser radar on a vehicle.

[0026] The embodiment of the present application provides a method for detecting window obstructions of a device, such as Figure 1 As shown, the specific process of the window obstruction detection method of the device can be as follows:

[0027] 101. Obtain a point cloud data frame collected by a target device, where the point cloud data frame includes detection data of multiple detection points.

[0028] Among them, the target device can be a device such as a laser radar, a depth camera, etc. that can collect point cloud data. The point cloud data frame includes detection data of multiple detection points obtained by the target device. The detection point can be a point in the point cloud data frame. Each detection point can correspond to a sampling point in space. The detection data can include reflection information of the sampling point to the laser emitted by the target device, such as the distance of the sampling point, the echo intensity, etc.

[0029] 102. Determine a first detection point among the multiple detection points according to detection data of the multiple detection points, where the first detection point is a detection point corresponding to an obstruction attached to a window of the target device.

[0030] Among them, the window of the target device can be a component of the target device, and the window of the target device can also be called a window piece or a lens. The window can protect the optical components inside the target device from being damaged by the external environment and allow the laser beam to pass through; the window can be located at the front end of the target device, facing the measured environment, or it can be located at other positions. The specific position of the window can be set as needed, and the position of the window is not limited here.

[0031] During the use of the target device, the window of the target device may be attached with dust, rain, or other debris. These obstructions attached to the window will absorb or scatter the laser, thereby affecting the laser echo intensity, etc., and the obstructions attached to the window are closer to the laser emitting component of the target device than other objects in the space. That is, if there is an obstruction on the window, it will affect the detection data of the detection point. Therefore, it is possible to determine whether the detection point meets the obstruction characteristics based on the detection data of the detection point. If it meets the obstruction characteristics, the detection point is determined to be the detection point corresponding to the obstruction on the window. The detection point corresponding to the obstruction on the window can be considered to be a point obtained based on the reflection information of the laser passing through the position of the obstruction on the window.

[0032] Based on the detection data of the detection point, determine whether the detection point meets the characteristics of the obstruction. Detection can be performed through a neural network model, or the distance and echo intensity of the sampling point can be extracted from the detection data of the detection point. The echo intensity corresponding to the detection point and the distance of the sampling point are compared with the preset echo intensity threshold and the sampling point distance threshold, respectively, to determine whether the detection point is the first detection point. For example, if the sampling point distance is less than the distance threshold, and / or the echo intensity is less than the echo intensity threshold, then the detection point is determined to be the first detection point.

[0033] 103. Perform obstruction detection processing on the window according to the position information of the first detection point to obtain a window obstruction detection result of the target device.

[0034] The position information can indicate the position of the first detection point in the point cloud. Based on the position information of the first detection point, the number of first detection points in the specified area in the point cloud data frame can be counted. Based on the number of first detection points located in the specified area, the window occlusion detection result of the target device can be determined. The window occlusion detection result can indicate whether there is an occlusion on the window of the target device. For example, if the number of first detection points located in the specified area is greater than a preset number threshold, the window occlusion detection result is determined to be that there is an occlusion on the window. If the number of first detection points located in the specified area is not greater than the preset number threshold, the window occlusion detection result is determined to be that there is no occlusion on the window.

[0035] Among them, the designated area can be flexibly configured according to application needs. For example, the point cloud area corresponding to the central area of ​​the window in the point cloud data frame can be used as the designated area, and the detection points in the point cloud area corresponding to the central area of ​​the window are the points detected by the laser passing through the central area of ​​the window.

[0036] Optionally, at least one of whether there is a block-shaped obstruction in the window and the degree of obstruction of the window can be detected according to the position information of the first detection point. That is, in one embodiment, the step of "performing an obstruction detection process on the window according to the position information of the first detection point to obtain a window obstruction detection result of the target device" includes:

[0037] According to the position information of the first detection point, the window is subjected to an occlusion degree detection process to obtain a first window occlusion object detection result of the target device;

[0038] and / or, performing block-shaped occlusion detection processing on the window of the target device according to the position information of the first detection point to obtain a second window occlusion detection result of the target device;

[0039] A window obstruction detection result is obtained according to the first window obstruction detection result and / or the second window obstruction detection result.

[0040] Among them, according to the position information of the first detection point, the window is subjected to occlusion degree detection processing. Specifically, according to the position information of the first detection point, the number of first detection points located in the central area of ​​the point cloud data frame is determined, and according to the number of first detection points located in the central area of ​​the point cloud data frame, the occlusion degree of the window is determined. For example, multiple numerical intervals can be set, each numerical interval corresponds to an occlusion degree, and the occlusion degree of the window is determined according to the number of first detection points in the central area of ​​the point cloud data frame and the numerical interval to which they belong, so as to obtain the first window occlusion detection result.

[0041] Optionally, an occlusion coefficient may be assigned to the first detection point according to the position information of the first detection point, and the occlusion coefficients may be summed to distinguish the influence of occlusions at different positions on the window. That is, in one embodiment, the step of "performing an occlusion degree detection process on the window according to the position information of the first detection point to obtain a first window occlusion detection result of the target device" includes:

[0042] Determining an occlusion coefficient of the first detection point based on the position information of the first detection point;

[0043] The occlusion coefficients of the first detection point are summed to obtain an occlusion index value corresponding to the point cloud data frame;

[0044] A first window occlusion object detection result of the target device is determined according to the occlusion index value.

[0045] For example, a mapping relationship between the position of the first detection point and the occlusion coefficient can be set in advance. The mapping relationship can be that the closer the first detection point is to the center of the point cloud data frame, the larger the occlusion coefficient is, and the closer the first detection point is to the edge of the point cloud data frame, the smaller the occlusion coefficient is. Based on the mapping relationship and the position information of the first detection point, the occlusion coefficient of the first detection point in the point cloud data frame is determined.

[0046] Optionally, different occlusion coefficients may be assigned to the central area of ​​the point cloud data frame and the first detection point outside the central area. The central area may be preset. That is, in one embodiment, the step of “determining the occlusion coefficient of the first detection point based on the position information of the first detection point” includes:

[0047] For each first detection point, if the first detection point is located in the central area of ​​the point cloud data frame, a first occlusion coefficient is assigned to the first detection point;

[0048] If the first detection point is located outside the central area of ​​the point cloud data frame, a second occlusion coefficient is assigned to the first detection point, and the first occlusion coefficient is greater than the second occlusion coefficient.

[0049] For example, for each first detection point of the point cloud data frame, the position information of the pre-set central area is estimated based on the position information of the first detection point to determine whether the first detection point is located in the central area. If the first detection point is located in the central area, the first occlusion coefficient is assigned to the first detection point, otherwise the first detection point is set to the second occlusion coefficient.

[0050] Since the first detection point is located in the central area of ​​the point cloud data frame, it means that there is an occlusion in the central area of ​​the window. The occlusion located in the central area of ​​the window has a greater impact on the window. Therefore, the occlusion coefficient of the first detection point located in the central area of ​​the point cloud data frame can be set to be larger than the occlusion coefficient of the first detection point located outside the central area of ​​the point cloud data frame, that is, the first occlusion coefficient is greater than the second occlusion coefficient.

[0051] After determining the occlusion coefficient of the first detection point in the point cloud data frame, the occlusion coefficient of the first detection point in the point cloud data frame can be summed to obtain the occlusion index value corresponding to the point cloud data frame, and then the first window occlusion detection result of the target device is determined according to the occlusion index value.

[0052] For example, the first window occlusion detection result of the target device can be determined based on the size relationship between the occlusion index value and the preset index threshold. The first window occlusion detection result can indicate whether there are a large number of occlusions attached to the window of the target device, and can also indicate whether the window of the target device is dirty.

[0053] Optionally, there may be multiple preset indicator thresholds. According to the magnitude relationship between the occlusion indicator value and the multiple preset indicator thresholds, the first frame detection result of the point cloud data frame is determined. The first frame detection result may be whether it is an occlusion point cloud frame and the corresponding occlusion level. That is, in one embodiment, the step of "determining the first window occlusion detection result of the target device according to the occlusion indicator value" includes:

[0054] Determine a first frame detection result of the point cloud data frame according to a magnitude relationship between the occlusion index value and a preset index threshold, wherein the first frame detection result indicates whether the point cloud data frame is an occluded point cloud frame and an occlusion level corresponding to the point cloud data frame;

[0055] A first window occlusion detection result is determined based on the first frame detection result, where the first window occlusion detection result includes an occlusion level of the window.

[0056] Among them, multiple preset indicator thresholds can be set, for example, there are two preset indicator thresholds A and B, where B is greater than A. If the occlusion index value is less than A, it is determined that the point cloud data frame is not an occluded point cloud frame. If the occlusion index value is greater than A and less than B, it is determined that the point cloud data frame is an occluded point cloud frame, and the occlusion level is light occlusion. If the occlusion index value is greater than A and greater than B, it is determined that the point cloud data frame is an occluded point cloud frame, and the occlusion level is heavy occlusion.

[0057] It is understandable that the number of indicator thresholds can be set according to the application scenario to subdivide the occlusion level, for example, it can be subdivided into light occlusion, moderate occlusion, and heavy occlusion.

[0058] The first frame detection result of the point cloud data frame may be used as the first window occlusion detection result of the target device, and the occlusion level of the window of the target device is the corresponding occlusion level of the point cloud data frame.

[0059] Optionally, the first window occlusion detection result of the target device may be determined in combination with the first frame detection result of the multi-frame point cloud data frame. That is, in one embodiment, the step of “determining the first window occlusion detection result based on the first frame detection result” includes:

[0060] If the first frame detection result indicates that the point cloud data frame is an occluded point cloud frame, the first window occlusion detection result is determined according to the first frame detection result in at least one frame of historical point cloud data frame.

[0061] Among them, at least one frame of historical point cloud data frame can be the previous frame or multiple frames of point cloud data frame in the time sequence. If the first frame detection result of the point cloud data frame indicates that the point cloud data frame is not an occluded point cloud frame, the first window occlusion detection result is determined to be that there is no occlusion on the window.

[0062] If the first frame detection result of the point cloud data frame indicates that the point cloud data frame is an occluded point cloud frame, and the first frame detection result of at least one frame of historical point cloud data frame indicates that the historical point cloud data frame is an occluded point cloud frame, that is, the point cloud data frame and at least one frame of historical point cloud data frame are both occluded point cloud frames, it is determined that the first window occlusion detection result is that there is an occlusion on the window.

[0063] For example, the occlusion coefficient of the first detection point located in the central area is x (x>1), and the occlusion coefficient of the first detection point located outside the central area is 1 for distance description. The flowchart for determining the detection result of the first window occlusion object can be as follows: Figure 2 As shown, Figure 2 Where k is the occlusion index value, and the initial occlusion index value can be set to 0. The detection data of the detection points are obtained from the point cloud data frame in sequence. If the detection point is determined to be the first detection point according to the detection data, it is determined whether the first detection point is located in the central area. If so, the occlusion index value is increased by x, otherwise the occlusion index value is increased by 1. When the last detection point is detected, the occlusion index value of the point cloud data frame can be obtained. If the occlusion index value is greater than the preset index threshold, the frame number is increased by 1. The initial frame number can be set to 0. If the occlusion index value is not greater than the preset index threshold, the frame number is set to 0. If the frame number is greater than the preset frame number threshold, it is determined that there is an occlusion on the window, which can also be considered as a dirty window. If the frame number is less than the preset frame number threshold, the next frame of the point cloud data frame is obtained for judgment.

[0064] Optionally, the occlusion level of the target device window can be determined based on the occlusion levels of the point cloud data frame and at least one frame of historical point cloud data frame. For example, the maximum occlusion level among the occlusion levels of the point cloud data frame and at least one frame of historical point cloud data frame can be used as the occlusion level of the target device window, or the maximum occlusion level among the occlusion levels of the point cloud data frame can be used as the occlusion level of the target device window.

[0065] The first window occlusion detection result can indicate whether there is an obstruction on the window of the target device and the degree of obstruction, while the second window occlusion detection result can indicate whether there is a block obstruction on the window of the target device. The second window occlusion detection result can be obtained by performing block obstruction detection processing on the window of the target device based on the position information of the first detection point.

[0066] According to the position information of the first detection point, block occlusion detection processing is performed on the window of the target device. For example, the first detection point may be clustered according to the position information of the first detection point to obtain a clustering result. Since the clustering process can aggregate the first detection points with similar position information into clusters, the second window occlusion detection result is obtained according to the number of first detection points in the cluster obtained by clustering. If the number of first detection points in the cluster obtained by clustering is greater than a preset number threshold, it is determined that the second window occlusion detection result is that a block occlusion exists on the window of the target device. If the number of first detection points in the cluster obtained by clustering is not greater than the preset number threshold, it is determined that the second window occlusion detection result is that there is no block occlusion on the window of the target device.

[0067] In one embodiment, the position information of the detection point in the point cloud data frame may be a row and column number, and the row and column number may be determined according to the acquisition angle of the detection point, wherein the acquisition angle may be considered as the angle at which the target device emits the laser, and the target device scans the point cloud according to the angular resolution within its field of view, and the angle formed by the acquisition angles of two adjacent points in the point cloud is the angular resolution. The acquisition angles of the detection points in the same point cloud column in the horizontal direction are the same, the acquisition angles of the detection points in different point cloud columns in the horizontal direction are different, and the angle formed by the acquisition angles of the detection points in adjacent point cloud columns in the horizontal direction is the angular resolution, the acquisition angles of the detection points in the same row in the vertical direction are the same, the acquisition angles of the detection points in different rows in the vertical direction are different, and the angle formed by the acquisition angles of the detection points in adjacent point cloud rows in the vertical direction is the angular resolution.

[0068] It is possible to determine whether there is an area containing a large number of first detection points in the point cloud data frame based on the position information of the first detection points in each column of the point cloud data frame, so as to determine whether there is a block occluder on the target device. That is, in one embodiment, the step of "performing block occluder detection processing on the window of the target device based on the position information of the first detection point to obtain a second window occluder detection result of the target device" may include:

[0069] According to the acquisition angle of the detection point, a point cloud column is obtained from the point cloud data frame, and the acquisition angle of the detection points in the point cloud column in the horizontal direction is the same;

[0070] Determine a column detection result of the point cloud column according to the position information of the first detection point in the point cloud column, the column detection result indicating whether the point cloud column contains a sub-block occlusion area;

[0071] The second window obstruction detection result is determined according to the column detection result.

[0072] Among them, the sub-block occlusion area of ​​the point cloud column can be the area on the point cloud column corresponding to the block occlusion on the window of the target device, and the block occlusion area is the area in the point cloud corresponding to the block occlusion on the window of the target device. According to the acquisition angle of the detection point of the point cloud data frame, the point cloud column is obtained from the point cloud data frame, and the number of first detection points with continuous positions is determined according to the position information of the first detection point in the point cloud column. The column detection result of the point cloud column is determined according to the number of detection points with continuous positions in the point cloud column. The column detection result indicates whether the currently acquired point cloud column contains a sub-block occlusion area. If the number of detection points with continuous positions exceeds a preset number threshold, it can be considered that the currently acquired point cloud column contains a sub-block occlusion area, otherwise it does not contain a sub-block occlusion area.

[0073] Optionally, in order to improve the accuracy of detection, it is also possible to determine whether the first detection point is a target first detection point located in a sub-block area corresponding to the block-shaped occluder in the point cloud column according to the first detection point and the associated detection point associated with the first detection point, and determine the column detection result according to the target first detection point. That is, in one embodiment, the step of “determining the column detection result of the point cloud column according to the position information of the first detection point in the point cloud column” includes:

[0074] Determine at least one position-associated detection point according to the position information of the first detection point in the point cloud column;

[0075] If at least one position-associated detection point is a detection point corresponding to an obstruction attached to a window of the target device, determining the first detection point as a target first detection point;

[0076] Count the number of first detection points of targets with continuous positions in the point cloud column to obtain the row count of the sub-block occlusion area in the point cloud column;

[0077] The column detection result is obtained according to the relationship between the row count and the preset row number threshold.

[0078] The detection points in each row of the point cloud column have different acquisition angles in the vertical direction, and the angle formed by the acquisition angles of two adjacent detection points in the point cloud column in the vertical direction is the angular resolution of the target device.

[0079] Among them, the position-associated detection point can be the detection point of the first K rows of the first detection point, K is an integer, such as 2. For a first detection point, if the position-associated detection point is the detection point corresponding to the obstruction attached to the window of the target device, then the first detection point is the target first detection point.

[0080] The target first detection points with continuous positions in the point cloud column may refer to point cloud columns with continuous row numbers and continuous column positions, and may refer to point cloud columns with continuous column numbers.

[0081] The number of first detection points of the target with continuous positions is counted to obtain the row count of the sub-block occlusion area in the point cloud column.

[0082] The column detection result is obtained based on the relationship between the row count of the sub-block occlusion area and the preset row number threshold. Specifically, if the row count of the sub-block occlusion area is greater than the preset row number threshold, it is determined that the point cloud column contains the sub-block occlusion area; if the row count of the sub-block occlusion area is less than or equal to the preset row number threshold, it is determined that the point cloud column does not contain the sub-block occlusion area.

[0083] According to the column detection result of the point cloud column, the second window occlusion detection result is determined. For example, if the number of point cloud columns with continuous column positions and containing sub-block occlusion areas is greater than a preset column number threshold, the second window occlusion detection result is determined to be a block occlusion on the window. Optionally, the second window occlusion detection result may be determined in combination with the position of the sub-block occlusion area in the point cloud column. That is, in one embodiment, the step of "determining the second window occlusion detection result according to the column detection result" may include:

[0084] Determine, according to the column detection result, point cloud columns in the point cloud data frame whose column positions are continuous and whose contained sub-block occlusion regions match, and obtain the column count of the block occlusion regions in the point cloud data frame;

[0085] Determine a second frame detection result of the point cloud data frame according to a magnitude relationship between the column count and a preset column number threshold, the second frame detection result indicating whether the point cloud data frame is an occluded point cloud frame;

[0086] The second window obstruction detection result is determined according to the second frame detection result.

[0087] The point cloud columns with continuous column positions may refer to point cloud columns with continuous column numbers.

[0088] Sub-block occlusion area matching may mean that the distance of the row position of the sub-block occlusion area is less than a preset distance threshold, and the row position of the block occlusion area may be the row number of the center detection point of the sub-block occlusion area in the point cloud column, or the row number of the first detection point of the block occlusion area.

[0089] According to the column detection result, the number of point cloud columns in the point cloud data frame whose column positions are continuous and whose contained block occlusion areas match can be determined, that is, the column count.

[0090] The second frame detection result of the point cloud data frame is determined according to the relationship between the column count and the preset column number threshold. Specifically, after determining the column detection results of all point cloud columns in the point cloud data frame, that is, after traversing all point cloud columns in the point cloud data frame, the column count and the preset column number threshold are compared to determine the second frame detection result.

[0091] The second frame detection result indicates whether the point cloud data frame is an occluded point cloud frame. For example, if the column count is greater than the preset column count threshold, the second frame detection result is determined to be that the point cloud data frame is an occluded point cloud frame; if the column count is less than or equal to the preset column count threshold, the second frame detection result is determined to be that the point cloud data frame is not an occluded point cloud frame.

[0092] If the second frame detection result of the point cloud data frame is that the point cloud data frame is an occluded point cloud frame, it can be determined that the second window occlusion detection result is that there is a block occlusion in the window. If the second frame detection result of the point cloud data frame is that the point cloud data frame is not an occluded point cloud frame, it can be determined that the second window occlusion detection result is that there is no block occlusion in the window.

[0093] Optionally, the second window occlusion detection result may be determined in combination with the second frame detection result of the historical point cloud data frame. That is, in one embodiment, the step of “determining the second window occlusion detection result according to the second frame detection result” includes:

[0094] If the second frame detection result indicates that the point cloud data frame is an occluded point cloud frame, the second window occlusion detection result is determined according to the second frame detection result of at least one frame of historical point cloud data frame.

[0095] Among them, at least one frame of historical point cloud data frame can be the first m frames of point cloud data frame, where m is an integer, such as 3.

[0096] If the second frame detection result of the point cloud data frame indicates that the point cloud data frame is not an occluded point cloud frame, it is determined that the second window occluder detection result is that there is no block occluder in the window.

[0097] If the second frame detection result of the point cloud data frame indicates that the point cloud data frame is an occluded point cloud frame, and at least one frame of historical point cloud data frames is an occluded point cloud frame, then the second window occlusion detection result is determined to be that there is a block occlusion in the window.

[0098] If one of the at least one historical point cloud data frame is not an occluded point cloud frame, it is determined that the second window occluder detection result is that there is no block occluder in the window.

[0099] Optionally, the position information of the occluder on the window may also be determined based on the position information of the block occluded area in the point cloud data frame and the historical point cloud data frame. That is, in one embodiment, the second frame detection result includes the position information of the block occluded area, and the second window occluder detection result also indicates the position information of the block occluder on the target device. The step of "determining the second window occluder detection result based on the second frame detection result of at least one frame of historical point cloud data frame" may include:

[0100] If the point cloud data frame and the historical point cloud data frame are both occluded point cloud frames, the position information of the block occlusion on the window is determined according to the position information of the block occlusion area in the point cloud data frame and the position information of the block occlusion area in the historical point cloud data frame.

[0101] For example, if the point cloud data frame and the historical point cloud data frame are both occluded point cloud frames, the position information of the block occlusion area on the window is determined based on the position information of the block occlusion area in the point cloud data frame and the average position information of the position information of the block occlusion area in the historical point cloud data frame. The position information of the block occlusion area can be the position information of the detection point at the center of the block occlusion area.

[0102] In one embodiment, the point cloud columns can be acquired in sequence according to the acquisition angle of the detection point to detect the block-shaped obstruction on the window. The specific process can be as follows: Figure 3 As shown, according to the acquisition angle of the detection point, a point cloud column is obtained from the point cloud data frame, and the detection points in the point cloud column are obtained in sequence. According to the detection data of the currently obtained detection point, it is determined whether the currently obtained detection point is the first detection point. If the currently obtained detection point is the first detection point, it is determined whether at least N of the first M detection points obtained are the first detection points (for example, whether at least one of the first two detection points obtained is the first detection point). If at least N of the first M detection points obtained are the first detection points, the currently obtained first detection point is determined as the target first detection point, and the row count is increased by 1 (the initial value of the row count may be 0). If the number of first detection points in the first M detection points obtained is less than N, the row count is set to 1. If the currently obtained detection point is not the first detection point, the row count is set to 0.

[0103] After the detection of the detection points in a point cloud column is completed, determine whether the row count is greater than the preset row number threshold. If so and the row count corresponding to the previously acquired point cloud column is greater than the preset row number threshold, then determine whether the distance between the row starting positions of the sub-block occlusion area of ​​the currently acquired point cloud column and the previously acquired point cloud column is less than the preset distance threshold. If so, increase the column count by 1. If the row count corresponding to the previously acquired point cloud column is not greater than the preset row number threshold, or the distance between the row starting positions of the sub-block occlusion area of ​​the currently acquired point cloud column and the previously acquired point cloud column is not less than the preset distance threshold, then set the column count to 1 (the initial value of the row count can be 0).

[0104] If the row count corresponding to the currently acquired point cloud column is not greater than the preset row count threshold, the column count is set to 0. Before setting the column count to 0, it can be determined whether the column count is greater than the preset column count threshold. If so, the point cloud data frame is determined to be the second detection result as an occluded point cloud frame, and the next point cloud data frame is obtained for processing. If not, the column count is set to 0, and the next point cloud column is obtained for processing.

[0105] In one embodiment, the Figure 4 The obstruction detection system shown implements the window obstruction detection method of the device provided in the embodiment of the present application. Figure 4 The obstruction detection system shown can be applied in a field programmable gate array (FPGA). FPGA can be based on Figure 4 The illustrated architecture implements the obstruction detection method provided in the embodiments of the present application.

[0106] The obstruction detection system may include a data packet parsing module, a detection point counting module, a data extraction module, an obstruction level classification module, a frame counting module, a row and column counting module, a block obstruction detection module, a level alarm module and a block obstruction alarm module, and each module may run in parallel.

[0107] Among them, the data packet parsing module is used to parse the data packet to obtain a point cloud data frame, and send the point cloud data frame to the data extraction module. The data extraction module can extract the detection data required to determine whether the detection point is the first detection point, and give it to the detection point counting module for detection point counting. The detection point counting module can accumulate the number of first detection points in the point cloud data frame and the number of first target detection points with continuous positions in each point cloud column.

[0108] The detection point counting module sends the number of the first detection points to the occlusion level division module and the frame counting module. The occlusion level division module divides the occlusion levels according to the number of the first detection points, and sends the occlusion levels to the level alarm module. The frame counting module accumulates the number of frames that are occlusion point cloud frames according to the frame counting module, and sends the accumulated number of frames to the level alarm module. The level alarm module alarms according to the number of frames and the occlusion level of the occlusion point cloud frames.

[0109] The number of first target detection points with continuous positions in the detection point counting module is sent to the row and column counting module. The row and column counting module calculates the rows and columns of the block occlusion area and sends the calculation results to the block occlusion alarm module. The block occlusion alarm module issues an alarm based on the calculation results.

[0110] As can be seen from the above, the embodiment of the present application obtains a point cloud data frame collected by the target device, and the point cloud data frame includes detection data of multiple detection points; based on the detection data of the multiple detection points, a first detection point among the multiple detection points is determined, and the first detection point is the detection point corresponding to the obstruction attached to the window of the target device; based on the position information of the first detection point, the window is processed for obstruction detection, and the window obstruction detection result of the target device is obtained, so as to accurately detect whether there is an obstruction attached to the window of the target device.

[0111] In order to facilitate better implementation of the window obstruction detection method of the device provided in the embodiment of the present application, a window obstruction detection device of the device is also provided in one embodiment. The meanings of the terms are the same as those in the window obstruction detection method of the device described above, and the specific implementation details can refer to the description in the method embodiment.

[0112] The window obstruction detection device of the device can be integrated into an electronic device, such as Figure 5 As shown, the window obstruction detection device of the device may include: an acquisition unit 301, a determination unit 302 and a detection unit 303, which are specifically as follows:

[0113] (1) An acquisition unit 301 is used to acquire a point cloud data frame collected by a target device, where the point cloud data frame includes detection data of multiple detection points.

[0114] (2) A determination unit 302 is used to determine a first detection point among the multiple detection points based on the detection data of the multiple detection points, where the first detection point is a detection point corresponding to the obstruction attached to the window of the target device.

[0115] (3) A detection unit 303 is used to perform an obstruction detection process on the window according to the position information of the first detection point, and obtain a window obstruction detection result of the target device.

[0116] In one embodiment, the detection unit 303 may also be used to:

[0117] According to the position information of the first detection point, the window is subjected to an occlusion degree detection process to obtain a first window occlusion object detection result of the target device;

[0118] and / or, performing block-shaped occlusion detection processing on the window of the target device according to the position information of the first detection point to obtain a second window occlusion detection result of the target device;

[0119] A window obstruction detection result is obtained according to the first window obstruction detection result and / or the second window obstruction detection result.

[0120] In one embodiment, the detection unit 303 can also be used for:

[0121] Determining an occlusion coefficient of the first detection point based on the position information of the first detection point;

[0122] The occlusion coefficients of the first detection point are summed to obtain an occlusion index value corresponding to the point cloud data frame;

[0123] A first window occlusion object detection result of the target device is determined according to the occlusion index value.

[0124] In one embodiment, the detection unit 303 may also be used to:

[0125] For each first detection point, if the first detection point is located in the central area of ​​the point cloud data frame, a first occlusion coefficient is assigned to the first detection point;

[0126] If the first detection point is located outside the central area of ​​the point cloud data frame, a second occlusion coefficient is assigned to the first detection point, and the first occlusion coefficient is greater than the second occlusion coefficient.

[0127] In one embodiment, the detection unit 303 may also be used to:

[0128] Determine a first frame detection result of the point cloud data frame according to a magnitude relationship between the occlusion index value and a preset index threshold, wherein the first frame detection result indicates whether the point cloud data frame is an occluded point cloud frame and an occlusion level corresponding to the point cloud data frame;

[0129] A first window occlusion detection result is determined based on the first frame detection result, where the first window occlusion detection result includes an occlusion level of the window.

[0130] In one embodiment, the detection unit 303 can also be used to: if the first frame detection result indicates that the point cloud data frame is an occluded point cloud frame, determine the first window occlusion detection result based on the first frame detection result of at least one frame of historical point cloud data frame.

[0131] In one embodiment, the detection unit 303 may also be used to: obtain a point cloud column from the point cloud data frame according to the acquisition angle of the detection point, wherein the acquisition angles of the detection points in the point cloud column in the horizontal direction are the same;

[0132] Determine a column detection result of the point cloud column according to the position information of the first detection point in the point cloud column, the column detection result indicating whether the point cloud column contains a sub-block occlusion area;

[0133] The second window obstruction detection result is determined according to the column detection result.

[0134] In one embodiment, the detection unit 303 may also be used to: determine at least one position-associated detection point according to the position information of the first detection point in the point cloud column;

[0135] If at least one position-associated detection point is a detection point corresponding to an obstruction attached to a window of the target device, determining the first detection point as a target first detection point;

[0136] Count the number of first detection points of targets with continuous positions in the point cloud column to obtain the row count of the sub-block occlusion area in the point cloud column;

[0137] The column detection result is obtained according to the relationship between the row count and the preset row number threshold.

[0138] In one embodiment, the detection unit 303 may also be used to:

[0139] Determine, according to the column detection result, point cloud columns in the point cloud data frame whose column positions are continuous and whose contained sub-block occlusion regions match, and obtain the column count of the block occlusion regions in the point cloud data frame;

[0140] Determine a second frame detection result of the point cloud data frame according to a magnitude relationship between the column count and a preset column number threshold, the second frame detection result indicating whether the point cloud data frame is an occluded point cloud frame;

[0141] The second window obstruction detection result is determined according to the second frame detection result.

[0142] In one embodiment, the detection unit 303 may also be used to:

[0143] If the column count is greater than the preset column count threshold, it is determined that the second frame detection result is that the point cloud data frame is an occluded point cloud frame.

[0144] In one embodiment, the detection unit 303 may also be used to:

[0145] If the second frame detection result indicates that the point cloud data frame is an occluded point cloud frame, the second window occlusion detection result is determined according to the second frame detection result of at least one frame of historical point cloud data frame.

[0146] In one embodiment, the detection unit 303 may also be used to:

[0147] If the point cloud data frame and the historical point cloud data frame are both occluded point cloud frames, the position information of the block occlusion on the window is determined according to the position information of the block occlusion area in the point cloud data frame and the position information of the block occlusion area in the historical point cloud data frame.

[0148] As can be seen from the above, the embodiment of the present application obtains the point cloud data frame collected by the target device through the acquisition unit 301, and the point cloud data frame includes detection data of multiple detection points; the determination unit 302 determines the first detection point among the multiple detection points based on the detection data of the multiple detection points, and the first detection point is the detection point corresponding to the obstruction attached to the window of the target device; the detection unit 303 performs obstruction detection processing on the window based on the position information of the first detection point, and obtains the window obstruction detection result of the target device, which can accurately detect whether there is an obstruction attached to the window of the target device.

[0149] The present application also provides an electronic device, such as Figure 6 As shown, it shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:

[0150] The electronic device may include components such as a processor 1001 with one or more processing cores, a memory 1002 with one or more computer-readable storage media, a power supply 1003, and an input unit 1004. Those skilled in the art will appreciate that Figure 5 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0151] The processor 1001 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 1002 and calling data stored in the memory 1002, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 1001 may include one or more processing cores; preferably, the processor 1001 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, computer program, etc., and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1001.

[0152] The memory 1002 can be used to store software programs and modules. The processor 1001 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002. The memory 1002 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 1002 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 1002 may also include a memory controller to provide the processor 1001 with access to the memory 1002.

[0153] The electronic device also includes a power supply 1003 for supplying power to each component. Preferably, the power supply 1003 can be logically connected to the processor 1001 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 1003 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0154] The electronic device may further include an input unit 1004, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0155] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 1001 in the electronic device will load the executable files corresponding to the processes of one or more computer programs into the memory 1002 according to the following instructions, and the processor 1001 will run the computer programs stored in the memory 1002, thereby realizing various functions, as follows:

[0156] Obtaining a point cloud data frame collected by a target device, where the point cloud data frame contains detection data of multiple detection points;

[0157] Determine, according to the detection data of the multiple detection points, a first detection point among the multiple detection points, where the first detection point is a detection point corresponding to the obstruction attached to the window of the target device;

[0158] According to the position information of the first detection point, the window is subjected to obstruction detection processing to obtain the window obstruction detection result of the target device, so as to accurately detect whether there is an obstruction attached to the window of the target device.

[0159] The specific implementation of the above operations can be found in the previous embodiments and will not be described in detail here.

[0160] As can be seen from the above, the embodiment of the present application obtains a point cloud data frame collected by the target device, and the point cloud data frame includes detection data of multiple detection points; based on the detection data of the multiple detection points, a first detection point among the multiple detection points is determined, and the first detection point is the detection point corresponding to the obstruction attached to the window of the target device; based on the position information of the first detection point, the window is processed for obstruction detection, and the window obstruction detection result of the target device is obtained, so as to accurately detect whether there is an obstruction attached to the window of the target device.

[0161] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the methods provided in various optional implementations in the above-mentioned embodiments.

[0162] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0163] To this end, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program can be loaded by a processor to execute the window obstruction detection method of any device provided in the embodiment of the present application.

[0164] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0165] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0166] Since the computer program stored in the computer-readable storage medium can execute the window obstruction detection method of any device provided in the embodiments of the present application, the beneficial effects that can be achieved by the window obstruction detection method of any device provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0167] The above is a detailed introduction to the window obstruction detection method, device, electronic device and computer-readable storage medium of a device provided in an embodiment of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for detecting window obstructions of a device, characterized in that: include: Acquire a point cloud data frame collected by a target device, wherein the point cloud data frame includes detection data of multiple detection points; Determine, according to the detection data of the multiple detection points, a first detection point among the multiple detection points, where the first detection point is a detection point corresponding to the obstruction attached to the window of the target device; According to the position information of the first detection point, the window is subjected to obstruction detection processing to obtain the window obstruction detection result of the target device.

2. The method according to claim 1, characterized in that The performing obstruction detection processing on the window according to the position information of the first detection point to obtain the window obstruction detection result of the target device includes: According to the position information of the first detection point, the window is subjected to an occlusion degree detection process to obtain a first window occlusion object detection result of the target device; and / or, performing block-shaped occlusion detection processing on the window of the target device according to the position information of the first detection point to obtain a second window occlusion detection result of the target device; The window obstruction detection result is obtained according to the first window obstruction detection result and / or the second window obstruction detection result.

3. The method according to claim 2, characterized in that The step of performing an occlusion degree detection process on the window according to the position information of the first detection point to obtain a first window occlusion detection result of the target device includes: Determining an occlusion coefficient of the first detection point based on the position information of the first detection point; Sum the occlusion coefficients of the first detection points to obtain an occlusion index value corresponding to the point cloud data frame; A first window occlusion object detection result of the target device is determined according to the occlusion index value.

4. The method according to claim 3, characterized in that The determining the occlusion coefficient of the first detection point based on the position information of the first detection point includes: For each of the first detection points, if the first detection point is located in the central area of ​​the point cloud data frame, assigning a first occlusion coefficient to the first detection point; If the first detection point is located outside the central area of ​​the point cloud data frame, a second occlusion coefficient is assigned to the first detection point, and the first occlusion coefficient is greater than the second occlusion coefficient.

5. The method according to claim 3, characterized in that: The determining the first window occlusion object detection result of the target device according to the occlusion index value includes: Determine a first frame detection result of the point cloud data frame according to a magnitude relationship between the occlusion index value and a preset index threshold, wherein the first frame detection result indicates whether the point cloud data frame is an occlusion point cloud frame and an occlusion level corresponding to the point cloud data frame; The first window occlusion detection result is determined based on the first frame detection result, and the first window occlusion detection result includes an occlusion level of the window.

6. The method according to claim 5, characterized in that The determining the first window occlusion detection result based on the first frame detection result includes: If the first frame detection result indicates that the point cloud data frame is an occluded point cloud frame, the first window occlusion detection result is determined according to the first frame detection result of at least one frame of historical point cloud data frame.

7. The method according to any one of claims 2 to 6, characterized in that: The step of performing block-shaped occlusion detection processing on the window of the target device according to the position information of the first detection point to obtain a second window occlusion detection result of the target device includes: According to the acquisition angle of the detection point, a point cloud column is acquired from the point cloud data frame, wherein the acquisition angles of the detection points in the point cloud column in the horizontal direction are the same; Determining a column detection result of the point cloud column according to position information of a first detection point in the point cloud column, wherein the column detection result indicates whether the point cloud column includes a sub-block occlusion area; The second window obstruction detection result is determined according to the column detection result.

8. The method according to claim 7, characterized in that The step of determining the column detection result of the point cloud column according to the position information of the first detection point in the point cloud column comprises: Determine at least one position-associated detection point according to the position information of the first detection point in the point cloud sequence; If the at least one position-associated detection point is a detection point corresponding to an obstruction attached to a window of the target device, determining the first detection point as a target first detection point; Counting the number of first detection points of the target with continuous positions in the point cloud column to obtain a row count of the sub-block occlusion area in the point cloud column; The column detection result is obtained according to the relationship between the row count and a preset row number threshold.

9. The method according to claim 8, characterized in that The determining the second window obstruction detection result according to the column detection result includes: Determine, according to the column detection result, point cloud columns in the point cloud data frame whose column positions are continuous and whose contained sub-block occlusion regions match, and obtain a column count of the block occlusion regions in the point cloud data frame; Determine a second frame detection result of the point cloud data frame according to a magnitude relationship between the column count and a preset column number threshold, wherein the second frame detection result indicates whether the point cloud data frame is an occluded point cloud frame; A second window obstruction detection result is determined according to the second frame detection result.

10. The method according to claim 9, characterized in that The determining the second frame detection result of the point cloud data frame according to the magnitude relationship between the column count and a preset column number threshold comprises: If the column count is greater than the preset column count threshold, it is determined that the second frame detection result is that the point cloud data frame is an occluded point cloud frame.

11. The method according to claim 9, characterized in that The determining the second window obstruction detection result according to the second frame detection result includes: If the second frame detection result indicates that the point cloud data frame is an occluded point cloud frame, the second window occlusion detection result is determined according to the second frame detection result of at least one frame of historical point cloud data frame.

12. The method according to claim 11, characterized in that The second frame detection result includes position information of the block occlusion area, and the second window occlusion detection result also indicates position information of the block occlusion on the target device. The determining the second window occlusion detection result according to the second frame detection result of at least one frame of historical point cloud data frame includes: If the point cloud data frame and the historical point cloud data frame are both occluded point cloud frames, the position information of the block occlusion on the window is determined according to the position information of the block occlusion area in the point cloud data frame and the position information of the block occlusion area in the historical point cloud data frame.

13. A device for detecting window obstructions of a device, characterized in that: include: An acquisition unit, used to acquire a point cloud data frame collected by a target device, wherein the point cloud data frame includes detection data of multiple detection points; a determining unit, configured to determine, according to the detection data of the multiple detection points, a first detection point among the multiple detection points, wherein the first detection point is a detection point corresponding to an obstruction attached to a window of the target device; The detection unit is used to perform obstruction detection processing on the window according to the position information of the first detection point to obtain the window obstruction detection result of the target device.

14. An electronic device, characterized in that: It comprises a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the window obstruction detection method of the device according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is loaded by a processor to execute the window obstruction detection method of the device according to any one of claims 1 to 12.

Images