Dynamic space information interpretation and obstacle detection method and device based on first visual angle

Abstract

The invention discloses a dynamic space information interpretation and obstacle detection method and device based on a first visual angle, and the method comprises the steps: obtaining a three-dimensional point cloud through image obtaining equipment, and obtaining a local space point cloud under a current visual angle; comparing the local space point cloud with the existing global point cloud to obtain a first global space point cloud; calculating distances between the local space point clouds and all points in the first global space point cloud, and merging the local point clouds greater than a preset distance into the first global space point cloud to obtain a second global space point cloud; compressing the second global space point cloud to obtain a third global space point cloud; filtering point cloud data with a preset height in the third global space point cloud as an obstacle point cloud; and projecting the point clouds except the obstacle point clouds in the third global space point clouds to an X / Y plane, normalizing the coordinates to obtain updated coordinates, and performing obstacle detection according to the updated coordinates. The obvious limitation in the aspects of dynamic space information interpretation and obstacle detection in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, electronic device, computer-readable storage medium and computer program product for dynamic spatial information interpretation and obstacle detection based on a first-person perspective. Background Art

[0002] In the field of computer vision, traditional visual methods mainly rely on 2D image processing technology. These methods include but are not limited to edge detection, corner detection, etc., which identify objects and scenes by analyzing the color, texture and other features of the image. Although these technologies have achieved certain results in some applications, they have obvious limitations. First, these methods are very sensitive to lighting conditions, resulting in large performance fluctuations under different lighting environments. Second, since they mainly process 2D images, they are limited in their ability to process complex three-dimensional structures. Finally, these methods have poor real-time performance and are difficult to meet the needs of fast dynamic scenes.

[0003] In recent years, deep learning methods, especially convolutional neural networks (CNNs), have made significant progress in the fields of image recognition and obstacle detection. These methods are able to use deep network structures to perform semantic segmentation on data acquired by sensors or directly detect obstacles. Deep learning methods improve the accuracy and robustness of recognition by learning high-level features of data. However, these methods also have some defects. The most important problem is that they require a large amount of labeled data for training, which not only increases the workload of data preparation but also limits the generalization ability of the model. In addition, the deployment of deep learning models is relatively difficult and requires high computing resources, which is a challenge in resource-constrained environments. Moreover, since deep learning models are usually trained on limited datasets, they may lack generalization ability when encountering unseen scenes, resulting in performance degradation.

[0004] In summary, existing technologies have obvious limitations in dynamic spatial information interpretation and obstacle detection, especially in terms of real-time performance, generalization ability, and the ability to handle complex three-dimensional structures. Summary of the invention

[0005] In view of this, the technical problem to be solved by the present invention is to provide a method, device, electronic device, computer-readable storage medium and computer program product for dynamic spatial information interpretation and obstacle detection based on the first perspective, so as to solve the obvious limitations of the prior art in dynamic spatial information interpretation and obstacle detection, especially in terms of real-time performance, generalization capability and processing capability of complex three-dimensional structures.

[0006] In order to solve the above technical problems, the present invention provides a method for interpreting dynamic spatial information and detecting obstacles based on a first-person perspective, the method comprising: Acquire a three-dimensional point cloud through an image acquisition device to obtain a local space point cloud at the current viewing angle; Compare the local space point cloud with the existing global point cloud, remove overlapping point clouds, and obtain a first global space point cloud; wherein the existing global point cloud includes a point cloud obtained from the first viewing angle; Calculate the distance between the local space point cloud and all points in the first global space point cloud, take the point clouds less than or equal to the preset distance as the coincident point clouds, merge the local point clouds greater than the preset distance into the first global space point cloud, and obtain the second global space point cloud; compressing the second global space point cloud to obtain a third global space point cloud; Filtering point cloud data of a preset height in the third global space point cloud as an obstacle point cloud; The point cloud excluding the obstacle point cloud in the third global space point cloud is projected onto the X / Y plane, and the coordinates are normalized to obtain updated coordinates, and obstacle detection is performed according to the updated coordinates.

[0007] Optionally, comparing the local space point cloud with an existing global point cloud and removing overlapping point clouds to obtain a first global space point cloud includes: Calculating the distance between the existing global point cloud and the local space point cloud; Filter the point cloud within a predetermined threshold distance range to obtain a preliminary point cloud index to be filtered; The overlapping point clouds are removed according to the preliminary point cloud index to be filtered to obtain a first global space point cloud.

[0008] Optionally, the method further comprises: The outliers of the preliminary point cloud index to be filtered are calculated by a statistical method, the outliers are removed, and the remaining point cloud index to be filtered is retained.

[0009] Optionally, the method further comprises: Calculate the angle between the remaining point cloud index to be filtered and the position of the intelligent body; The point cloud indexes that meet the predetermined threshold angle range are retained, and the remaining point cloud indexes are the point cloud indexes that need to be filtered in the end.

[0010] Optionally, calculating the angle between the remaining to-be-filtered point cloud index and the position of the agent includes: Step S1: Calculate the vector X between the remaining point cloud to be filtered and the intelligent agent; Step S2: normalize vector X to obtain X'; Step S3: Normalize the agent's viewing direction to obtain Y' Step S4: Calculate the angle between vectors X' and Y', that is, calculate the cosine angle by dot product, and convert the unit into degrees to obtain D; Step S5: Calculate the maximum value Max and the minimum value Min of D to obtain the range L, retain a certain range threshold r=0.1, and obtain the range Min': Min+L*r, Max': Max-L*r; Step S6: filter the index of the angle D that satisfies the range calculated in step S5, and specifically has different ranges for different axes, X axis: Min-Max'; Y axis: Min-Max'; Z axis: Min'-Max'.

[0011] Optionally, calculating the outliers of the preliminary point cloud index to be filtered by a statistical method, removing the outliers, and retaining the remaining point cloud index to be filtered includes: By calculating the mean and the square difference of the preliminary point cloud indexes to be filtered, the point cloud indexes to be filtered whose mean and the square difference meet the preset threshold range are retained.

[0012] Optionally, compressing the second global space point cloud to obtain a third global space point cloud includes: Downsampling the second global space point cloud based on a uniform distribution; According to the preset data level of the point cloud, the search radius and the threshold of the number of neighboring points are obtained; Downsampling is performed based on the search radius and the threshold of the number of neighboring points to obtain the third global space point cloud.

[0013] A second aspect of the present invention provides a dynamic spatial information interpretation and obstacle detection device based on a first perspective, the device comprising: A first acquisition module is used to acquire a three-dimensional point cloud through an image acquisition device to obtain a local space point cloud under a current viewing angle; A second acquisition module is used to compare the local space point cloud with an existing global point cloud, remove overlapping point clouds, and obtain a first global space point cloud; wherein the existing global point cloud includes a point cloud obtained from a first viewing angle; A third acquisition module is used to calculate the distance between the local space point cloud and all points in the first global space point cloud, take the point cloud less than or equal to the preset distance as the coincident point cloud, and merge the local point cloud greater than the preset distance into the first global space point cloud to obtain a second global space point cloud; A compression module, used for compressing the second global space point cloud to obtain a third global space point cloud; A filtering module, used for filtering point cloud data of a preset height in the third global space point cloud as an obstacle point cloud; The detection module is used to project the point cloud in the third global space point cloud except the obstacle point cloud onto the X / Y plane, normalize the coordinates to obtain updated coordinates, and perform obstacle detection according to the updated coordinates.

[0014] Optionally, the second acquisition module includes: A first calculation unit, used for calculating the distance between the existing global point cloud and the local space point cloud; A screening unit, used to screen point clouds within a predetermined threshold distance range to obtain a preliminary point cloud index to be filtered; The first acquisition unit is used to remove overlapping point clouds according to the preliminary point cloud index to be filtered to obtain a first global space point cloud.

[0015] Optionally, the second acquisition module further includes: The second acquisition unit is used to calculate the outlier points of the preliminary point cloud index to be filtered by a statistical method, remove the outlier points, and retain the remaining point cloud index to be filtered.

[0016] Optionally, the second acquisition module further includes: A second calculation unit, used to calculate the angle between the remaining to-be-filtered point cloud index and the position of the intelligent body; The third acquisition unit is used to retain the point cloud indexes that meet the predetermined threshold angle range, and the other remaining point cloud indexes are the point cloud indexes that need to be filtered in the end.

[0017] According to a third aspect of the present invention, there is provided an electronic device, comprising: Processor; and Memory for storing programs, The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to any one of the above-mentioned first aspects.

[0018] A fourth aspect of the present invention provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute a method according to any one of the above-mentioned first aspects.

[0019] A fifth aspect of the present invention provides a computer program product, wherein the computer program product comprises instructions, and when the instructions are executed, the computer executes any one of the methods described in the first aspect.

[0020] Through this optional embodiment, a three-dimensional point cloud is acquired through an image acquisition device to obtain a local space point cloud under the current viewing angle; the local space point cloud is compared with an existing global point cloud, and overlapping point clouds are removed to obtain a first global space point cloud; wherein the existing global point cloud includes a point cloud obtained from the first viewing angle; the distance between the local space point cloud and all points in the first global space point cloud is calculated, and point clouds less than or equal to a preset distance are regarded as overlapping point clouds, and local point clouds greater than the preset distance are merged into the first global space point cloud to obtain a second global space point cloud; the second global space point cloud is compressed to obtain a third global space point cloud; point cloud data of a preset height in the third global space point cloud is filtered as an obstacle point cloud; the point cloud in the third global space point cloud excluding the obstacle point cloud is projected onto the X / Y plane, and the coordinates are normalized to obtain updated coordinates, and obstacle detection is performed according to the updated coordinates. The obvious limitations in the prior art in the interpretation of dynamic spatial information and obstacle detection are solved, especially in terms of real-time performance, generalization ability and processing ability of complex three-dimensional structures, and the accuracy, robustness and real-time performance of the system are improved.

[0021] It should be understood that the above general description and the following detailed description are merely exemplary and illustrative and are not intended to limit the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings below are part of the specification of the present invention, and illustrate exemplary embodiments of the present invention. Together with the description of the specification, the accompanying drawings are used to explain the principles of the present invention.

[0023] Figure 1 It is a flowchart of a method for interpreting dynamic spatial information and detecting obstacles based on a first perspective according to an embodiment of the present invention.

[0024] Figure 2 A structural block diagram of a dynamic spatial information interpretation and obstacle detection device based on a first-person perspective provided in a specific embodiment of the present invention; Figure 3 A structural block diagram of a second acquisition module provided in a specific embodiment of the present invention; Figure 4 Another structural block diagram of a second acquisition module provided in a specific embodiment of the present invention; Figure 5 Another structural block diagram of the second acquisition module provided in a specific embodiment of the present invention; Figure 6 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0025] In order to make the objectives, technical solutions and advantages of the embodiments of the present invention more clearly understood, the spirit of the contents disclosed by the present invention will be clearly explained with the accompanying drawings and detailed descriptions below. After understanding the embodiments of the contents of the present invention, any technician in the relevant technical field can change and modify the techniques taught by the contents of the present invention without departing from the spirit and scope of the contents of the present invention.

[0026] The exemplary embodiments and descriptions of the present invention are used to explain the present invention, but are not intended to limit the present invention. In addition, elements / components with the same or similar reference numerals used in the drawings and embodiments are used to represent the same or similar parts.

[0027] The terms “first”, “second”, etc. used in this document do not particularly refer to an order or sequence, nor are they used to limit the present invention. They are only used to distinguish elements or operations described with the same technical terms.

[0028] The directional terms used in this document, such as up, down, left, right, front or back, etc., are only used to refer to the directions of the drawings. Therefore, the directional terms used are used to illustrate and not to limit the present invention.

[0029] The words “include,” “including,” “have,” “contain,” etc. used in this document are open-ended terms, meaning including but not limited to.

[0030] As used herein, "and / or" includes any or all combinations of the items described.

[0031] As used herein, “plurality” includes “two” and “more than two”; as used herein, “plurality groups” includes “two groups” and “more than two groups”.

[0032] The terms "substantially" and "approximately" used herein are used to modify any quantity or error that may vary slightly, but these slight changes or errors do not change their essence. Generally speaking, the range of slight changes or errors modified by such terms may be 20% in some embodiments, 10% in some embodiments, 5% in some embodiments, or other values. Those skilled in the art should understand that the aforementioned values ​​can be adjusted according to actual needs and are not limited thereto.

[0033] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0034] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but not be limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or a system having A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but not be limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or a system having A, B, C, etc.). Those skilled in the art should also understand that any transitional conjunctions and / or phrases that substantially represent two or more optional items, whether in the specification, claims, or drawings, should be understood to give the possibility of including one of these items, either of these items, or both of these items. For example, the phrase "A or B" should be understood to include the possibilities of "A" or "B", or "A and B".

[0035] By analyzing existing technical solutions, we found that the difficulty of first-person perspective dynamic spatial information interpretation and obstacle detection lies in how to effectively handle complex changes in dynamic environments while maintaining high accuracy and real-time performance. Specifically, the challenges include but are not limited to: Adapt to complex and ever-changing scenarios: The environment faced by the intelligent agent may be constantly changing, with new scenes and objects appearing and old objects disappearing. The algorithm needs to be flexible enough to cope with various situations, update its understanding of the scene in a timely manner, and ensure stable and reliable performance.

[0036] Data redundancy and key information extraction: As the intelligent agent explores the environment, the amount of first-person data acquired increases dramatically. It becomes a major challenge to efficiently extract useful information from massive data and reduce unnecessary computational overhead.

[0037] In this embodiment, a dynamic spatial information interpretation and obstacle detection method based on the first perspective is provided. Figure 1 is a flow chart of a method for interpreting dynamic spatial information and detecting obstacles based on a first-person perspective according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps: Step S101: Acquire a three-dimensional point cloud through an image acquisition device to obtain a local space point cloud at the current viewing angle. In an optional embodiment, depth information is obtained through a depth camera or other sensor, and the depth information is converted into 3D point cloud data, that is, each pixel corresponds to a point in 3D space. Use a voxel downsampling method to reduce the point cloud density, thereby reducing the amount of calculation and removing noise, and obtain a local space point cloud at the current viewing angle.

[0038] Step S102: Compare the local space point cloud with the existing global point cloud, remove the overlapping point clouds, and obtain the first global space point cloud; wherein the existing global point cloud includes the point cloud obtained from the first viewing angle, thereby filtering the global space point cloud. By comparing and fusing the local space point cloud with the global point cloud, the global point cloud is effectively updated, ensuring the integrity and real-time performance of the global point cloud, thereby improving the comprehensiveness of the spatial information.

[0039] Step S103: Calculate the distance between the local space point cloud and all points in the first global space point cloud, take the point cloud less than or equal to the preset distance as the overlapped point cloud, merge the local point cloud greater than the preset distance into the first global space point cloud, and obtain the second global space point cloud, thereby updating the global space point cloud. In an optional example, calculate the distance between the local point cloud and all points in the global point cloud to determine which points can be considered overlapped. Retain or merge the point cloud according to the set threshold to update the global point cloud model. First, calculate the distance between the current local point cloud and the global point cloud, set different retention distances according to different orders of magnitude of the point cloud, obtain the distance threshold corresponding to the current number of global point clouds, record the point cloud less than a certain distance threshold as the overlapped point cloud, and merge the remaining current local point cloud into the global point cloud. By calculating the distance between the local space point cloud and all points in the global point cloud, and merging the point cloud according to the preset distance, the method can more accurately identify and merge the overlapped point cloud, thereby enhancing the reliability of obstacle detection.

[0040] Step S104: compress the second global space point cloud to obtain a third global space point cloud, which reduces the data volume of the global space point cloud, improves the data processing speed, and facilitates storage and transmission. Further, in an optional embodiment, the second global space point cloud data is preprocessed, including denoising, filtering, gridding and other steps to improve the effect of subsequent compression processing. Step S105: filter the point cloud data of the preset height in the third global space point cloud as an obstacle point cloud.

[0041] Step S106: Project the point cloud except the obstacle point cloud in the third global space point cloud onto the X / Y plane, normalize the coordinates, obtain the updated coordinates, and perform obstacle detection according to the updated coordinates. Projecting the point cloud except the obstacle point cloud onto the X / Y plane and normalizing the coordinates can more accurately perform obstacle detection on the two-dimensional plane, reducing interference factors in the three-dimensional space. In an optional embodiment, obstacle detection is exemplified and explained, and the height is filtered: the point cloud data is the 3D coordinates of XYZ. For example, it is necessary to filter the point cloud data with a height Z range of 0-100 as obstacle detection, that is, only the point cloud data of 0-100 on the Z axis is retained, and projected onto the XY plane, which can be regarded as a top view, one part of which is the point cloud data with only XY coordinates retained after filtering, and the other part is the detected straight line obstacle. By filtering the point cloud data of the preset height as the obstacle point cloud, potential obstacles can be quickly identified, unnecessary complex calculations are avoided, and the speed of obstacle detection is improved. In another optional embodiment, first, according to different downstream requirements, point cloud data of different heights are filtered as pre-selected obstacle point clouds. Project the remaining points onto the X / Y plane and normalize the coordinates to obtain the new coordinates X'Y'. Initialize the 2D image with the current data coordinates, set the color value of the obstacle location, set the obstacle to 255, and set the others to 0. This optional implementation emphasizes the flexible adjustment of point cloud processing parameters according to actual needs (such as different downstream needs), and proposes that an incremental update strategy can be applied to optimize the update process of the global point cloud model, thereby improving the response speed and efficiency of the system. Use Hough Transform to detect straight line features, identify obstacle edges, and return the starting coordinates of all straight lines. Apply the traditional Hough Transform to the 2D projection image after 3D point cloud processing to detect straight obstacle edges, which provides a novel and effective way to extract key obstacles from complex first-person perspective information.

[0042] The linear coordinates are denormalized to the original 3D space and a 3D bounding box is calculated to define the obstacle range.

[0043] Through the above method steps, the method of comparing and updating the local point cloud with the global point cloud can adapt to environmental changes in real time and ensure that the intelligent agent's understanding of the surrounding environment is always up to date. This method not only takes into account static environmental factors, but also can identify and respond to dynamic obstacles. It is suitable for a variety of environments and scenarios, is not restricted by specific conditions, and can be widely used in fields such as autonomous driving, robot navigation, and drone monitoring. It can more accurately interpret dynamic spatial information, reduce blind spots and errors caused by viewing angle limitations, and can quickly update the global point cloud and perform obstacle detection. Therefore, it has good real-time performance, can respond to environmental changes in a timely manner, and ensure the safety and reliability of the system.

[0044] The above step S102 involves comparing the local space point cloud with the existing global point cloud, removing overlapping point clouds, and obtaining a first global space point cloud. In an optional embodiment, the distance between the existing global point cloud and the local space point cloud is calculated, and the point cloud within a predetermined threshold distance range is screened to obtain a preliminary point cloud index to be filtered. The overlapping point cloud is removed according to the preliminary point cloud index to be filtered to obtain the first global space point cloud. The local point cloud is compared with the existing global point cloud, and the overlapping part is removed to avoid redundancy. This step is completed by filtering three times. This step is the first filtering. The first filtering is to filter out nearby points based on the distance threshold. By calculating the distance between the global space point cloud and the current local space point cloud, the point cloud within a certain threshold range is screened to obtain a preliminary point cloud index to be filtered. Here, the certain threshold range, for example, the threshold is 5-25cm, and the acquisition method is obtained based on statistical analysis of past data.

[0045] After that, a second filtering is performed, and the outliers of the preliminary point cloud index to be filtered are calculated by statistical methods, the outliers are removed, and the remaining point cloud indexes to be filtered are retained. In an optional embodiment, by calculating the mean and square difference of the preliminary point cloud index to be filtered, the point cloud index to be filtered whose mean and square difference meet the preset threshold range is retained. Specifically, by calculating the mean and square difference of this part of the index, (x-mean) / std is used to retain the index to be filtered that meets the threshold of 0.3-1.0. The second filtering uses statistical methods to remove outliers to ensure the quality of the point cloud. Point cloud compression is performed using a downsampling method based on uniform distribution and a threshold of the number of neighboring points, which not only maintains the key geometric features of the scene, but also significantly reduces the amount of data and improves processing efficiency.

[0046] Then, the third filtering is performed to calculate the angle between the remaining point cloud index to be filtered and the position of the intelligent body, and the point cloud index within the predetermined threshold angle range is retained. The other remaining point cloud indexes are the point cloud indexes that need to be filtered. In a specific optional embodiment, the specific implementation method of calculating the angle between the remaining point cloud index to be filtered and the position of the intelligent body is: Step S1: Calculate the vector X between the remaining point cloud to be filtered and the intelligent agent; Step S2: normalize vector X to obtain X'; Step S3: Normalize the agent's viewing direction to obtain Y' Step S4: Calculate the angle between vectors X' and Y', that is, calculate the cosine angle by dot product, and convert the unit into degrees to obtain D; Step S5: Calculate the maximum value Max and the minimum value Min of D to obtain the range L, retain a certain range threshold r=0.1, and obtain the range Min': Min+L*r, Max': Max-L*r; Step S6: filter the index of the angle D that meets the calculation range in step S5, with different ranges for different axes, X axis: Min-Max'; Y axis: Min-Max'; Z axis: Min'-Max'. Calculate the angle between the point cloud to be filtered and the position of the intelligent body, retain the point cloud index that meets a certain threshold range, and the remaining indexes are the point cloud indexes that need to be filtered in the end.

[0047] This optional embodiment introduces a multi-stage point cloud filtering mechanism, and refines the global space point cloud by three filterings of distance threshold, statistical method to remove outliers and angle limit, so as to ensure the accuracy of data and reduce redundancy. The point cloud set is further refined according to the angle between the point and the intelligent body by using angle limit through the third filtering. The above step S104 involves compressing the second global space point cloud to obtain the third global space point cloud. In an optional embodiment, the second global space point cloud is downsampled based on uniform distribution, and the search radius and the threshold of the number of neighboring points are obtained according to the preset data level of the point cloud. Downsampling is performed based on the search radius and the threshold of the number of neighboring points to obtain the third global space point cloud. After the above downsampling steps, a point cloud data set with moderate density and maintained accuracy is obtained. The downsampled point cloud data set is used as the latest global space point cloud information for subsequent spatial information interpretation, obstacle detection or other applications. It can significantly reduce the amount of data and improve data processing efficiency while maintaining the characteristics of point cloud data, and is suitable for various application scenarios that require efficient processing of three-dimensional point cloud data.

[0048] In summary, this optional embodiment implements: (1) a multi-stage point cloud filtering method: a method for interpreting first-person spatial information in a dynamic environment, which refines the global spatial point cloud through three filtering steps (distance threshold screening, statistical outlier removal, and angle restriction) to reduce redundant data and maintain high accuracy; (2) a local and global point cloud comparison and update mechanism: an intelligent agent environment perception system, which includes an update module that can compare locally acquired point cloud data with an existing global point cloud model in real time, and retain or merge point clouds according to a set distance threshold, thereby achieving rapid response to dynamic environmental changes; (3) an obstacle detection technology combined with Hough transform point cloud processing: a method based on first-person point cloud data acquired by a depth camera or other sensor, by projecting the remaining points onto the X / Y plane and using Hough transform to detect straight line features to identify the edge of obstacles, and a technology that returns the starting coordinates of all straight lines to define the range of obstacles.

[0049] In this embodiment, a dynamic spatial information interpretation and obstacle detection device based on a first perspective is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the term "module" may be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0050] The embodiment of the present invention provides a dynamic spatial information interpretation and obstacle detection device based on a first-person perspective, such as Figure 2 As shown, the device comprises: A first acquisition module 21 is used to acquire a three-dimensional point cloud through an image acquisition device to obtain a local space point cloud under a current viewing angle; A second acquisition module 22 is used to compare the local space point cloud with an existing global point cloud, remove overlapping point clouds, and obtain a first global space point cloud; wherein the existing global point cloud includes a point cloud obtained from a first viewing angle; A third acquisition module 23 is used to calculate the distance between the local space point cloud and all points in the first global space point cloud, take the point cloud less than or equal to the preset distance as the coincident point cloud, and merge the local point cloud greater than the preset distance into the first global space point cloud to obtain a second global space point cloud; A compression module 24, configured to compress the second global space point cloud to obtain a third global space point cloud; A filtering module 25, configured to filter point cloud data at a preset height in the third global space point cloud as an obstacle point cloud; The detection module 26 is used to project the point cloud in the third global space point cloud except the obstacle point cloud onto the X / Y plane, normalize the coordinates to obtain updated coordinates, and perform obstacle detection according to the updated coordinates.

[0051] Alternatively, if Figure 3 As shown, the second acquisition module 22 includes: A first calculation unit 221, used to calculate the distance between the existing global point cloud and the local space point cloud; A screening unit 222 is used to screen point clouds within a predetermined threshold distance range to obtain a preliminary point cloud index to be filtered; The first acquisition unit 223 is used to remove overlapping point clouds according to the preliminary point cloud index to be filtered to obtain a first global space point cloud.

[0052] Alternatively, if Figure 4 As shown, the second acquisition module 22 also includes: The second acquisition unit 224 is used to calculate the outlier points of the preliminary point cloud index to be filtered by a statistical method, remove the outlier points, and retain the remaining point cloud index to be filtered.

[0053] Alternatively, if Figure 5 As shown, the second acquisition module 22 also includes: A second calculation unit 226 is used to calculate the angle between the remaining to-be-filtered point cloud index and the position of the agent; The third acquisition unit 227 is used to retain the point cloud indexes that meet the predetermined threshold angle range, and the other remaining point cloud indexes are the point cloud indexes that need to be filtered in the end.

[0054] The further functional description of each of the above modules is the same as that of the above corresponding embodiments and will not be repeated here.

[0055] The exemplary embodiment of the present invention further provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program that can be executed by the at least one processor, and the computer program is used to enable the electronic device to perform a method according to an embodiment of the present invention when executed by the at least one processor.

[0056] Exemplary embodiments of the present invention also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to perform a method according to an embodiment of the present invention.

[0057] An exemplary embodiment of the present invention further provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor of a computer, the computer is used to enable the computer to perform a method according to an embodiment of the present invention.

[0058] refer to Figure 6 , a block diagram of an electronic device that can be used as a server or client of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0059] like Figure 6As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0060] A plurality of components in the electronic device 600 are connected to the I / O interface 605, including: an input unit 606, an output unit 607, a storage unit 608, and a communication unit 609. The input unit 606 may be any type of device capable of inputting information to the electronic device 600, and the input unit 606 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 607 may be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 608 may include, but is not limited to, a disk, an optical disk. The communication unit 609 allows the electronic device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0061] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above. For example, in some embodiments, the music data processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 600 via the ROM 602 and / or the communication unit 609. In some embodiments, the computing unit 601 may be configured to perform a method according to an embodiment of the present invention in any other appropriate manner (e.g., by means of firmware).

[0062] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0063] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0064] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0065] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0066] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0067] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0068] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A method for interpreting dynamic spatial information and detecting obstacles based on the first-person perspective, characterized in that: The method comprises: Acquire a three-dimensional point cloud through an image acquisition device to obtain a local space point cloud at the current viewing angle; Compare the local space point cloud with the existing global point cloud, remove overlapping point clouds, and obtain a first global space point cloud; wherein the existing global point cloud includes a point cloud obtained from the first viewing angle; Calculate the distance between the local space point cloud and all points in the first global space point cloud, take the point clouds less than or equal to the preset distance as the coincident point clouds, merge the local point clouds greater than the preset distance into the first global space point cloud, and obtain the second global space point cloud; compressing the second global space point cloud to obtain a third global space point cloud; Filtering point cloud data of a preset height in the third global space point cloud as an obstacle point cloud; The point cloud excluding the obstacle point cloud in the third global space point cloud is projected onto the X / Y plane, and the coordinates are normalized to obtain updated coordinates, and obstacle detection is performed according to the updated coordinates.

2. The method for dynamic spatial information interpretation and obstacle detection based on the first perspective according to claim 1, characterized in that: Comparing the local space point cloud with the existing global point cloud and removing overlapping point clouds to obtain a first global space point cloud includes: Calculating the distance between the existing global point cloud and the local space point cloud; Filter the point cloud within a predetermined threshold distance range to obtain a preliminary point cloud index to be filtered; The overlapping point clouds are removed according to the preliminary point cloud index to be filtered to obtain a first global space point cloud.

3. The method for dynamic spatial information interpretation and obstacle detection based on the first perspective according to claim 2, characterized in that: The method further comprises: The outliers of the preliminary point cloud index to be filtered are calculated by a statistical method, the outliers are removed, and the remaining point cloud index to be filtered is retained.

4. The method for dynamic spatial information interpretation and obstacle detection based on the first perspective according to claim 3, characterized in that: The method further comprises: Calculate the angle between the remaining point cloud index to be filtered and the position of the intelligent body; The point cloud indexes that meet the predetermined threshold angle range are retained, and the remaining point cloud indexes are the point cloud indexes that need to be filtered in the end.

5. The method for dynamic spatial information interpretation and obstacle detection based on the first perspective according to claim 4, characterized in that: Calculating the angle between the remaining to-be-filtered point cloud index and the position of the agent includes: Step S1: Calculate the vector X between the remaining point cloud to be filtered and the intelligent agent; Step S2: normalize vector X to obtain X'; Step S3: Normalize the agent's viewing direction to obtain Y' Step S4: Calculate the angle between vectors X' and Y', that is, calculate the cosine angle by dot product, and convert the unit into degrees to obtain D; Step S5: Calculate the maximum value Max and the minimum value Min of D to obtain the range L, retain a certain range threshold r=0.1, and obtain the range Min': Min+L*r, Max': Max-L*r; Step S6: filter the index of the angle D that satisfies the range calculated in step S5, and specifically has different ranges for different axes, X axis: Min-Max'; Y axis: Min-Max'; Z axis: Min'-Max'.

6. The method for interpreting dynamic spatial information and detecting obstacles based on the first perspective according to claim 3, characterized in that: Calculating the outliers of the preliminary point cloud index to be filtered by a statistical method, removing the outliers, and retaining the remaining point cloud index to be filtered includes: By calculating the mean and the square difference of the preliminary point cloud indexes to be filtered, the point cloud indexes to be filtered whose mean and the square difference meet the preset threshold range are retained.

7. The method for dynamic spatial information interpretation and obstacle detection based on the first perspective according to claim 1, characterized in that: Compressing the second global space point cloud to obtain a third global space point cloud includes: Downsampling the second global space point cloud based on a uniform distribution; According to the preset data level of the point cloud, the search radius and the threshold of the number of neighboring points are obtained; Downsampling is performed based on the search radius and the threshold of the number of neighboring points to obtain the third global space point cloud.

8. A dynamic spatial information interpretation and obstacle detection device based on the first perspective, characterized in that: The device comprises: A first acquisition module is used to acquire a three-dimensional point cloud through an image acquisition device to obtain a local space point cloud under a current viewing angle; A second acquisition module is used to compare the local space point cloud with an existing global point cloud, remove overlapping point clouds, and obtain a first global space point cloud; wherein the existing global point cloud includes a point cloud obtained from a first viewing angle; A third acquisition module is used to calculate the distance between the local space point cloud and all points in the first global space point cloud, take the point cloud less than or equal to the preset distance as the coincident point cloud, and merge the local point cloud greater than the preset distance into the first global space point cloud to obtain a second global space point cloud; A compression module, used for compressing the second global space point cloud to obtain a third global space point cloud; A filtering module, used for filtering point cloud data of a preset height in the third global space point cloud as an obstacle point cloud; The detection module is used to project the point cloud in the third global space point cloud except the obstacle point cloud onto the X / Y plane, normalize the coordinates to obtain updated coordinates, and perform obstacle detection according to the updated coordinates.

9. The first-viewpoint dynamic spatial information interpretation and obstacle detection device according to claim 8, characterized in that: The second acquisition module includes: A first calculation unit, used for calculating the distance between the existing global point cloud and the local space point cloud; A screening unit, used to screen point clouds within a predetermined threshold distance range to obtain a preliminary point cloud index to be filtered; The first acquisition unit is used to remove overlapping point clouds according to the preliminary point cloud index to be filtered to obtain a first global space point cloud.

10. The method for interpreting dynamic spatial information and detecting obstacles based on the first perspective according to claim 9, characterized in that: The second acquisition module also includes: The second acquisition unit is used to calculate the outlier points of the preliminary point cloud index to be filtered by a statistical method, remove the outlier points, and retain the remaining point cloud index to be filtered.

11. The method for interpreting dynamic spatial information and detecting obstacles based on the first perspective according to claim 10, characterized in that: The second acquisition module also includes: A second calculation unit, used to calculate the angle between the remaining to-be-filtered point cloud index and the position of the intelligent body; The third acquisition unit is used to retain the point cloud indexes that meet the predetermined threshold angle range, and the other remaining point cloud indexes are the point cloud indexes that need to be filtered in the end.

12. An electronic device, comprising: processor; as well as Memory for storing programs, The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 7.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1-7.

14. A computer program product, characterized in that The computer program product comprises instructions which, when executed, cause a computer to perform the method of any one of claims 1 to 7.

Images