A method, apparatus, equipment and medium for detecting obstacles in a scrap steel yard.

Abstract

This application relates to the field of computer technology and provides a method, apparatus, equipment, and medium for obstacle detection in a scrap steel yard. The method includes: acquiring three-dimensional point cloud data of the target scrap steel yard environment; preprocessing the three-dimensional point cloud data to obtain scrap steel operation environment point cloud data; transmitting the scrap steel operation environment point cloud data to a pre-trained classification model to obtain category information and coordinate information of multiple obstacles; and performing surface fitting based on the point cloud data corresponding to each obstacle to obtain the surface geometric information of each obstacle, thereby completing the detection of obstacles within the target scrap steel yard. The solution in this application realizes the identification of obstacles within the scrap steel yard, and by combining the surface fitting method, improves the efficiency and accuracy of acquiring obstacle category information, coordinate information, and surface geometric information.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a method, apparatus, equipment, and medium for detecting obstacles in a scrap steel yard. Background Technology

[0002] Scrap steel is a fundamental raw material for steel production, but the level of automation in its logistics and transportation processes is relatively low. Limited by existing intelligent operating conditions, scrap steel production and transportation are currently quite primitive, relying entirely on manual operation in harsh working environments. Therefore, the need for research into intelligent systems and algorithms for scrap steel processing is increasingly urgent.

[0003] Currently, the level of intelligence in scrap steel operations is mainly limited by the level of intelligent sensing. Scrap steel yards typically present complex environmental conditions, such as stacks of scrap steel of different sizes and shapes, varying lighting conditions, and severe interference from metallic magnetic fields and dust. Therefore, research on intelligent sensing technology for scrap steel yards is quite challenging. Furthermore, the scrap steel operation process requires real-time and accurate detection of the surrounding environment to determine the presence of obstacles such as people and vehicles in the current work area, and to promptly send information such as the size and distance of obstacles to the intelligent decision-making system to ensure the safety and efficiency of the operation.

[0004] Due to limitations imposed by sensor characteristics and scene conditions, obstacle data is difficult to collect completely. Traditional 2D image or 3D point cloud processing methods are easily affected by similar factors, leading to a decrease in detection accuracy. Moreover, multiple processing steps, such as target detection and tracking, are usually required to obtain complete obstacle information. Furthermore, there are many types of loading trucks in scrap steel scenes, with varying sizes and dimensions. If traditional manual feature extraction and classification methods are used, it will not only be time-consuming but also difficult to accurately obtain semantic information of the target objects.

[0005] Therefore, improving the efficiency and accuracy of obstacle detection in scrap steel yards is an urgent problem to be solved. Summary of the Invention

[0006] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a method, apparatus, equipment and medium for detecting obstacles in scrap steel yards, so as to solve the problem of how to improve the efficiency and accuracy of obstacle detection in scrap steel yards in the prior art.

[0007] To achieve the above and other related objectives, this application provides a method for detecting obstacles in a scrap steel yard, the method comprising:

[0008] Acquire environmental 3D point cloud data of the target scrap steel stockpile;

[0009] The environmental three-dimensional point cloud data is preprocessed to obtain scrap steel operation environment point cloud data;

[0010] The point cloud data of the scrap steel operation environment is transmitted to a pre-trained classification model to obtain the category information and coordinate information of multiple obstacles;

[0011] By performing surface fitting on the point cloud data corresponding to each obstacle, the surface geometric information of each obstacle is obtained, thus completing the detection of obstacles in the target scrap steel yard.

[0012] In one embodiment of this application, after obtaining the surface geometry information of each obstacle, the method further includes:

[0013] The obstacle's category information, coordinate information, and surface geometry information are transmitted to a pre-configured visualization module for displaying the obstacle; and / or,

[0014] The obstacle's category information, coordinate information, and surface geometry information are transmitted to a pre-configured decision system for obstacle avoidance decision-making.

[0015] In one embodiment of this application, the environmental three-dimensional point cloud data is preprocessed to obtain scrap steel operation environment point cloud data, including:

[0016] Ground estimation is performed on the three-dimensional point cloud data of the environment to obtain a corresponding planar model of the ground;

[0017] The vertical distance from each point in the environmental 3D point cloud data to the planar model is obtained. When the vertical distance is greater than a preset distance threshold, the point is regarded as a non-ground point to obtain a non-ground point cloud.

[0018] The non-ground point cloud is filtered to obtain a filtered non-ground point cloud.

[0019] Based on the preset spatial location, the region of interest of the filtered non-ground point cloud is extracted to obtain the point cloud data of the scrap steel operation environment.

[0020] In one embodiment of this application, the point cloud data of the scrap steel operation environment is transmitted to a pre-trained classification model to obtain category information and coordinate information of multiple obstacles, including:

[0021] The scrap steel operation environment point cloud data is transmitted to a pre-trained classification model to extract features from each point cloud in the scrap steel operation environment point cloud data, thereby obtaining a global feature vector.

[0022] Semantic segmentation is performed based on the global feature vector to obtain the category information and coordinate information of the multiple obstacles.

[0023] In one embodiment of this application, feature extraction is performed on each point cloud in the scrap steel operation environment point cloud data to obtain a global feature vector, including:

[0024] The point cloud data of the scrap steel operation environment is divided into multiple levels, and multiple center points are selected as key points in each level;

[0025] Based on a preset radius, points around the key point are selected to obtain multiple local point cloud subsets corresponding to different levels;

[0026] Extract the initial local feature vectors from the local point cloud subsets corresponding to each level, and pool the initial local feature vectors to obtain the target local feature vectors corresponding to each level.

[0027] The target local feature vectors corresponding to each level are globally pooled to obtain the global feature vectors.

[0028] In one embodiment of this application, semantic segmentation is performed based on the global feature vector to obtain the category information and coordinate information of the plurality of obstacles, including:

[0029] The global feature vector is mapped to the probability distribution of a preset obstacle category using a pre-trained classifier to obtain the obstacle category label corresponding to each point.

[0030] Extract multiple point cloud clusters within the region of interest, and obtain the category information based on the number of obstacle category labels in each point cloud cluster;

[0031] The coordinate information is obtained based on the center coordinates of each point cloud cluster.

[0032] In one embodiment of this application, surface fitting is performed based on the point cloud data corresponding to each obstacle to obtain the surface geometric information of each obstacle, including:

[0033] The normal vector of the point cloud data corresponding to each obstacle is obtained by using the covariance matrix;

[0034] Based on the normal vector, surface fitting is performed on the point cloud data corresponding to each obstacle to obtain the fitted surface;

[0035] The surface geometry information is obtained based on the fitted surface.

[0036] In one embodiment of this application, a scrap steel yard obstacle detection device is also provided, the device comprising:

[0037] The data acquisition module is used to acquire the environmental three-dimensional point cloud data of the target scrap steel stockpile;

[0038] The preprocessing module is used to preprocess the three-dimensional point cloud data of the environment to obtain point cloud data of the scrap steel operation environment;

[0039] The obstacle classification module is used to transmit the point cloud data of the scrap steel operation environment to a pre-trained classification model to obtain the category information and coordinate information of multiple obstacles;

[0040] The surface fitting module is used to perform surface fitting based on the point cloud data corresponding to each obstacle to obtain the surface geometric information of each obstacle, thereby completing the detection of obstacles in the target scrap yard.

[0041] In one embodiment of this application, an electronic device is also provided, the electronic device comprising:

[0042] one or more processors;

[0043] A storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the scrap yard obstacle detection method as described above.

[0044] In one embodiment of this application, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a computer processor, causes the computer to perform the scrap steel yard obstacle detection method as described above.

[0045] The beneficial effects of this invention are:

[0046] First, 3D point cloud data of the target scrap steel yard environment is acquired. Then, the 3D point cloud data is preprocessed to obtain scrap steel operation environment point cloud data. Next, the scrap steel operation environment point cloud data is transmitted to a pre-trained classification model to obtain the category and coordinate information of multiple obstacles. Finally, surface fitting is performed based on the point cloud data corresponding to each obstacle to obtain the surface geometric information of each obstacle, thus completing the detection of obstacles within the target scrap steel yard. In this invention, the classification model is a pre-trained 3D deep learning model. Considering the complex and varied nature of scrap steel yard environments, it can adapt to different scrap steel yard scenarios and requirements, avoiding redundant development for customized scenarios. The same algorithm can identify obstacles in multiple scenarios, overcoming the limitations of traditional scrap steel yard obstacle detection methods, and possessing high flexibility, scalability, and efficiency. The deep learning-based classification model can learn point cloud data end-to-end, possessing stronger representation capabilities. Through a multi-layered neural network structure, it can learn the complex features of scrap steel yard obstacles, including shape, size, and spatial relationships, thereby improving the accuracy, efficiency, and robustness of detection. Compared to traditional obstacle detection methods, this invention can obtain the surface geometric information of obstacles in scrap steel yards. This geometric information can play a greater role in decision-making tasks, such as collision detection and path planning. By understanding the geometry of obstacles, obstacle avoidance or path planning can be performed more accurately, thereby improving the safety and efficiency of the system.

[0047] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0049] Figure 1 This is a schematic diagram illustrating the implementation environment of the obstacle detection method for scrap steel stockpile, as shown in an exemplary embodiment of this application.

[0050] Figure 2 This is a schematic flowchart illustrating an exemplary embodiment of the method for detecting obstacles in a scrap steel yard.

[0051] Figure 3 This is a block diagram illustrating an obstacle detection device for a scrap steel yard, as shown in an exemplary embodiment of this application;

[0052] Figure 4 A schematic diagram of the structure of a computer system suitable for an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0053] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0054] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0055] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0056] Figure 1 This is a schematic diagram illustrating the implementation environment of the obstacle detection method for scrap steel stockpile, as shown in an exemplary embodiment of this application.

[0057] Reference Figure 1 As shown, the implementation environment may include an intelligent decision-making system, a sensing industrial control computer, a sensing switch, a 3D laser scanner, and a positioning device (e.g., GPS, IMU). The technical solution provided in this application embodiment can be applied to the sensing industrial control computer. The 3D laser scanner and positioning device collect environmental 3D point cloud data and transmit the environmental 3D point cloud data to the sensing industrial control computer through the sensing switch. The sensing industrial control computer performs obstacle detection in the scrap steel yard based on the environmental 3D point cloud data, and then transmits the category information, coordinate information, and surface geometric information of the detected obstacles to the intelligent decision-making system for obstacle avoidance decision-making.

[0058] In one embodiment of this application, a sensing industrial control computer acquires three-dimensional point cloud data of the target scrap steel yard environment; preprocesses the three-dimensional point cloud data to obtain scrap steel operation environment point cloud data; transmits the scrap steel operation environment point cloud data to a pre-trained classification model to obtain category information and coordinate information of multiple obstacles; performs surface fitting based on the point cloud data corresponding to each obstacle to obtain the surface geometric information of each obstacle, thus completing the detection of obstacles in the target scrap steel yard. In this invention, the classification model is a pre-trained three-dimensional deep learning model. Considering the complex and varied nature of scrap steel yard environments, it can adapt to different scrap steel yard scenarios and requirements, avoiding redundant development of customized scenarios. The same algorithm can identify obstacles in multiple scenarios, overcoming the limitations of traditional methods for obstacle detection in scrap steel yards, and possessing high flexibility, scalability, and efficiency. The deep learning-based classification model can perform end-to-end learning of point cloud data, possessing stronger representation capabilities. Through a multi-layered neural network structure, it can learn the complex features of obstacles in the scrap steel yard, including shape, size, and spatial relationships, thereby improving the accuracy, efficiency, and robustness of detection. Compared to traditional obstacle detection methods, this invention can obtain the surface geometric information of obstacles in scrap steel yards. This geometric information can play a greater role in decision-making tasks, such as collision detection and path planning. By understanding the geometry of obstacles, obstacle avoidance or path planning can be performed more accurately, thereby improving the safety and efficiency of the system.

[0059] The above section introduced exemplary implementation environments for applying the technical solutions of this application. Next, we will continue to introduce the obstacle detection method for scrap steel yards of this application.

[0060] To address the problem of improving the efficiency and accuracy of obstacle detection in existing technologies, embodiments of this application propose a method for detecting obstacles in scrap steel yards, a device for detecting obstacles in scrap steel yards, an electronic device, a computer-readable storage medium, and a computer program product. These embodiments will be described in detail below.

[0061] Please see Figure 2 , Figure 2 This is a schematic flowchart illustrating an exemplary embodiment of a scrap steel yard obstacle detection method, which can be applied to... Figure 1 The implementation environment is shown. It should be understood that this method can also be applied to other exemplary implementation environments and specifically executed by devices in other implementation environments. This embodiment does not limit the implementation environment to which the method is applicable.

[0062] like Figure 2 As shown, in an exemplary embodiment, the scrap steel yard obstacle detection method includes at least steps S210 to S240, which are described in detail below:

[0063] In step S210, environmental three-dimensional point cloud data of the target scrap steel stockpile is acquired.

[0064] For example, multiple 3D laser scanners pre-installed in the target scrap yard and positioning devices corresponding to each 3D laser scanner are used to acquire environmental three-dimensional point cloud data in real time at a preset frequency. The positioning devices can be, for example, GPS (Global Positioning System) or IMU (Inertial Measurement Unit).

[0065] In step S220, the three-dimensional point cloud data of the environment is preprocessed to obtain the point cloud data of the scrap steel operation environment.

[0066] For example, data preprocessing may include: ground segmentation and ROI (region of interest) extraction.

[0067] In step S230, the point cloud data of the scrap steel operation environment is transmitted to a pre-trained classification model to obtain the category information and coordinate information of multiple obstacles.

[0068] For example, the classification model adopts a three-dimensional feature fusion network structure. By inputting the point cloud data of the scrap steel operation environment into the classification model, the category information and coordinate information of obstacles such as pedestrians and vehicles can be obtained.

[0069] In step S240, surface fitting is performed based on the point cloud data corresponding to each obstacle to obtain the surface geometric information of each obstacle, thus completing the detection of obstacles in the target scrap steel yard.

[0070] As can be seen from steps S210 to S240 above, the scheme proposed in this embodiment uses a pre-trained 3D deep learning model for classification. Considering the complex and varied environment of scrap steel yards, it can adapt to different scenarios and needs, avoiding redundant development for customized scenarios. The same algorithm can identify obstacles in multiple scenarios, overcoming the limitations of traditional methods for obstacle detection in scrap steel yards, and offering high flexibility, scalability, and efficiency. The deep learning-based classification model can learn point cloud data end-to-end, possessing stronger representation capabilities. Through a multi-layered neural network structure, it can learn the complex features of obstacles in scrap steel yards, including shape, size, and spatial relationships, thereby improving the accuracy, efficiency, and robustness of detection. Compared to traditional obstacle detection methods, this invention can obtain the surface geometric information of obstacles in scrap steel yards. This geometric information can play a greater role in decision-making tasks, such as collision detection and path planning. By understanding the geometry of obstacles, obstacle avoidance or path planning can be performed more accurately, improving the system's safety and efficiency.

[0071] In one embodiment of this application, after obtaining the surface geometry information of each obstacle, the method further includes:

[0072] The obstacle's category information, coordinate information, and surface geometry information are transmitted to a pre-configured visualization module for displaying the obstacle; and / or,

[0073] The obstacle's category information, coordinate information, and surface geometry information are transmitted to a pre-configured decision system for obstacle avoidance decision-making.

[0074] For example, a 3D point cloud visualization module is used to display the category, coordinates, and surface geometry of obstacles in a visual format, showing the obstacle category, coordinates, and surface shape in real time to facilitate better understanding of the data and results. Simultaneously, the category, coordinates, and surface geometry of each obstacle are transmitted to the intelligent decision-making system to provide data support for subsequent obstacle avoidance decisions.

[0075] In one embodiment of this application, the environmental three-dimensional point cloud data is preprocessed to obtain scrap steel operation environment point cloud data, including:

[0076] Ground estimation is performed on the three-dimensional point cloud data of the environment to obtain a corresponding planar model of the ground;

[0077] The vertical distance from each point in the environmental 3D point cloud data to the planar model is obtained. When the vertical distance is greater than a preset distance threshold, the point is regarded as a non-ground point to obtain a non-ground point cloud.

[0078] The non-ground point cloud is filtered to obtain a filtered non-ground point cloud.

[0079] Based on the preset spatial location, the region of interest of the filtered non-ground point cloud is extracted to obtain the point cloud data of the scrap steel operation environment.

[0080] For example, the Singular Value Decomposition (SVD) fitting method is used to estimate the ground surface of the 3D point cloud data of the environment, fitting a planar model corresponding to the ground. By calculating the vertical distance from points to the fitted ground plane, the ground point cloud is distinguished from non-ground point clouds. Based on a preset threshold, points with a distance less than the threshold are considered ground points and removed from the original point cloud. Voxel filtering is performed on the remaining non-ground point cloud to remove noise points or outliers, improving the quality and accuracy of the point cloud data. Then, based on spatial location, the region of interest (ROI) is extracted to obtain the point cloud data of the scrap metal operation environment.

[0081] In one embodiment of this application, the point cloud data of the scrap steel operation environment is transmitted to a pre-trained classification model to obtain category information and coordinate information of multiple obstacles, including:

[0082] The scrap steel operation environment point cloud data is transmitted to a pre-trained classification model to extract features from each point cloud in the scrap steel operation environment point cloud data, thereby obtaining a global feature vector.

[0083] Semantic segmentation is performed based on the global feature vector to obtain the category information and coordinate information of the multiple obstacles.

[0084] In one embodiment of this application, feature extraction is performed on each point cloud in the scrap steel operation environment point cloud data to obtain a global feature vector, including:

[0085] The point cloud data of the scrap steel operation environment is divided into multiple levels, and multiple center points are selected as key points in each level;

[0086] Based on a preset radius, points around the key point are selected to obtain multiple local point cloud subsets corresponding to different levels;

[0087] Extract the initial local feature vectors from the local point cloud subsets corresponding to each level, and pool the initial local feature vectors to obtain the target local feature vectors corresponding to each level.

[0088] The target local feature vectors corresponding to each level are globally pooled to obtain the global feature vectors.

[0089] For example, the preprocessed point cloud data is divided into multiple layers, each corresponding to a different point cloud density. This hierarchical structure allows the network to process point cloud data at different layers, thereby effectively handling point clouds of different scales and complexities.

[0090] At each level, the Set Abstraction operation is used to extract local features from the point cloud. SetAbstraction consists of two sub-operations:

[0091] Sampling: At each level, the farthest point sampling method (FPS) is used to sample the input points and select several center points as key points, so that the original point cloud is reduced from a high-density point cloud to a low-density set of key points.

[0092] Grouping: Using the Ball query grouping method, points within a certain radius are taken around the key points obtained in the previous sampling operation to form local region groups, resulting in multiple local point cloud subsets corresponding to different levels.

[0093] Through the sampling and grouping operations in the previous step, local point cloud subsets corresponding to multiple levels were obtained. Next, a PointNet multi-layer neural network was used in each local region to extract local features. After inputting the point cloud of the local region into the PointNet model, an MLP perceptron was used to learn and extract the initial local feature vectors from the local point cloud subsets corresponding to each level. Then, a max pooling layer was used to pool the initial local feature vectors to form a more global feature representation (i.e., the target local feature vector).

[0094] For each level, the Set Abstraction operation is performed again to extract features from higher levels. Repeated sampling and grouping operations generate higher-level local point clouds and features. At the highest level, a global pooling operation is performed to pool the features of the entire point cloud, obtaining a global feature vector.

[0095] In one embodiment of this application, semantic segmentation is performed based on the global feature vector to obtain the category information and coordinate information of the plurality of obstacles, including:

[0096] The global feature vector is mapped to the probability distribution of a preset obstacle category using a pre-trained classifier to obtain the obstacle category label corresponding to each point.

[0097] Extract multiple point cloud clusters within the region of interest, and obtain the category information based on the number of obstacle category labels in each point cloud cluster;

[0098] The coordinate information is obtained based on the center coordinates of each point cloud cluster.

[0099] For example, using a global feature vector as input, a trained classifier maps the global features to the probability distribution of corresponding categories, predicting the obstacle category to which each point belongs, including pedestrians, dump trucks, fixed obstacles (walls, pillars, cone-shaped traffic barriers, guardrails), etc. The classifier's output provides a category label for each point, thus assigning each point in the point cloud to the corresponding obstacle category. Using the K-Means clustering method, all point cloud clusters within the ROI are extracted, and the obstacle category label for each point cloud cluster is statistically analyzed. The category with the highest percentage is used as the semantic label for the point cloud cluster, thereby extracting the point cloud data and center coordinates of each obstacle.

[0100] In one embodiment of this application, surface fitting is performed based on the point cloud data corresponding to each obstacle to obtain the surface geometric information of each obstacle, including:

[0101] The normal vector of the point cloud data corresponding to each obstacle is obtained by using the covariance matrix;

[0102] Based on the normal vector, surface fitting is performed on the point cloud data corresponding to each obstacle to obtain the fitted surface;

[0103] The surface geometry information is obtained based on the fitted surface.

[0104] For example, surface features are extracted from detected obstacles to describe their geometric properties such as shape and curvature. First, a method based on the covariance matrix is ​​used to calculate the normal vector of the point cloud data with obstacle labels output in the previous step. Using the normal vector, a least-squares fitting method is used to fit the surface of the obstacle's point cloud data, minimizing the distance between the points in the point cloud and the fitted surface model, thus obtaining the fitted surface for each obstacle. Based on the fitted surface, the surface geometric information is obtained.

[0105] Figure 3 This is a block diagram illustrating an obstacle detection device for a scrap metal yard, as shown in an exemplary embodiment of this application. The device can be applied to... Figure 1 The implementation environment shown is not limited to this embodiment. This device can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applicable.

[0106] like Figure 3 As shown, this exemplary scrap yard obstacle detection device includes:

[0107] Data acquisition module 301 is used to acquire environmental three-dimensional point cloud data of the target scrap steel stockpile;

[0108] Preprocessing module 302 is used to preprocess the environmental three-dimensional point cloud data to obtain scrap steel operation environment point cloud data;

[0109] The obstacle classification module 303 is used to transmit the point cloud data of the scrap steel operation environment to a pre-trained classification model to obtain the category information and coordinate information of multiple obstacles;

[0110] The surface fitting module 304 is used to perform surface fitting based on the point cloud data corresponding to each obstacle to obtain the surface geometric information of each obstacle and complete the detection of obstacles in the target scrap yard.

[0111] In this exemplary scrap steel yard obstacle detection device, the classification model is a pre-trained 3D deep learning model. Addressing the complex and ever-changing environment of scrap steel yards, it can adapt to different scenarios and needs, avoiding redundant development for customized scenarios. The same algorithm can identify obstacles in multiple scenarios, overcoming the limitations of traditional scrap steel yard obstacle detection methods and offering high flexibility, scalability, and efficiency. The deep learning-based classification model can learn point cloud data end-to-end, possessing stronger representational capabilities. Through a multi-layered neural network structure, it can learn the complex features of obstacles in scrap steel yards, including shape, size, and spatial relationships, thereby improving detection accuracy, efficiency, and robustness. Compared to traditional obstacle detection methods, this invention can obtain the surface geometric information of obstacles within the scrap steel yard. This geometric information can play a greater role in decision-making tasks, such as collision detection and path planning. By understanding the geometry of obstacles, obstacle avoidance or path planning can be performed more accurately, improving system safety and efficiency.

[0112] It should be noted that the scrap steel yard obstacle detection device and the scrap steel yard obstacle detection method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the scrap steel yard obstacle detection device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0113] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the scrap steel yard obstacle detection method provided in the above embodiments.

[0114] Figure 4 A schematic diagram of a computer system suitable for an electronic device according to an embodiment of this application is shown. It should be noted that... Figure 4 The computer system 400 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0115] like Figure 4 As shown, the computer system 400 includes a Central Processing Unit (CPU) 401, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 402 or programs loaded from Storage Unit 408 into Random Access Memory (RAM) 403, such as performing the methods described in the above embodiments. The RAM 403 also stores various programs and data required for system operation. The CPU 401, ROM 402, and RAM 403 are interconnected via a bus 404. An Input / Output (I / O) interface 405 is also connected to the bus 404.

[0116] The following components are connected to I / O interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to I / O interface 405 as needed. A removable medium 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 410 as needed so that computer programs read from it can be installed into storage section 408 as needed.

[0117] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by central processing unit (CPU) 401, it performs various functions defined in the system of this application.

[0118] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0119] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0120] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0121] Another aspect of this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer's processor, causes the computer to perform the scrap yard obstacle detection method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into that electronic device.

[0122] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the scrap yard obstacle detection method provided in the various embodiments described above.

[0123] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for detecting obstacles in a scrap steel yard, characterized in that, The method includes: Acquire environmental 3D point cloud data of the target scrap steel stockpile; The environmental three-dimensional point cloud data is preprocessed to obtain scrap steel operation environment point cloud data; The point cloud data of the scrap steel operation environment is transmitted to a pre-trained classification model to obtain the category information and coordinate information of multiple obstacles; Based on the point cloud data corresponding to each obstacle, surface fitting is performed to obtain the surface geometric information of each obstacle, thereby completing the detection of obstacles in the target scrap steel yard. The point cloud data of the scrap steel operation environment is transmitted to a pre-trained classification model to obtain the category information and coordinate information of multiple obstacles, including: The scrap steel operation environment point cloud data is transmitted to a pre-trained classification model to extract features from each point cloud in the scrap steel operation environment point cloud data, thereby obtaining a global feature vector. Semantic segmentation is performed based on the global feature vector to obtain the category information and coordinate information of the multiple obstacles; Feature extraction is performed on each point cloud in the scrap steel operation environment point cloud data to obtain a global feature vector, including: The point cloud data of the scrap steel operation environment is divided into multiple levels, and multiple center points are selected as key points in each level; Based on a preset radius, points around the key point are selected to obtain multiple local point cloud subsets corresponding to different levels; Extract the initial local feature vectors from the local point cloud subsets corresponding to each level, and pool the initial local feature vectors to obtain the target local feature vectors corresponding to each level. The target local feature vectors corresponding to each level are globally pooled to obtain the global feature vectors; Surface fitting is performed on the point cloud data corresponding to each obstacle to obtain the surface geometric information of each obstacle, including: The normal vector of the point cloud data corresponding to each obstacle is obtained by using the covariance matrix; Based on the normal vector, surface fitting is performed on the point cloud data corresponding to each obstacle to obtain the fitted surface; The surface geometry information is obtained based on the fitted surface.

2. The method for detecting obstacles in a scrap steel yard according to claim 1, characterized in that, After obtaining the surface geometry information of each obstacle, the process further includes: The obstacle's category information, coordinate information, and surface geometry information are transmitted to a pre-configured visualization module for displaying the obstacle; and / or, The obstacle's category information, coordinate information, and surface geometry information are transmitted to a pre-configured decision system for obstacle avoidance decision-making.

3. The method for detecting obstacles in a scrap steel yard according to claim 1, characterized in that, The environmental 3D point cloud data is preprocessed to obtain scrap steel operation environment point cloud data, including: Ground estimation is performed on the three-dimensional point cloud data of the environment to obtain a corresponding planar model of the ground; The vertical distance from each point in the environmental 3D point cloud data to the planar model is obtained. When the vertical distance is greater than a preset distance threshold, the point is regarded as a non-ground point to obtain a non-ground point cloud. The non-ground point cloud is filtered to obtain a filtered non-ground point cloud. Based on the preset spatial location, the region of interest of the filtered non-ground point cloud is extracted to obtain the point cloud data of the scrap steel operation environment.

4. The method for detecting obstacles in a scrap steel yard according to claim 3, characterized in that, Semantic segmentation is performed based on the global feature vector to obtain the category information and coordinate information of the multiple obstacles, including: The global feature vector is mapped to the probability distribution of a preset obstacle category using a pre-trained classifier to obtain the obstacle category label corresponding to each point. Extract multiple point cloud clusters within the region of interest, and obtain the category information based on the number of obstacle category labels in each point cloud cluster; The coordinate information is obtained based on the center coordinates of each point cloud cluster.

5. A scrap steel yard obstacle detection device based on the scrap steel yard obstacle detection method according to any one of claims 1-4, characterized in that, The device includes: The data acquisition module is used to acquire the environmental three-dimensional point cloud data of the target scrap steel stockpile; The preprocessing module is used to preprocess the three-dimensional point cloud data of the environment to obtain point cloud data of the scrap steel operation environment; The obstacle classification module is used to transmit the point cloud data of the scrap steel operation environment to a pre-trained classification model to obtain the category information and coordinate information of multiple obstacles; The surface fitting module is used to perform surface fitting based on the point cloud data corresponding to each obstacle to obtain the surface geometric information of each obstacle, thereby completing the detection of obstacles in the target scrap yard.

6. An electronic device, characterized in that, The electronic device includes: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the scrap yard obstacle detection method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the computer's processor, causes the computer to perform the scrap yard obstacle detection method as described in any one of claims 1 to 4.

Images