End-to-end scatterer contour modeling method based on online difficult case mining

Abstract

The invention provides an end-to-end scatterer contour modeling method based on online difficult case mining. The end-to-end scatterer contour modeling method comprises the steps of obtaining to-be-modeled point cloud group data and a pre-trained contour modeling network; wherein the contour modeling network comprises a backbone network, a point cloud group detection network, a vertex detection network and an edge detection network, and the point cloud group detection network is a detection network integrated with an online difficult case mining module; inputting the to-be-modeled point cloud group data into the backbone network to obtain point cloud feature data; inputting the point cloud feature data into a point cloud group detection network to obtain a point cloud group containing vertexes; performing vertex detection and edge detection on the point cloud group containing the vertexes based on the vertex detection network and the edge detection network to generate a wireframe model; and generating a watertight plane based on the wireframe model to obtain a scatterer contour model. The method not only improves the recognition precision of the positive sample, but also enhances the robustness of the negative sample, and effectively solves the performance problem caused by the imbalance of the positive and negative samples in the traditional point cloud group detection.

Description

Technical Field

[0001] The present invention relates to the technical field of contour modeling, and in particular to an end-to-end scatterer contour modeling method based on online hard example mining. Background Art

[0002] 3D reconstruction refers to the process of reconstructing the 3D information of an object from single-view images, multi-view images, or radar point clouds. It is an important research direction in computer vision and graphics, and plays an important role in many fields such as urban planning, emergency rescue, channel modeling, virtual reality, and medical imaging.

[0003] In recent years, with the development of laser scanning technology, the acquisition of laser radar data has become more convenient, making light detection and ranging (LiDAR) the main means of obtaining three-dimensional information of large-scale urban scenes. The point cloud obtained by LiDAR is usually dense and relatively regular, so it is very suitable for building reconstruction. Traditional contour modeling methods rely on geometric primitive extraction, which is difficult to handle data missing, and the steps involved, such as normal estimation and plane extraction, may lead to cumulative errors. Some methods obtain implicit expressions through neural networks to reconstruct small triangular mesh models of objects. Due to the storage of a large amount of vertex and patch information, the reconstruction results obtained by this method often occupy a large storage space. In contrast, wireframes focus on edge structures and are very suitable for describing buildings with a small number of obvious crease edges and smoothness. Other methods try to recover object wireframe models from point clouds, but point-by-point detection in large-scale point cloud data is very time-consuming. PC2WF first proposed to quickly screen point cloud groups containing vertices through point cloud group detection, which can be regarded as positive samples, while the remaining point cloud groups that do not contain key vertices are regarded as negative samples. PC2WF uses a balanced positive and negative sample strategy when training the model. However, in actual application scenarios, the number of negative samples that do not contain key vertices usually far exceeds the number of positive samples that contain vertices. This imbalance in sample distribution leads to the insufficient generalization ability of the trained network for negative samples in actual applications. Summary of the invention

[0004] The present invention provides an end-to-end scatterer contour modeling method based on online hard example mining, which is used to solve the problem of the limitation of point cloud group detection performance caused by unbalanced positive and negative samples in the existing 3D reconstruction method, improves the recognition accuracy of positive samples, i.e. roof point clouds, and enhances the robustness to negative samples such as background point clouds and noise. The technical solutions proposed by the present invention are as follows: In a first aspect, the present invention provides an end-to-end scatterer profile modeling method based on online hard example mining, comprising: Obtaining point cloud group data to be modeled and a pre-trained contour modeling network; wherein the contour modeling network includes a backbone network, a point cloud group detection network, a vertex detection network and an edge detection network, and the point cloud group detection network is a detection network integrated with an online difficult example mining module; Inputting the point cloud group data to be modeled into the backbone network to obtain point cloud feature data; Inputting the point cloud feature data into the point cloud group detection network to obtain a point cloud group including vertices; Performing vertex detection and edge detection on the point cloud group containing vertices based on the vertex detection network and the edge detection network to generate a wireframe model; A watertight plane is generated based on the wireframe model to obtain a scatterer contour model.

[0005] Optionally, the contour modeling network is trained in the following manner: Input the point cloud group samples into the pre-established contour modeling network, and calculate the first loss, the second loss and the third loss respectively; wherein the first loss is the loss of the point cloud group detection network, the second loss is the loss of the vertex detection network, and the third loss is the loss of the edge detection network; Calculate the overall loss based on the first loss, the second loss and the third loss; The network parameters of the point cloud group detection network, the vertex detection network and the edge detection network are updated according to the overall loss to obtain the pre-trained contour modeling network.

[0006] Optionally, the point cloud group detection network includes a read-only network and a read-write network; The first loss is determined in the following manner: Inputting the point cloud group sample into the read-only network to obtain a first prediction result; wherein the first prediction result is the probability that the point cloud group sample is a positive sample; Calculate sample loss based on the true label of the point cloud group sample and the first prediction result; According to the sample loss of all samples and a preset hard example ratio, a plurality of hard example point cloud group samples are obtained by screening from all samples; Inputting the difficult point cloud group sample into the read-write network to obtain a second prediction result; wherein the second prediction result is the probability that the difficult point cloud group sample is a positive sample; The first loss is calculated according to the true labels of all point cloud group samples and the second prediction results.

[0007] Optionally, the second loss is determined by: Inputting vertex samples into the vertex detection network to obtain vertex detection results, wherein the vertex detection results include vertex predicted positions and vertex predicted numbers; The actual vertex positions of the vertex samples are obtained, and the second loss is calculated according to the predicted vertex positions of all vertex samples, the actual vertex positions, and the predicted number of vertices.

[0008] Optionally, the third loss is determined in the following manner: Each edge sample is input into the edge detection network to obtain the corresponding edge detection results; wherein the edge detection result is the probability that the edge sample is a positive sample; The third loss is calculated according to the true labels of all edge samples and the edge detection result.

[0009] Optionally, the method further comprises: Obtain the number of correctly predicted vertices, the number of incorrectly predicted vertices, and the number of vertices incorrectly predicted as counterexamples; wherein, for a predicted vertex, if the distance between the vertex and the vertex closest to it in the benchmark true value is less than a predefined threshold, the vertex is considered to be a correctly predicted vertex; Calculating vertex detection accuracy according to the number of correctly predicted vertices and the number of incorrectly predicted vertices; Calculating a vertex detection recall rate according to the number of correctly predicted vertices and the number of vertices incorrectly predicted as counterexamples; The vertex detection accuracy and vertex detection recall are used to evaluate the vertex detection effect of the vertex detection network.

[0010] In a second aspect, the present invention further provides an end-to-end scatterer profile modeling device based on online hard example mining, comprising the following modules: A data acquisition module, used to acquire the point cloud group data to be modeled and a pre-trained contour modeling network; wherein the contour modeling network includes a backbone network, a point cloud group detection network, a vertex detection network and an edge detection network, and the point cloud group detection network is a detection network integrated with an online difficult example mining module; A feature extraction module, used for inputting the point cloud group data to be modeled into the backbone network to obtain point cloud feature data; A first detection module, used for inputting the point cloud feature data into the point cloud group detection network to obtain a point cloud group containing vertices; A second detection module, configured to perform vertex detection and edge detection on the point cloud group containing vertices based on the vertex detection network and the edge detection network to generate a wireframe model; The contour modeling module is used to generate a watertight plane based on the wireframe model to obtain a scatterer contour model.

[0011] In a third aspect, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the end-to-end scatterer profile modeling method based on online difficult example mining as described in the first aspect above is implemented.

[0012] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the end-to-end scatterer profile modeling method based on online difficult example mining as described in the first aspect above.

[0013] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the end-to-end scatterer profile modeling method based on online difficult example mining as described in the first aspect above.

[0014] Based on the above technical solution, the beneficial effects of the present invention compared with the prior art are as follows: The end-to-end scatterer contour modeling method based on online hard example mining provided by the present invention processes the input point cloud data by using a point cloud group detection network integrated with an online hard example mining module. The online hard example mining module can dynamically identify and focus on those samples that are difficult to identify, that is, hard examples, that is, roof point clouds with complex shapes, occlusion or low contrast. Under the guidance of the online hard example mining module, the point cloud group detection network can more effectively learn the key features of the building contour. This process not only improves the recognition accuracy of positive samples (i.e., roof point clouds), but also enhances its robustness to negative samples (such as background point clouds, noise, etc.), effectively solving the performance problems caused by the imbalance of positive and negative samples in traditional point cloud group detection. The point cloud group containing key vertices is further processed using the trained vertex detection and edge prediction network. The two networks work together to accurately identify the vertex position and edge information of the roof, thereby generating a lightweight roof wireframe model. This step makes full use of the key features extracted by the point cloud group detection network and further simplifies the representation of the roof contour. After obtaining the wireframe model of the roof, a watertight plane is generated based on the wireframe model. This plane not only covers all vertices, edges, and segments in the wireframe model, but also ensures the continuity and non-porousness of the plane, and finally obtains an accurate and complete scatterer contour model.

[0015] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0016] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of the end-to-end scatterer profile modeling method based on online hard example mining provided by the present invention.

[0019] Figure 2 It is the overall network architecture diagram of the contour modeling network provided by the present invention.

[0020] Figure 3 This is an example diagram of the SimulateRoof_Wireframe data set provided by the present invention.

[0021] Figure 4 It is a structural schematic diagram of the point cloud group detection network provided by the present invention.

[0022] Figure 5 It is a schematic diagram of the visualization results of the ablation experiment provided by the present invention.

[0023] Figure 6 It is a schematic diagram of qualitative evaluation results on the SimulateRoof_Wireframe dataset provided by the present invention.

[0024] Figure 7 It is a structural schematic diagram of an end-to-end scatterer profile modeling device based on online hard example mining provided by the present invention.

[0025] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] There are two main approaches to recovering 3D building models from airborne LiDAR data: model-driven and data-driven. The model-driven approach reconstructs the model by matching the geometry of the roof, which ensures that the topology of the reconstructed model is correct. Specifically, it relies on a predefined model library consisting of basic building shapes, such as flat roofs, triangular roofs, asymmetric gable roofs, double-pitched roofs, pyramidal roofs, and mansard roofs. Reconstruction is performed by selecting the model from the library that best matches the point cloud data. However, this approach may encounter difficulties if certain roof shapes are missing from the model library or if the roof structure is too complex. In addition, due to the limited number of shapes in the model library, this limits production accuracy and may not accurately model complex roof structures.

[0028] Data-driven methods usually need to be performed in two steps: geometric primitive extraction and building structure reasoning. This method starts with extracting basic geometric shapes, such as planes, cylinders, cones, spheres, or tori, and then analyzes the topological relationship of these primitives in two-dimensional or three-dimensional space. In urban scenes, planes are a major geometric structure, so a variety of plane extraction algorithms for point clouds have been developed, such as the RANdom SAmple Consensus (RANSAC) algorithm and the region growing algorithm. Nan and Wonka proposed a data-driven method called Polyfit in 2017, which combines the RANSAC algorithm with contextual knowledge to reconstruct lightweight polygonal surfaces from point clouds. However, this reconstruction process relies on the extraction of geometric primitives, and the extraction of some geometric primitives relies on the normal information of the point cloud, which is sometimes difficult to obtain, and the normal estimation process may introduce errors. Obviously, these traditional methods are not end-to-end and cannot avoid the accumulated errors generated in different stages, resulting in the final 3D roof model may not be optimal.

[0029] With the rapid development of deep learning in the field of point cloud processing, several deep learning-based 3D reconstruction methods have emerged. These methods can construct triangular meshes from point clouds and better recover irregularly shaped surfaces by learning implicit representations of surfaces. However, the models recovered by these methods are often large. For buildings with regular shapes and large point cloud sizes, lightweight 3D wireframe models are more suitable. Lightweight 3D wireframe models provide a comprehensive representation of the shape and structural information of 3D objects, and the models are more concise and convenient for subsequent processing and analysis. Some methods attempt to reconstruct wireframe models from point clouds, such as Point2Roof, which determines vertices point by point from all input point clouds, and PC2WF, which innovatively introduces point cloud group detection to quickly screen point cloud group data containing vertices, but the imbalance of positive and negative samples limits the performance of point cloud group detection.

[0030] In order to solve the problem of missing data sets for wireframe reconstruction of buildings and the limitation of unbalanced positive and negative samples on point cloud group detection performance in existing 3D reconstruction methods, the present invention constructs a large-scale synthetic data set for training and testing of building wireframe reconstruction. At the same time, the present invention proposes an innovative end-to-end building outline modeling method. In the training process of this method, all losses can be back-propagated, which effectively avoids the generation of cumulative errors. In addition, by introducing a point cloud group detection network, dense point cloud data can be processed quickly, and combined with an online difficult example mining module, the network can fully learn difficult point cloud group samples to improve detection efficiency.

[0031] Combine the following Figure 1-Figure 6 The end-to-end scatterer profile modeling method based on online hard example mining of the present invention is described.

[0032] Reference Figure 1 As shown, the method includes the following: Step S101, obtaining point cloud group data to be modeled and a pre-trained contour modeling network.

[0033] The above-mentioned point cloud group data to be modeled is usually obtained through laser radar (LiDAR), drone photography or other 3D scanning equipment, and includes three-dimensional point cloud information of the roof of the building.

[0034] The above pre-trained contour modeling network is a deep learning model. Figure 2 As shown in the figure, it includes a backbone network (FCFG), a point cloud group detection network, a vertex detection network, and an edge detection network. These networks have been trained with a large amount of data and have the ability to extract roof contours from point clouds. The point cloud group detection network integrates an Online Hard Example Mining (OHEM) module to improve the detection capability of complex roof contours. The Online Hard Example Mining module can dynamically select difficult-to-identify samples for additional training during the training process, thereby improving detection accuracy. The introduction of the Online Hard Example Mining module enables the model to perform targeted training on difficult-to-identify samples and improve overall detection performance.

[0035] The core of the contour modeling network is the Online Hard Example Mining -Point Cloud Detection (OHEM-PCD) module, also known as the point cloud group detection network mentioned above, which is responsible for detecting point cloud groups containing key vertices. The point cloud group detection network performs classification tasks through a series of efficient convolution and normalization operations, which are designed to identify point cloud groups containing vertices. In order to further improve the detection accuracy, the point cloud group detection network will filter out samples with large losses after each detection and perform additional forward and backward propagation on these samples. This strategy enables the network to focus more on point cloud groups that are difficult to identify, thereby improving the overall detection performance.

[0036] After preliminary screening by the point cloud group detection network, only the point cloud groups identified as containing vertices are passed to subsequent processing modules. These modules include the vertex detection network and the edge detection network, which are designed as lightweight convolutional networks specifically for recovering the wireframe model of the roof from the screened point cloud groups.

[0037] In terms of vertex detection, the input of the vertex detection network is the point cloud group features and point cloud coordinates, and the output is the vertex coordinates. The vertex detection network is responsible for accurately locating the vertices of the roof, providing a basis for subsequent edge detection and model reconstruction. The edge detection network is used to determine the edge of the roof, with the input being the edge features and the output being the probability that the edge is a valid edge. These edges define the outline of the roof. Through the collaborative work of these two networks, a three-dimensional wireframe model of the roof can be reconstructed. Finally, in order to generate a complete three-dimensional model of the roof, the present invention also includes a step of converting the wireframe model into a watertight plane. This step ensures the continuity and integrity of the roof model, making the final roof model suitable for various subsequent applications, such as urban planning, architectural analysis, and virtual reality.

[0038] Step S102: input the point cloud group data to be modeled into the backbone network to obtain point cloud feature data.

[0039] The backbone network is responsible for extracting rich feature information from the input point cloud data. This feature information is crucial for subsequent point cloud group detection, vertex detection, and edge detection. The backbone network converts the original point cloud data into a high-dimensional feature vector through a series of convolution, pooling and other operations. The extracted feature information can more accurately reflect the spatial structure and texture information of the point cloud data, providing strong support for subsequent detection tasks.

[0040] The present invention uses a fully convolutional geometric features (FCGF) network as the backbone network, and its core function is to extract rich feature information from the point cloud. The FCGF network processes the point cloud data and outputs the feature representation of each point in the point cloud. These feature representations are compact and capture a wide range of spatial context information, so they can be used for subsequent point cloud group detection, classification and other tasks. For each point in the input point cloud data, the FCGF network outputs a 32-dimensional feature vector. These feature vectors contain rich information such as the position, shape, texture, etc. of the point in space.

[0041] Step S103: input the point cloud feature data into the point cloud group detection network to obtain a point cloud group containing vertices.

[0042] The point cloud group detection network uses the feature information extracted from the backbone network to group the point cloud data and identify the point cloud groups containing vertices. Through a series of efficient convolution and normalization operations, as well as the assistance of the OHEM module, the network can accurately identify the point cloud groups containing vertices. This reduces the complexity of subsequent processing and improves the efficiency of the overall algorithm.

[0043] Step S104: performing vertex detection and edge detection on the point cloud group containing vertices based on the vertex detection network and the edge detection network to generate a wireframe model.

[0044] Taking roof outline modeling as an example, the vertex detection network is responsible for identifying specific roof vertices from the point cloud group containing roof vertices. These vertices represent key locations of the roof, such as the ridge and eaves. The edge detection network is responsible for detecting the edges of the roof outline, that is, the line segments connecting the roof vertices. Through the collaborative work of these two networks, a preliminary roof wireframe model can be generated.

[0045] Step S105: Generate a watertight plane based on the wireframe model to obtain a scatterer contour model.

[0046] After obtaining the wireframe model, a series of geometric processing operations (such as interpolation and smoothing) are used to convert the wireframe model into a watertight plane. A watertight plane is a continuous plane without holes that can more accurately reflect the true shape of the roof. Finally, the watertight plane is output as the roof outline model, and the generated roof outline model is more accurate, continuous and smooth.

[0047] The end-to-end scatterer contour modeling method based on online difficult example mining provided by the present invention uses a point cloud group detection network integrated with an online difficult example mining module to process the input point cloud data. The online difficult example mining module can dynamically identify and focus on those samples that are difficult to classify, i.e., difficult examples. In the building contour modeling, these difficult examples correspond to roof point clouds with complex shapes, occlusions or low contrast, which are often the key to the problem of imbalance between positive and negative samples. Guided by the online difficult example mining module, the point cloud group detection network can more effectively learn the key features of the building contour, even if these features are unevenly distributed in positive and negative samples. This process not only improves the recognition accuracy of the model for positive samples (i.e., roof point clouds), but also enhances its robustness to negative samples (such as background point clouds, noise, etc.). Next, the point cloud group containing key vertices is further processed using the trained vertex detection and edge prediction network. Taking the roof contour modeling as an example, the two networks work together to accurately identify the vertex position and edge information of the roof, thereby generating a lightweight roof wireframe model. This step makes full use of the key features extracted by the point cloud group detection network and further simplifies the representation of the roof outline. After obtaining the roof wireframe model, a watertight plane is generated based on the wireframe model through further geometric processing and optimization algorithms. This plane not only covers all vertices, edges, and segments in the wireframe model, but also ensures the continuity and non-porousness of the plane. Through this step, an accurate and complete roof outline model is finally obtained.

[0048] Throughout the process, the application of the online hard example mining module enables the model to focus more on those hard example point cloud group samples that are critical to performance improvement. At the same time, through the collaborative work of vertex detection and edge prediction networks, as well as subsequent geometric processing and optimization steps, the accuracy and robustness of the roof profile model are further improved, solving the problem of the limitation of positive and negative sample imbalance on point cloud group detection performance.

[0049] The present invention was tested on a synthetic dataset simulateRoof_Wireframe to verify its performance and accuracy. Wichmann et al. created a publicly available 3D building reconstruction dataset called RoofN3D in 2018 and 2019, which covers three basic roof types: pyramidal, hip, and cable roofs. Subsequently, Li et al. generated a larger roof dataset SimulateRoof based on the RoofN3D expansion in 2022, increasing the number of roof types to 16. However, the SimulateRoof dataset only provides sparse point clouds and mesh models, which fails to fully meet the needs of roof wireframe reconstruction.

[0050] In order to solve this problem and better restore the wireframe model of the building, the present invention further developed a new synthetic dataset SimulateRoof_Wireframe based on SimulateRoof. This dataset contains 17,600 roof models, which are divided into training sets and test sets in a ratio of 10:1 to ensure the adequacy of model training and the accuracy of testing. Each roof model includes three forms: point cloud, wireframe model and mesh model, providing comprehensive reference data for roof wireframe reconstruction. Through testing on the SimulateRoof_Wireframe dataset, the present invention demonstrated its excellent performance and high accuracy in the task of roof wireframe reconstruction. Figure 3 Shown is an example of the SimulateRoof_Wireframe dataset.

[0051] In the wireframe reconstruction process of a 3D building model, the first step is to identify the position of the vertices from the huge point cloud data. However, the point cloud data of buildings is often huge in scale and mainly consists of regular planar structures. Compared with the point cloud data of indoor objects, direct vertex detection will greatly increase the computational burden and lead to low efficiency. In order to solve this problem, the present invention refers to the PatchNet network proposed by PC2WF to pre-filter out a large number of irrelevant or noisy point clouds, thereby reducing the computational pressure of subsequent processing.

[0052] For architectural point clouds, most point cloud groups do not contain vertices. Using balanced positive and negative samples to train the point cloud group detection network will result in insufficient generalization of the model for point cloud groups that do not contain vertices in practical applications. At the same time, since the training data may also contain a large number of simple samples, that is, samples that the model can easily classify correctly, the model spends most of the time processing these simple samples during training and cannot fully learn the characteristics of difficult point cloud group samples, thereby affecting the generalization ability of the model. To this end, the present invention designs a point cloud group detection network to prevent the model from being overly biased towards the majority category or simple samples.

[0053] The online hard example mining module is responsible for detecting point cloud groups containing key vertices. The point cloud group detection network performs the classification task through a series of efficient convolution and normalization operations, which are designed to identify point cloud groups containing vertices. To further improve the detection accuracy, the point cloud group detection network will filter out samples with large losses after each detection and perform additional forward and backward propagation on these samples. This strategy allows the network to focus more on point cloud groups that are difficult to identify, thereby improving the overall detection performance.

[0054] In an optional embodiment, Figure 4The details of the point cloud group detection network are shown. The above point cloud group detection network includes a read-only network and a read-write network. During the training process, the loss of the point cloud group detection network is determined as follows: S201, inputting the point cloud group sample into the read-only network to obtain a first prediction result; First, the pre-prepared point cloud group samples are input into the read-only network. The main function of the read-only network is to make a preliminary prediction and determine the probability that the point cloud group sample is a positive sample (i.e., the point cloud containing key roof information). The read-only network outputs the first prediction result, i.e., the probability value of each point cloud group sample being a positive sample.

[0055] Specifically, the above read-only network includes convolutional layer 1 (Convolutional Layer 1, Conv1), convolutional layer 2 (Convolutional Layer 2, Conv2), maximum pooling layer 2D (Max Pooling Layer 2D, MaxPool2D) and convolutional layer 3 (Convolutional Layer 3, Conv3).

[0056] The processing process of the read-only network is as follows: the point cloud group samples are sent to the Conv1 layer of the read-only network as input data. The Conv1 layer performs a convolution operation on the input point cloud data to extract a preliminary feature map. After the convolution operation, the feature map is normalized by the BatchNorm layer to accelerate the training process and improve the stability of the model. Then, the ReLU activation function is applied to the feature map to increase the nonlinearity of the network. The Conv2 layer receives the feature map output by the Conv1 layer and performs further convolution operations to extract higher-level features. Similarly, BatchNorm and ReLU processing are applied to the output of the Conv2 layer. The MaxPool2D layer performs a maximum pooling operation on the output of the Conv2 layer to reduce the size of the feature map while retaining the most important feature information. Pooling operations help reduce the amount of calculation and overfitting. The Conv3 layer receives the output of the MaxPool2D layer and continues to extract features. The output of the Conv3 layer is used as the final output of the read-only network to output the probability that the point cloud group sample is a positive sample, which is the first prediction result mentioned above.

[0057] S202: Calculate sample loss based on the true label of the point cloud group sample and the first prediction result.

[0058] Using the true label of the point cloud group sample (i.e., the sample category manually annotated) and the first prediction result output by the read-only network, the sample loss of each sample is calculated to measure the difference between the prediction result and the true label. The present invention calculates the sample loss using the following formula: in, For the The loss of samples, For the The true label of the point cloud group samples, the positive sample value is 1, and the negative sample value is 0. Prediction of the first The probability that a point cloud group is a positive sample.

[0059] S203: According to the sample loss of all samples and a preset hard example ratio, a plurality of hard example point cloud group samples are obtained by screening from all samples.

[0060] The OHEM module dynamically selects difficult point cloud group samples for training based on the loss value to improve the performance of the model. The read-only network provides preliminary classification predictions and loss calculations, while the read-write network performs targeted training on the screened difficult point cloud group samples. These two networks together constitute the application of the OHEM module in point cloud detection tasks. The OHEM module screens out difficult point cloud group samples from all samples based on the sample loss of all samples and a pre-set difficulty ratio. The difficulty ratio is a hyperparameter used to control the number and difficulty of the screened difficult point cloud group samples. Samples with higher sample losses are considered difficult point cloud group samples.

[0061] set up For all sample sets, For the samples, is the total number of samples, is the ratio of difficult examples, then the one with the highest loss value is selected samples, and obtain the sample set of the hard point cloud group : S204: Input the difficult point cloud group samples into the read-write network to obtain a second prediction result.

[0062] The selected difficult point cloud group samples are input into the read-write network. Unlike the read-only network, the read-write network is trainable and can update weights based on the input data. The read-write network outputs the second prediction result, which is the probability that the difficult point cloud group sample is a positive sample.

[0063] Specifically, the above reading and writing network also includes convolutional layer 1 (Convolutional Layer 1, Conv1), convolutional layer 2 (Convolutional Layer 2, Conv2), maximum pooling layer 2D (Max Pooling Layer 2D, MaxPool2D) and convolutional layer 3 (Convolutional Layer 3, Conv3).

[0064] The processing of the read-write network is as follows: The input of the read-write network is the hard point cloud group sample selected by the OHEM module. The Conv1, Conv2, and Conv3 layers of the read-write network are similar to the corresponding layers in the read-only network, performing convolution operations, BatchNorm, and ReLU processing. These layers continue to extract and transform features. The MaxPool2D layer is used to further reduce the size of the feature map. The output layer of the read-write network after the Conv3 layer can be a softmax layer, which is used to output the probability that the hard point cloud group sample is a positive sample, that is, the second prediction result mentioned above.

[0065] S205: Calculate a first loss according to the true labels of all point cloud group samples and the second prediction results.

[0066] The first loss is calculated using the true labels of all point cloud group samples and the second prediction results output by the read-write network. This loss value will be used for subsequent network training and optimization.

[0067] Point cloud group detection is a binary classification task. The present invention adopts binary cross entropy to calculate the first loss , calculate the loss for each point cloud group sample, and average the losses of all samples to get the loss of the entire point cloud group detection task. The loss function is as follows: in, is the number of point cloud groups, For the The true label of the point cloud group samples, the positive sample value is 1, and the negative sample value is 0. Predict the first The probability that a point cloud group is a positive sample.

[0068] This loss function is not only used to evaluate model performance, but also to screen samples of difficult point cloud groups for further training in the read-write network. During training, the weights of the read-write network are updated through the back-propagation algorithm to minimize the loss function and improve task performance.

[0069] By introducing an online difficult example mining module, the present invention enables the point cloud group detection network to focus more on samples that are difficult to classify, thereby improving the detection accuracy of complex roof contours. The read-write network can learn more feature information about complex roof contours through in-depth learning and weight updating of difficult example point cloud group samples, thereby enhancing the generalization ability of the model. The combined use of the read-only network and the read-write network enables the point cloud group detection network to improve training efficiency while ensuring accuracy. The read-only network is used to quickly screen difficult example point cloud group samples, while the read-write network focuses on in-depth learning of these difficult examples. This method is particularly suitable for scenes with imbalanced positive and negative samples. By dynamically screening difficult example point cloud group samples and conducting targeted training, the point cloud group detection network can better adapt to this imbalance, thereby improving the overall detection performance. In summary, the point cloud group detection network achieves efficient and accurate detection of complex roof contours by introducing a read-only network and a read-write network, and combining it with an online difficult example mining module. This method not only improves detection accuracy and generalization ability, but also optimizes training efficiency, and is particularly suitable for scenes with imbalanced positive and negative samples.

[0070] In an optional embodiment, vertex detection is a regression task. The goal of the vertex detection task is to accurately predict the vertex positions of geometric shapes in the point cloud, which is crucial for applications such as shape reconstruction and 3D printing. The present invention calculates the loss for each predicted vertex position and averages the losses of all predicted vertices to obtain the loss of the entire vertex detection task. The loss of the vertex detection network is referred to as the second loss , calculated using mean square error (MSE). The second loss Determined by: S301 . Input vertex samples into the vertex detection network to obtain vertex detection results, wherein the vertex detection results include predicted vertex positions and predicted vertex quantities.

[0071] S302, obtaining the actual vertex position of the vertex sample, and calculating the second loss according to the vertex predicted positions of all vertex samples, the actual vertex positions, and the predicted number of vertices: Among them, P is the number of predicted vertices, Predict the position of the vertex, is the actual vertex position.

[0072] In an optional embodiment, the goal of the edge detection task is to identify edge points in the point cloud, which is very important for subsequent tasks such as geometric shape analysis and reconstruction. Edge detection is also a loss function for a binary classification task, and is also calculated using binary cross entropy. It calculates the loss for each edge sample and averages the losses of all edge samples to obtain the loss of the entire edge detection task, hereinafter referred to as the third loss. The specific calculation process is as follows: S401. Input each edge sample into an edge detection network to obtain corresponding edge detection results; wherein the edge detection result is the probability that the edge sample is a positive sample.

[0073] S402: Calculate the third loss according to the true labels of all edge samples and the edge detection results. : in, is the number of edge samples, For the The true label of the edge samples, the positive sample value is 1, and the negative sample value is 0. Predict the first The probability that an edge is a positive sample.

[0074] The structures of the above vertex detection network and edge detection network can refer to the description in the prior art and will not be repeated here.

[0075] In an optional embodiment, when training the complete contour modeling network, an overall loss function is used to minimize the loss of all tasks. The overall loss is composed of the point cloud group detection loss (i.e., the first loss mentioned above), the vertex detection loss (i.e., the second loss mentioned above), and the edge detection loss (i.e., the third loss mentioned above), and is balanced by the weights and To adjust the relative importance between them. The contour modeling network training process is as follows: S501 , input the point cloud group samples into a pre-established contour modeling network, and calculate the first loss, the second loss and the third loss respectively.

[0076] The point cloud group samples are used as input data, which contain the 3D shape and contour information of the object. The point cloud group detection loss (first loss), vertex detection loss (second loss) and edge detection loss (third loss) are calculated respectively.

[0077] The first loss is used to evaluate the point cloud group detection network's ability to recognize input point cloud group samples, ensuring that the network can accurately detect and extract key features in the point cloud. The vertex detection network is responsible for identifying vertex information in the point cloud. The second loss is used to measure the network's performance on the vertex detection task to ensure the accuracy of the vertex position. The edge detection network is used to detect edge information in the point cloud, and the third loss is used to evaluate the network's effect on the edge detection task to ensure the clarity and continuity of the edge.

[0078] S502: Calculate the overall loss based on the first loss, the second loss and the third loss. The overall loss value comprehensively reflects the overall performance of the three network modules in the current training state.

[0079] When training the complete network, the present invention balances the weights and To minimize the overall loss , the overall loss function is: S503: updating the network parameters of the point cloud group detection network, the vertex detection network and the edge detection network according to the overall loss to obtain the pre-trained contour modeling network.

[0080] According to the overall loss value, the network parameters of the point cloud group detection network, vertex detection network, and edge detection network are updated using optimization methods such as back propagation algorithm and gradient descent method. This step aims to reduce the overall loss value by adjusting the network parameters, thereby improving the performance of the model. Repeat the above steps S501-503 for multiple iterations of training until the preset training rounds are reached or the model performance is no longer significantly improved.

[0081] Specifically, during the training process, the point cloud group samples are first predicted and the loss is calculated through the read-only network. The difficult point cloud group samples are screened out according to the sample loss, and these samples are sent to the read-write network for further training. At the same time, the edge detection task and the vertex detection task are also trained under the guidance of their respective loss functions. The network weights are updated through the back-propagation algorithm to minimize the overall loss. The training process usually includes multiple iteration cycles, and each cycle adjusts the training strategy according to the current network weights and loss values.

[0082] The present invention comprehensively considers the losses in multiple aspects such as point cloud group detection, vertex detection and edge detection, and the training process can more comprehensively evaluate and improve the performance of the model. This helps the model to accurately identify the contours and shape features of objects in complex scenes. Since a variety of loss functions and different network modules are used in the training process, the model can better adapt to different types of point cloud data and contour features. This enhances the generalization ability of the model, enabling it to perform well in unknown or new scenes. The training process gradually reduces the loss value of the model through iterative training and optimization of network parameters, thereby improving the performance of the model. This helps to optimize the utilization of computing resources and reduce unnecessary computing overhead and time costs.

[0083] In an optional embodiment, in the vertex detection task, in order to comprehensively evaluate the performance of the vertex detection network, the present invention introduces vertex detection precision (VP) and vertex detection recall ( ) are two key indicators to evaluate the effect of vertex detection. The specific definitions of VP and VR are as follows: Where TP is the number of correctly predicted vertices, i.e., vertices found in both the ground truth and the predictions. FP is the number of incorrectly predicted vertices. FN is the number of vertices that are incorrectly predicted as counterexamples. The higher the VP and VR values, the better the vertex predictions. For each predicted vertex, if the distance between the vertex and its nearest vertex in the ground truth is less than a predefined threshold, , then the vertex is considered to be the correctly predicted vertex.

[0084] The present invention can also use the vertex group detection accuracy (Patch Precision, PP) and the vertex group detection recall rate (Patch Recall, PR) to evaluate the vertex group detection effect, and use the edge detection accuracy (Edge Precision, EP) and the edge detection recall rate (Edge Recall, ER) to evaluate the edge detection effect. The calculation of PP, PR, EP and ER is similar to VP and VR.

[0085] This paper takes vertex detection precision (VP) and vertex detection recall as examples to illustrate the calculation process: S601. Obtain the number of correctly predicted vertices, the number of incorrectly predicted vertices, and the number of vertices incorrectly predicted as counterexamples; wherein, for a predicted vertex, if the distance between the vertex and the vertex closest to it in the reference true value is less than a predefined threshold, the vertex is regarded as a correctly predicted vertex.

[0086] First, obtain all vertex positions predicted by the vertex detection network and the corresponding benchmark real vertex positions, which are obtained by manual annotation or high-precision sensors. For each predicted vertex, find the vertex closest to it in the benchmark real value and calculate the distance between them. According to a predefined threshold (the threshold is determined according to the task requirements and the accuracy of the point cloud data), if the distance between a predicted vertex and its corresponding benchmark real vertex is less than the threshold, the vertex is considered to be correctly predicted. Count the number of all correctly predicted vertices, recorded as TP (TruePositives).

[0087] For those predicted vertices whose distance exceeds the threshold, they are regarded as incorrectly predicted vertices. The number of all incorrectly predicted vertices is counted and recorded as FP (False Positives) and FN (False Negatives). Among them, FP represents the number of negative samples that are incorrectly predicted as positive examples, and FN represents the number of positive samples that are incorrectly predicted as counter examples, that is, those vertices that actually exist but have not been detected.

[0088] S602: Calculate vertex detection accuracy according to the number of correctly predicted vertices and the number of incorrectly predicted vertices, where the vertex detection accuracy reflects the accuracy of the vertex detection network in predicting vertices.

[0089] Vertex detection accuracy is the ratio of the number of correctly predicted vertices to the number of all predicted vertices (including both correctly and incorrectly predicted vertices), that is: S603: Calculate the vertex detection recall rate according to the number of correctly predicted vertices and the number of vertices incorrectly predicted as counter-examples, which reflects the ability of the vertex detection network in detecting real vertices.

[0090] Vertex Detection Recall It is the ratio of the number of correctly predicted vertices to the number of all true vertices (including vertices that are correctly predicted and vertices that are incorrectly predicted as counterexamples), that is: S604: Evaluate the vertex detection effect of the vertex detection network using vertex detection accuracy and vertex detection recall.

[0091] The performance of the vertex detection network can be comprehensively evaluated using the two indicators of vertex detection precision and vertex detection recall. High precision indicates that the network has a high accuracy in predicting vertices, while high recall indicates that the network can detect more real vertices. By adjusting the network structure, optimizing the algorithm, or increasing the training data, the values ​​of these two indicators can be further improved, thereby improving the performance of the vertex detection network.

[0092] By introducing two indicators, vertex detection accuracy and recall rate, the present invention can more comprehensively evaluate the performance of the vertex detection network, rather than just relying on a single accuracy or recall rate indicator. These two indicators can provide powerful guidance for network optimization. For example, when the accuracy is low, it may be necessary to increase the regularization term or adjust the network structure to reduce overfitting; and when the recall rate is low, it may be necessary to increase the training data or improve the data enhancement strategy to improve the network's ability to detect real vertices. By continuously optimizing the vertex detection network, its effect in practical applications can be improved. For example, in 3D reconstruction tasks, more accurate vertex detection can generate more refined 3D models.

[0093] All codes in this paper are implemented based on the PyTorch framework, and experiments are carried out on a server equipped with GeForce RTX 4090. The device video memory is 24GB, the operating system version is Ubuntu 22.04, the cuda version is 11.1, and the pytorch version is 1.9.0.

[0094] The Adam optimizer was used during the training process, the learning rate was set to 0.001, and a weight decay of 1e-4 was applied to prevent overfitting. In order to further refine the learning rate adjustment strategy, a learning rate scheduler was set so that the learning rate would be automatically halved after every 10 epochs, so as to gradually refine the weight adjustment of the model. A total of 20 epochs were set for the entire training process, and the batch size was set to 512. In the training of each epoch, the model will traverse the entire training data set, perform forward propagation on each small batch sample, calculate the loss function, and then perform backpropagation and weight optimization update. At the end of each epoch, in addition to automatically updating the learning rate, the current model state will also be saved so that it can be continued for training or used for model evaluation.

[0095] In order to comprehensively evaluate the actual effect of Online Hard Example Mining (OHEM) in the point cloud group detection network, the present invention conducted ablation experiments on the SimulateRoof_Wireframe dataset. These experiments are specifically designed to compare the performance of the point cloud group detection network in terms of accuracy and recall when the online hard example mining function is enabled and disabled.

[0096] Table 1 shows the results of the ablation experiment. The results of the ablation experiment on the SimulateRoof_Wireframe dataset show that when OHEM is not used, the average precision (PP) of the point cloud group detection is 0.22 and the average recall rate (PR) is 0.95. When the OHEM-PCD module with the OHEM mechanism is applied, the average precision is increased to 0.92 and the average recall rate is increased to 0.96. This result shows that the OHEM mechanism significantly improves the model's ability to identify difficult samples, especially when dealing with unbalanced datasets, OHEM can effectively improve model performance.

[0097] Table 1 In order to more intuitively see the effect of the OHEM module in point cloud group detection, Figure 5 The visualization results of PatchNet and OHEM-PCD in point cloud group detection are shown. Specifically, while PatchNet identifies positive samples containing vertices, it also mistakenly marks a large number of negative samples as positive samples, which shows that it has certain limitations in distinguishing positive and negative samples. Relatively speaking, the OHEM-PCD method shows more superior performance. It not only successfully identifies positive samples containing vertices, but also effectively filters out negative samples that do not contain vertices, thereby obtaining more accurate detection results. This phenomenon proves the effectiveness of the OHEM mechanism in improving the model's ability to recognize negative samples.

[0098] Table 2 shows the quantitative evaluation results of different models on the SimulateRoof_Wireframe dataset. It can be seen that the performance of different models on the four key indicators of VP, VR, EP and ER. .

[0099] In all four indicators, the proposed method shows significant performance advantages, with vertex detection accuracy and vertex detection recall reaching 0.97 and 0.98, and edge detection accuracy and edge detection recall reaching 0.98 and 0.92 respectively. These results show that the method of the present invention can effectively detect vertex positions from point cloud data and accurately construct a complete wireframe structure. This performance improvement is due to the introduction of the online hard example mining (OHEM) mechanism, which enables the model to focus more on samples that are difficult to identify, thereby improving the generalization ability and overall performance of the model.

[0100] In order to more intuitively demonstrate the effect of the method proposed by the present invention, Figure 6 The visualization results of the proposed method are shown.

[0101] Table 2 The present invention constructs a large-scale synthetic dataset SimulateRoof_Wireframe for training and testing of building wireframe reconstruction. At the same time, the present invention proposes a new end-to-end building outline modeling method, which effectively improves the point cloud group detection accuracy by adding a point cloud group detection module based on online hard case mining, thereby improving the subsequent vertex detection and edge detection accuracy, obtaining a clean building wireframe model, and providing a new solution for efficient processing of building point clouds.

[0102] The end-to-end scatterer profile modeling device based on online hard example mining provided by the present invention is described below. The end-to-end scatterer profile modeling device based on online hard example mining described below and the end-to-end scatterer profile modeling method based on online hard example mining described above can be referenced to each other.

[0103] The end-to-end scatterer profile modeling device based on online hard example mining provided by the present invention refers to Figure 7 As shown, including: The data acquisition module 701 is used to acquire the point cloud group data to be modeled and the pre-trained contour modeling network; wherein the contour modeling network includes a backbone network, a point cloud group detection network, a vertex detection network and an edge detection network, and the point cloud group detection network is a detection network integrated with an online difficult example mining module; A feature extraction module 702 is used to input the point cloud group data to be modeled into the backbone network to obtain point cloud feature data; A first detection module 703, used for inputting the point cloud feature data into the point cloud group detection network to obtain a point cloud group containing vertices; A second detection module 704 is used to perform vertex detection and edge detection on the point cloud group containing vertices based on the vertex detection network and the edge detection network to generate a wireframe model; The contour modeling module 705 is used to generate a watertight plane based on the wireframe model to obtain a scatterer contour model.

[0104] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the end-to-end scatterer profile modeling method based on online hard example mining.

[0105] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0106] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the end-to-end scatterer profile modeling method based on online hard example mining provided by the above methods.

[0107] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the end-to-end scatterer profile modeling method based on online hard example mining provided by the above methods.

[0108] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0109] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An end-to-end scatterer profile modeling method based on online hard example mining, characterized in that: include: Obtaining point cloud group data to be modeled and a pre-trained contour modeling network; wherein the contour modeling network includes a backbone network, a point cloud group detection network, a vertex detection network and an edge detection network, and the point cloud group detection network is a detection network integrated with an online difficult example mining module; Inputting the point cloud group data to be modeled into the backbone network to obtain point cloud feature data; Inputting the point cloud feature data into the point cloud group detection network to obtain a point cloud group including vertices; Performing vertex detection and edge detection on the point cloud group containing vertices based on the vertex detection network and the edge detection network to generate a wireframe model; A watertight plane is generated based on the wireframe model to obtain a scatterer contour model.

2. The end-to-end scatterer profile modeling method based on online hard example mining according to claim 1 is characterized in that: The contour modeling network is trained in the following way: Input the point cloud group samples into the pre-established contour modeling network, and calculate the first loss, the second loss and the third loss respectively; wherein the first loss is the loss of the point cloud group detection network, the second loss is the loss of the vertex detection network, and the third loss is the loss of the edge detection network; Calculate the overall loss based on the first loss, the second loss and the third loss; The network parameters of the point cloud group detection network, the vertex detection network and the edge detection network are updated according to the overall loss to obtain the pre-trained contour modeling network.

3. The end-to-end scatterer profile modeling method based on online hard example mining according to claim 2, characterized in that: The point cloud group detection network includes a read-only network and a read-write network; The first loss is determined in the following manner: Inputting the point cloud group sample into the read-only network to obtain a first prediction result; wherein the first prediction result is the probability that the point cloud group sample is a positive sample; Calculate sample loss based on the true label of the point cloud group sample and the first prediction result; According to the sample loss of all samples and a preset hard example ratio, a plurality of hard example point cloud group samples are obtained by screening from all samples; Inputting the difficult point cloud group sample into the read-write network to obtain a second prediction result; wherein the second prediction result is the probability that the difficult point cloud group sample is a positive sample; The first loss is calculated according to the true labels of all point cloud group samples and the second prediction results.

4. The end-to-end scatterer profile modeling method based on online hard example mining according to claim 2, characterized in that: The second loss is determined in the following manner: Inputting vertex samples into the vertex detection network to obtain vertex detection results, wherein the vertex detection results include vertex predicted positions and vertex predicted numbers; The actual vertex positions of the vertex samples are obtained, and the second loss is calculated according to the predicted vertex positions of all vertex samples, the actual vertex positions, and the predicted number of vertices.

5. The end-to-end scatterer profile modeling method based on online hard example mining according to claim 2, characterized in that: The third loss is determined in the following manner: Each edge sample is input into the edge detection network to obtain the corresponding edge detection results; wherein the edge detection result is the probability that the edge sample is a positive sample; The third loss is calculated according to the true labels of all edge samples and the edge detection result.

6. The end-to-end scatterer profile modeling method based on online hard example mining according to claim 1, characterized in that: The method further comprises: Obtain the number of correctly predicted vertices, the number of incorrectly predicted vertices, and the number of vertices incorrectly predicted as counterexamples; wherein, for a predicted vertex, if the distance between the vertex and the vertex closest to it in the benchmark true value is less than a predefined threshold, the vertex is considered to be a correctly predicted vertex; Calculating vertex detection accuracy according to the number of correctly predicted vertices and the number of incorrectly predicted vertices; Calculating a vertex detection recall rate according to the number of correctly predicted vertices and the number of vertices incorrectly predicted as counterexamples; The vertex detection accuracy and vertex detection recall are used to evaluate the vertex detection effect of the vertex detection network.

7. An end-to-end scatterer profile modeling device based on online hard example mining, characterized in that: include: A data acquisition module, used to acquire the point cloud group data to be modeled and a pre-trained contour modeling network; wherein the contour modeling network includes a backbone network, a point cloud group detection network, a vertex detection network and an edge detection network, and the point cloud group detection network is a detection network integrated with an online difficult example mining module; A feature extraction module, used for inputting the point cloud group data to be modeled into the backbone network to obtain point cloud feature data; A first detection module, used for inputting the point cloud feature data into the point cloud group detection network to obtain a point cloud group containing vertices; A second detection module, configured to perform vertex detection and edge detection on the point cloud group containing vertices based on the vertex detection network and the edge detection network to generate a wireframe model; The contour modeling module is used to generate a watertight plane based on the wireframe model to obtain a scatterer contour model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the end-to-end scatterer profile modeling method based on online hard example mining as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the end-to-end scatterer profile modeling method based on online hard example mining as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the end-to-end scatterer profile modeling method based on online hard example mining as described in any one of claims 1 to 6 is implemented.

Images