Camera point clustering method, selection method and related device

Abstract

The embodiments of the present application disclose a camera point clustering method, a selection method, and related devices. The clustering method includes: using a pre-trained feature extraction model to extract features of each image in an image set; clustering the features into multiple first-parent clustering clusters based on prototypes; and clustering the features of each image in each first-parent clustering cluster into multiple child clustering clusters based on density; determining the features corresponding to the center point of the features of each image in each child clustering cluster, and clustering the features corresponding to each center point into multiple second-parent clustering clusters based on density; fusing multiple second-parent clustering clusters and each child clustering cluster to obtain the clustering results of the camera points of the image set. Through this application, the technical problem in the related art that it is difficult to obtain accurate point information for most image sets due to factors such as different collection personnel, collection in different time periods, and unrecorded points is solved, and the technical effect of obtaining accurate point information is achieved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a camera point clustering method, a selection method, and related devices. Background Art

[0002] Existing computer vision service image sets are typically composed of data collected from multiple service locations. This location information can help users and developers select key locations (saving manpower), determine the adequacy of the image set's locations (using the total number of locations), determine the balance of location data (using the number of images at different locations), and create heterogeneous test sets (to assess model generalization).

[0003] Most existing technologies distinguish scenes based on semantic information, such as train station, bus station, tourist attraction 1, and tourist attraction 2; or they only obtain point information through camera numbers. However, most image sets do not retain camera number information, or the number information is retained but requires manual verification one by one to verify the acquisition.

[0004] However, due to factors such as different collection personnel, collection in different time periods, and unrecorded points, it is difficult to obtain accurate point information in most image sets.

[0005] For the above problems, no effective solutions have been proposed yet. Summary of the Invention

[0006] The embodiments of the present application provide a camera point clustering method, a selection method, and related devices to at least solve the technical problem in the related art that it is difficult to obtain accurate point information in most image sets due to factors such as different collection personnel, collection in different time periods, and unrecorded points.

[0007] According to one aspect of an embodiment of the present application, a method for clustering camera points is provided, comprising: acquiring an image set captured by cameras at different points; extracting features of each image in the image set using a pre-trained feature extraction model, wherein the feature extraction model is trained based on images annotated with camera point information; clustering the features into a plurality of first parent clustering clusters based on prototypes, wherein the number of images in each clustering cluster is not greater than a set value; and clustering the features of each image in each of the first parent clustering clusters into a plurality of child clustering clusters based on density; determining features corresponding to center points of features of each image in each of the child clustering clusters, and clustering features corresponding to each of the center points into a plurality of second parent clustering clusters based on density; and fusing the plurality of second parent clustering clusters and the child clustering clusters to obtain a clustering result of the camera points in the image set.

[0008] Optionally, a pre-trained feature extraction model is used to extract features of each image in the image set, including: obtaining a feature map of each image in the image set based on the feature extraction model, dividing the feature map of each image in the image set into multiple regions for pooling to obtain multiple regional features of each image in the image set, and performing feature extraction on the multiple regional features of each image in the image set to obtain features of each image in the image set.

[0009] Optionally, after fusing the multiple second parent clusters and each of the child clusters to obtain the clustering results of the camera points of the image set, the method further includes: determining the point complexity of each camera point, wherein the point complexity is used to characterize the degree of convergence of the clustering results of the camera points; judging whether the point complexity of each of the camera points is greater than a preset point complexity threshold; if the point complexity of the camera point is greater than the point complexity threshold, adjusting the clustering parameters and re-clustering the camera points of the image set.

[0010] Optionally, adjusting the clustering parameters and re-clustering the camera points of the image set includes: reducing the similarity distance parameter based on density clustering, and re-clustering the camera points of the image set until the number of camera points of the image set are clustered reaches a set number, or the point complexity of each camera point is not greater than the point complexity threshold.

[0011] Optionally, the method further includes: determining the number of images required for different dimensions; obtaining images captured by cameras at different points based on the number of images required for the different dimensions, and labeling the camera point information of all images to obtain a training set; and using the training set to train the feature extraction model.

[0012] Optionally, the clustering method adopted in the prototype clustering is Kmeans clustering, and the clustering method adopted in the density clustering is DBscan clustering.

[0013] According to another aspect of an embodiment of the present application, a method for selecting camera points is also provided, including: obtaining features of each image in an image set and a clustering result of the camera points of the image set, wherein the image set includes images captured by cameras at different points; obtaining the point center and point complexity of each camera point according to the features of each image in the image set and the clustering result of the camera points of the image set; clustering the point centers of each of the camera points to obtain similar camera points; and selecting camera points according to the point complexity and the similar camera points to obtain a target camera point.

[0014] Optionally, camera point selection is performed according to the point complexity and the similar camera points to obtain a target camera point, including: sorting the similar camera points according to the point complexity, and selecting the first N similar camera points as the target camera point, wherein the value of N is determined according to the number of the similar camera points and a preset selection ratio.

[0015] According to another aspect of an embodiment of the present application, a camera point clustering device is also provided, including: a first acquisition module for acquiring an image set captured by cameras at different points; an extraction module for using a pre-trained feature extraction model to extract features of each image in the image set, wherein the feature extraction model is trained based on images marked with camera point information; a first clustering module for clustering the features into multiple first parent clustering clusters based on prototypes, wherein the number of images in each clustering cluster is not greater than a set value; and clustering the features of each image in each of the first parent clustering clusters into multiple child clustering clusters based on density; a second clustering module for determining the features corresponding to the center point of the features of each image in each of the child clustering clusters, and clustering the features corresponding to each of the center points into multiple second parent clustering clusters based on density; a fusion module for fusing the multiple second parent clustering clusters and each of the child clustering clusters to obtain a clustering result of the camera points of the image set.

[0016] According to another aspect of an embodiment of the present application, a camera point selection device is also provided, including: a second acquisition module, used to obtain the characteristics of each image in an image set and the clustering results of the camera points of the image set, wherein the image set includes images captured by cameras at different points; a third acquisition module, used to obtain the point center and point complexity of each camera point according to the characteristics of each image in the image set and the clustering results of the camera points of the image set; a third clustering module, used to cluster the point centers of each of the camera points to obtain similar camera points; a selection module, used to select camera points according to the point complexity and the similar camera points to obtain the target camera point.

[0017] According to another aspect of an embodiment of the present application, an electronic device is further provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the steps of any one of the above methods.

[0018] According to another aspect of an embodiment of the present application, a computer-readable storage medium is further provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the steps of any one of the above methods.

[0019] In an embodiment of the present application, a set of images captured by cameras at different points is obtained; a pre-trained feature extraction model is used to extract features of each image in the image set; the features are clustered based on prototypes into multiple first parent clustering clusters, where the number of images in each clustering cluster is not greater than a set value; and the features of each image in each first parent clustering cluster are clustered based on density into multiple child clustering clusters; the features corresponding to the center points of the features of each image in each child clustering cluster are determined, and the features corresponding to each center point are clustered based on density into multiple second parent clustering clusters; the multiple second parent clustering clusters and the child clustering clusters are fused to obtain the clustering results of the camera points of the image set, thereby solving the technical problem in the related art that it is difficult to obtain accurate point information for most image sets due to factors such as different collection personnel, collection in different time periods, and unrecorded points, thereby achieving the technical effect of obtaining accurate point information. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0021] Figure 1 A flowchart of a camera point clustering method provided in an embodiment of the present application;

[0022] Figure 2 A schematic diagram of a feature extraction model in a training phase provided in an embodiment of the present application;

[0023] Figure 3 A schematic diagram of a feature extraction model in the application stage provided in an embodiment of the present application;

[0024] Figure 4 A flowchart of another camera point clustering method provided in an embodiment of the present application;

[0025] Figure 5 A flowchart of a method for selecting camera points provided in an embodiment of the present application;

[0026] Figure 6 A flowchart of another method for selecting camera points provided in an embodiment of the present application;

[0027] Figure 7 A schematic diagram of a camera point clustering device provided in an embodiment of the present application;

[0028] Figure 8 A schematic diagram of a camera point selection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0030] It should be noted that the terms "first," "second," and the like in the specification, claims, and drawings of this application are used to distinguish between different objects, rather than to define a specific order. The steps shown in the flowcharts of the accompanying drawings can be executed in a computer system, such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown.

[0031] According to one aspect of an embodiment of the present application, a camera point clustering method is provided. Figure 1 A flowchart of a camera point clustering method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes the following steps:

[0032] Step S102, obtaining a set of images captured by cameras at different locations;

[0033] A camera, at a fixed position and angle, continuously captures a scene (background information) called a camera position or position. For an image, the portion that continuously changes within a certain data collection period is called the foreground; the portion that remains unchanged within a certain data collection period is called the background.

[0034] Step S104: extracting features of each image in the image set using a pre-trained feature extraction model, wherein the feature extraction model is trained based on images annotated with camera position information;

[0035] Step S106: clustering the features into a plurality of first parent clusters based on the prototype, wherein the number of images in each cluster is not greater than a set value; and clustering the features of each image in each first parent cluster into a plurality of child clusters based on density;

[0036] Step S108, determining the features corresponding to the center points of the features of each image in each child cluster, and clustering the features corresponding to each center point into multiple second parent clusters based on density;

[0037] Step S110 , fusing multiple second parent clusters and each child cluster to obtain a clustering result of the camera points of the image set.

[0038] The above setting value can be set based on the number of images in the image set. For example, the larger the number of images in the image set, the larger the setting value; the smaller the number of images in the image set, the smaller the setting value. Point information can be obtained based on the clustering results of the camera points.

[0039] Through the above steps, it is possible to obtain a set of images collected by cameras at different points; use a pre-trained feature extraction model to extract the features of each image in the image set; cluster the features based on prototypes into multiple first parent clustering clusters, where the number of images in each cluster is not greater than a set value; and cluster the features of each image in each first parent clustering cluster into multiple child clustering clusters based on density; determine the features corresponding to the center point of the features of each image in each child clustering cluster, and cluster the features corresponding to each center point into multiple second parent clustering clusters based on density; fuse multiple second parent clustering clusters and each child clustering cluster to obtain the clustering results of the camera points of the image set, thereby solving the technical problem in related technologies that it is difficult to obtain accurate point information for most image sets due to factors such as different collection personnel, collection in different time periods, and unrecorded points, and achieving the technical effect of obtaining accurate point information.

[0040] After obtaining the clustering results for the camera points in an image set, the total number of camera points can be used to determine whether the image set contains sufficient camera points. The distribution of the number of different camera points can be used to determine whether the image set is point-balanced. Based on the point information, the different camera points can be divided into different groups to construct training and test sets. The test set includes homologous test sets and heterologous test sets. Homologous test sets refer to test sets with the same camera points as the training set, while heterologous test sets refer to test sets with different camera points.

[0041] In an optional embodiment, a pre-trained feature extraction model is used to extract features of each image in the image set, including: obtaining a feature map of each image in the image set based on the feature extraction model, dividing the feature map of each image in the image set into multiple regions for pooling to obtain multiple regional features of each image in the image set, and performing feature extraction on the multiple regional features of each image in the image set to obtain features of each image in the image set.

[0042] For image feature extraction, the commonly used feature extraction method is to use a general model to obtain the feature map, then directly use global pooling to extract the corresponding features, and then send them to the fully connected layer to extract the corresponding dimensional features. Since there are regional changes in some images of the same camera point, when some areas change, it will cause the problem of feature instability. It is difficult to obtain accurate image features using commonly used feature extraction methods. Therefore, the feature extraction model of this application adds a regional feature extraction layer. The regional feature extraction layer uses a method of pooling multiple feature regions to divide the feature map into multiple regions, and performs pooling of multiple regions separately, and then sends it to the fully connected layer to extract the corresponding dimensional features.

[0043] Figure 2 A schematic diagram of a feature extraction model in the training phase provided in an embodiment of the present application is shown in FIG. Figure 2 As shown in the figure, during the training phase of the feature extraction model, the feature extraction model includes a general model, a regional feature extraction layer, a fully connected layer, and a deep learning convolutional layer of a classification layer. In addition, the feature extraction model is trained using images in the training set, and the classification layer is used to treat images at the same camera point as one category. The feature extraction model is continuously optimized through loss calculation and back propagation.

[0044] Figure 3 A schematic diagram of a feature extraction model in the application stage provided in an embodiment of the present application is shown in FIG. Figure 3 As shown in the figure, in the application stage of the feature extraction model, the trained feature extraction model includes a general model, a regional feature extraction layer, and a deep learning convolution layer of a fully connected layer; in addition, based on the features of each image in the image set extracted by the trained feature extraction model, a clustering method is used to obtain the corresponding point information.

[0045] It should be noted that the general model, also known as the deep convolution classification model, is used to obtain the feature map of the image; the regional feature extraction layer is used to divide the feature map of the image into multiple regions, and perform pooling of multiple regions separately to obtain multiple regional features; the fully connected layer is used to extract the features of the image based on multiple regional features.

[0046] In an embodiment of the present application, when part of the image area changes, only some dimensional features are affected. The fully connected layer will learn the relationship between different feature areas and ignore the changed areas, effectively avoiding the adverse effects of image area changes on feature stability. Therefore, the pre-trained feature extraction model is used to extract the features of the image at the same camera point, thereby improving the feature stability of the image.

[0047] Because clustering methods are applicable to image sets from any business, they may have various camera positions. In actual testing, the distances between different camera positions vary, making it impossible to use a single similarity parameter to distinguish all camera positions. For example, the first image belongs to camera position 1, the second image belongs to camera position 2, and the third and fourth images both belong to camera position 3. The first and second images belong to different camera positions, but their camera positions are similar, resulting in a close feature distance, denoted by α. The first and third images belong to different camera positions, resulting in a far distance, denoted by β. The third and fourth images belong to the same camera position, resulting in a close feature distance, denoted by γ. The similarity distance reflects the degree of similarity between camera position images and is related to the degree of variation in the foreground and background of the images. For example, the first and second images belong to different camera positions, but the foreground and background differences between the two images are relatively small. The third and fourth images belong to the same camera position, resulting in a small background difference but a large foreground difference. The similarity distance relationship among the three is β > γ > α. When the selected similarity threshold ε, β>ε>γ>α, camera positions 1 and 2 are clustered into one camera position, and camera position 3 is clustered into one camera position. Therefore, it is not possible to use a single similarity threshold and a universal clustering method to classify the above image into three camera positions. Therefore, given the varying similarities between different image sets and different camera positions, a single similarity distance parameter cannot be used to solve the camera position clustering problem for all image sets.

[0048] To address the above technical issues, an optional embodiment of the present application provides a camera point clustering method, the specific implementation steps of which are as follows:

[0049] Step S110: After fusing the plurality of second parent clusters and the child clusters to obtain a clustering result of the camera points of the image set, determining the point complexity of each camera point, wherein the point complexity is used to represent the degree of convergence of the clustering result of the camera points;

[0050] Step S112: determining whether the point complexity of each camera point is greater than a preset point complexity threshold;

[0051] Step S114: If the point complexity of the camera points is greater than the point complexity threshold, the clustering parameters are adjusted and the camera points of the image set are re-clustered.

[0052] Furthermore, the above-mentioned step S114 includes: reducing the similarity distance parameter based on density clustering, and re-clustering the camera points of the image set until the number of camera points of the image set are clustered reaches a set number of times, or the point complexity of each camera point is not greater than the point complexity threshold.

[0053] The point complexity of the above camera points can be calculated by class diameter, F norm of all points to the center point, etc.

[0054] In an embodiment of the present application, by comparing the point complexity of each camera point with a preset point complexity threshold, when the point complexity of the camera point is greater than the point complexity threshold, the clustering parameters are adjusted, and the camera points of the image set are re-clustered, thereby solving the camera point clustering problem of all image sets, so that the camera point clustering method can obtain a more robust clustering effect on more business image sets.

[0055] Figure 4 This is a flowchart of another camera point clustering method provided in an embodiment of the present application, such as Figure 4 As shown, it is divided into the following 5 steps:

[0056] Step 1. Feature block clustering. First, use Kmeans clustering to cluster the features of each image in the image set into multiple first-level parent clusters. Then use DBscan clustering to cluster each first-level parent cluster to obtain multiple child clusters.

[0057] Step 2. Secondary center clustering. Continue using DBscan clustering to cluster the features corresponding to the center points of each image within each child cluster into multiple second-level parent clusters. Then, fuse these multiple second-level parent clusters with each child cluster to obtain the clustering results of the camera points in the image set, thus obtaining the point information.

[0058] Step 3. Calculate the point complexity. Based on the clustering results of the camera points in the image set, traverse all camera points and obtain the point complexity of each camera point.

[0059] Step 4. Determine the point complexity. Determine whether the point complexity of each camera point is greater than the preset point complexity threshold η. If the point complexity of the camera point is greater than the point complexity threshold η, the clustering effect is considered poor, and the current camera point has many images from other camera points, so re-cluster. In other words, it is necessary to adjust the clustering parameters and re-cluster the camera points in the image set. If the point complexity of the camera point is less than or equal to the point complexity threshold η, the clustering is considered good and the clustering result is retained.

[0060] Step 5. Re-clustering. Obtain the decay count and similarity distance, where the initial values ​​of the decay count and the similarity distance are both pre-set; determine whether the decay count is 0; if it is not 0, reduce the similarity distance parameter ε and the decay count τ by 1 (τ = τ - 1), re-cluster the data to be re-clustered, and obtain a new clustering result; if the decay count is 0, the current clustering result is considered sufficiently similar, and the data to be re-clustered is placed in the retained clustering result, ending the entire process.

[0061] In an optional embodiment, the above method also includes: determining the number of images required for different dimensions; obtaining images captured by cameras at different points based on the number of images required for different dimensions, and labeling the camera point information of all images to obtain a training set; and using the training set to train the feature extraction model.

[0062] Optionally, the training set can be constructed by obtaining images captured by multiple camera points according to the number of images required for different dimensions, where the different dimensions include but are not limited to application scenarios, image variability, and point differences.

[0063] Optionally, by collecting images from different application scenarios, all application scenarios required by the business can be covered, including but not limited to companies, schools, kitchens, hospitals, roads, factories, construction sites, mines, ports, oceans, fields, mountains, forests, lakes, infrared, black light, etc.

[0064] Optionally, by collecting images with different degrees of image variation, similar features are extracted for images captured at the same camera point with different degrees of background variation. The above-mentioned image variation includes but is not limited to light changes (such as early morning, noon, evening, night, etc.), changes in crowd density, foreground occlusions (such as vehicles, foreign objects, etc.), seasonal changes (such as spring, summer, autumn, winter, etc.), weather changes (sunny, cloudy, rainy, snowy, etc.), and color changes (such as lights of different colors, etc.). For example, images of the color, water level, and seasonal changes at the same camera point in a pool; images of light changes at the same camera point on a road; images of different goods at the same camera point in a factory. Adding such images to the training set allows the feature extraction model to learn the changes at the same point, making the features of the two similar. It should be noted that the background variation can be divided into multiple levels, and the proportion of images of different levels can be controlled to ensure the convergence and robustness of the feature extraction model.

[0065] Optionally, by collecting the differences between different camera points in the same application scenario, features with large differences are extracted for images captured by different camera points in the same application scenario. The above-mentioned point differences include but are not limited to differences between indoors and outdoors (such as the same scene, indoors and outdoors), differences between different floors (such as the same scene, different floors), differences at different angles and heights (such as the same scene, different points shot), differences at different zoom levels (such as the same scene, different distances shot), etc. For example, images of a conveyor belt from different camera points are artificially divided into two points because of different shooting positions; images of a road from different camera points are artificially divided into two points because of different shooting angles. Adding such images to the training set will help the feature extraction model distinguish between similar camera points.

[0066] Furthermore, based on the number of images required for different dimensions, images collected by cameras at different points are obtained, and the camera point information of all images is annotated to obtain a training set, including: setting the number of images required for different application scenarios; setting the number of images required for the degree of image change at the same camera point (controlling the ratio of the number of small, medium and large changes); setting the number of images required for the position differences of different camera points in the same application scenario, and then collecting and annotating images in a targeted manner to construct a training set.

[0067] In an embodiment of the present application, by constructing a training set for a business scenario based on the above method, the feature distances of images at the same camera point extracted by the feature extraction model can be made as close as possible, and the feature distances of images at different camera points can be made as far as possible; and changes in the foreground (people flow, occlusion, light, color, etc.) and partial background changes can be overcome.

[0068] In an optional embodiment, the clustering method adopted based on prototype clustering is Kmeans clustering, and the clustering method adopted based on density clustering is DBscan clustering.

[0069] Because the feature dimensionality of camera point clustering is large, approximately 512, direct DBscan clustering can cause memory overflows and memory corruption when the image dataset is large, such as 1 million. However, simply partitioning the features (for example, sorting the 1 million features, where features numbered 1 to 100,000 are grouped into a first-parent cluster, and the remaining first-parent clusters are grouped sequentially) and then using DBscan clustering results in a significant increase in noise classes, resulting in low clustering accuracy. This is because the same camera point features are distributed across different first-parent clusters, and DBscan clustering results in a higher number of noise classes.

[0070] In order to solve the problems of memory corruption caused by DBscan clustering and low accuracy of simple block clustering in ultra-large-scale image sets, in an embodiment of the present application, the features are divided into multiple first-parent clustering clusters based on Kmeans clustering, which can solve the camera point clustering of ultra-large-scale image sets (more than 10 million data, 512-dimensional features); compared with the direct simple block division of features, the first-parent clustering cluster mapping obtained based on Kmeans clustering is tighter, and each camera point will be concentrated in several first-parent clustering clusters, effectively reducing the noise class (outliers) generated by DBscan clustering.

[0071] In addition, Kmeans clustering is a general unsupervised clustering method that requires specifying the number of clusters; DBscan clustering is a general unsupervised clustering method based on density clustering that requires specifying the similarity distance parameter.

[0072] According to another aspect of the embodiment of the present application, a method for selecting camera points is also provided. Figure 5 A flowchart of a method for selecting camera points provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the method includes the following steps:

[0073] Step S502: obtaining features of each image in an image set and clustering results of camera positions in the image set, wherein the image set includes images captured by cameras at different positions;

[0074] Step S504: obtaining the point center and point complexity of each camera point based on the features of each image in the image set and the clustering result of the camera points in the image set;

[0075] Step S506: clustering the point centers of each camera point to obtain similar camera points;

[0076] Step S508 : Select camera points based on point complexity and similar camera points to obtain target camera points.

[0077] It should be noted that the features of each image in the image set and the clustering results of the camera points in the image set can be obtained using the aforementioned camera point clustering method. Furthermore, the clustering method used to cluster the point centers of each camera point is DBscan clustering, and the clustering steps are similar to those of the aforementioned camera point clustering method.

[0078] Through the above steps, the features of each image in the image set and the clustering results of the camera points of the image set can be obtained; based on the features of each image in the image set and the clustering results of the camera points of the image set, the point center and point complexity of each camera point can be obtained respectively; the point centers of each camera point are clustered to obtain similar camera points; camera points are selected based on the point complexity and similar camera points to obtain target camera points, thereby solving the technical problem in related technologies that the traditional method cannot know which points are repeated, resulting in low efficiency in camera point data collection, and achieving the technical effect of improving data collection efficiency.

[0079] In an optional embodiment, camera points are selected based on point complexity and similar camera points to obtain target camera points, including: sorting similar camera points according to point complexity, and selecting the first N similar camera points as target camera points, wherein the value of N is determined based on the number of similar camera points and a preset selection ratio.

[0080] In an embodiment of the present application, similar camera points are sorted by point complexity to select target camera points, which can save data collectors the time spent on point collection and improve data collection efficiency.

[0081] When a business has many camera points, such as 1,000 camera points. Traditionally, data collectors need to collect data from 1,000 camera points, or randomly collect data from some camera points. Traditional methods cannot know which camera points are repeated. Based on this problem, an embodiment of the present application proposes another method for selecting camera points. This method for selecting camera points will recommend collection points (corresponding to the above-mentioned target camera points), and data collectors only need to collect data according to the recommended collection points.

[0082] Figure 6 A flowchart of another method for selecting camera points provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, it includes the following processes:

[0083] Step 1. Obtaining point information. A pre-trained feature extraction model is used to extract features from each image in the image set. The camera point clustering method described above is used to obtain the clustering results for the camera points in the image set. Based on the features of each image in the image set and the clustering results for the camera points in the image set, the point center and point complexity of each camera point are obtained. Point complexity is used to indicate the degree of convergence of the camera point clustering results.

[0084] Step 2. Search for similar camera points. Use the point center of each camera point to cluster and obtain the clustering results of similar camera points. Determine whether the point complexity of each camera point is greater than the preset point complexity threshold; if the point complexity of the camera point is greater than the point complexity threshold, adjust the clustering parameters and use the point center of each camera point to re-cluster. Among them, adjusting the clustering parameters and re-clustering the point center of each camera point includes: reducing the similarity distance parameter based on density clustering, and re-clustering using the point center of each camera point until the number of times the point center of each camera point is clustered reaches the set number, or the point complexity of each camera point is not greater than the point complexity threshold. It should be noted that clustering using the point center of each camera point is based on density clustering.

[0085] Step 3. Select similar camera points. Based on the number m of similar camera points, control the selection ratio k, sort the similar camera points by point complexity, and select the top N similar camera points (N = m * k) as the target camera points.

[0086] According to another aspect of the embodiment of the present application, a camera point clustering device is also provided. Figure 7 A schematic diagram of a camera point clustering device provided in an embodiment of the present application, such as Figure 7 As shown, the camera point clustering device includes: a first acquisition module 702, an extraction module 704, a first clustering module 706, a second clustering module 708, and a fusion module 710. The camera point clustering device is described in detail below.

[0087] A first acquisition module 702 is used to acquire a set of images captured by cameras at different locations;

[0088] An extraction module 704, connected to the first acquisition module 702, is configured to extract features of each image in the image set using a pre-trained feature extraction model, wherein the feature extraction model is trained based on images annotated with camera position information;

[0089] A first clustering module 706 is connected to the extraction module 704 and is configured to cluster the features into a plurality of first parent clusters based on the prototype, wherein the number of images in each cluster is not greater than a set value; and cluster the features of each image in each first parent cluster into a plurality of child clusters based on density.

[0090] The second clustering module 708 is connected to the first clustering module 706 and is used to determine the features corresponding to the center points of the features of each image in each child cluster, and cluster the features corresponding to each center point into multiple second parent clusters based on density;

[0091] The fusion module 710 is connected to the second clustering module 708 and is used to fuse multiple second parent clusters and each child cluster to obtain a clustering result of the camera points of the image set.

[0092] In an embodiment of the present application, the camera point clustering device obtains an image set collected by cameras at different points; uses a pre-trained feature extraction model to extract features of each image in the image set; clusters the features into multiple first parent clustering clusters based on prototypes, with the number of images in each clustering cluster not exceeding a set value; and clusters the features of each image in each first parent clustering cluster into multiple child clustering clusters based on density; determines the features corresponding to the center point of the features of each image in each child clustering cluster, and clusters the features corresponding to each center point into multiple second parent clustering clusters based on density; fuses multiple second parent clustering clusters and each child clustering cluster to obtain the clustering results of the camera points in the image set, thereby solving the technical problem in the related art that it is difficult to obtain accurate point information for most image sets due to factors such as different collection personnel, collection in different time periods, and unrecorded points, thereby achieving the technical effect of obtaining accurate point information.

[0093] It should be noted here that the above-mentioned first acquisition module 702, extraction module 704, first clustering module 706, second clustering module 708 and fusion module 710 correspond to steps S102 to S110 in the method embodiment. The examples and application scenarios implemented by the above-mentioned modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned method embodiment.

[0094] According to another aspect of the embodiment of the present application, a device for selecting camera points is also provided. Figure 8 A schematic diagram of a camera point selection device provided in an embodiment of the present application, such as Figure 8 As shown, the camera point selection device includes: a second acquisition module 802, a third acquisition module 804, a third clustering module 806 and a selection module 808. The camera point selection device is described in detail below.

[0095] A second acquisition module 802 is configured to acquire features of each image in an image set and clustering results of camera positions in the image set, wherein the image set includes images captured by cameras at different positions;

[0096] The third acquisition module 804 is connected to the second acquisition module 802 and is used to obtain the point center and point complexity of each camera point according to the characteristics of each image in the image set and the clustering result of the camera points in the image set;

[0097] The third clustering module 806 is connected to the third acquisition module 804 and is used to cluster the point centers of each camera point to obtain similar camera points;

[0098] The selection module 808 is connected to the third clustering module 806 and is used to select camera points according to point complexity and similar camera points to obtain target camera points.

[0099] In an embodiment of the present application, the camera point selection device can obtain the characteristics of each image in the image set and the clustering results of the camera points of the image set; obtain the point center and point complexity of each camera point according to the characteristics of each image in the image set and the clustering results of the camera points of the image set; cluster the point centers of each camera point to obtain similar camera points; select camera points according to the point complexity and similar camera points to obtain target camera points, thereby solving the technical problem in the related art that the traditional method cannot know which points are repeated, resulting in low efficiency in camera point data collection, and achieving the technical effect of improving data collection efficiency.

[0100] It should be noted here that the above-mentioned second acquisition module 802, third acquisition module 804, third clustering module 806 and selection module 808 correspond to steps S502 to S508 in the method embodiment. The examples and application scenarios implemented by the above-mentioned modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned method embodiment.

[0101] According to another aspect of an embodiment of the present application, an electronic device is also provided, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the steps of any one of the above methods.

[0102] According to another aspect of an embodiment of the present application, a computer-readable storage medium is further provided, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the steps of any of the above methods.

[0103] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0104] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0105] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0106] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0107] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A camera point clustering method, characterized in that: include: Obtain a set of images collected by cameras at different locations; Extracting features of each image in the image set using a pre-trained feature extraction model, wherein the feature extraction model is trained on images annotated with camera position information; Clustering the features into a plurality of first parent clusters based on prototypes, wherein the number of images in each cluster is not greater than a set value; and clustering the features of each image in each of the first parent clusters into a plurality of child clusters based on density; Determining a feature corresponding to a center point of a feature of each image in each of the child clusters, and clustering the features corresponding to each of the center points into a plurality of second parent clusters based on density; The plurality of second parent clusters and each of the child clusters are fused to obtain a clustering result of the camera points of the image set.

2. The method according to claim 1, characterized in that A pre-trained feature extraction model is used to extract features of each image in the image set, including: Based on the feature extraction model, a feature map of each image in the image set is obtained, the feature map of each image in the image set is divided into multiple regions for pooling to obtain multiple regional features of each image in the image set, and feature extraction is performed on the multiple regional features of each image in the image set to obtain features of each image in the image set.

3. The method according to claim 1, characterized in that After fusing the plurality of second parent clusters and each of the child clusters to obtain a clustering result of the camera points of the image set, the method further includes: Determine the point complexity of each camera point, wherein the point complexity is used to characterize the degree of convergence of the clustering results of the camera points; Determining whether the point complexity of each camera point is greater than a preset point complexity threshold; If the point complexity of the camera points is greater than the point complexity threshold, the clustering parameters are adjusted, and the camera points of the image set are re-clustered.

4. The method according to claim 3, characterized in that The clustering parameters are adjusted and the camera points of the image set are re-clustered, including: The similarity distance parameter based on density clustering is reduced, and the camera points of the image set are re-clustered until the number of camera points of the image set are clustered reaches a set number of times, or the point complexity of each camera point is no greater than the point complexity threshold.

5. The method according to claim 1, wherein The method further comprises: Determine the number of images required for different dimensions; Obtain images captured by cameras at different locations based on the number of images required for the different dimensions, and annotate the camera location information of all images to obtain a training set; The feature extraction model is trained using the training set.

6. The method according to claim 1, characterized in that The clustering method used in the prototype clustering is Kmeans clustering, and the clustering method used in the density clustering is DBscan clustering.

7. A method for selecting camera points, characterized in that: include: Obtaining features of each image in an image set and clustering results of camera positions in the image set, wherein the image set includes images captured by cameras at different positions; According to the features of each image in the image set and the clustering results of the camera points in the image set, the point center and point complexity of each camera point are obtained respectively; Clustering the point centers of the camera points to obtain similar camera points; Selecting camera points according to the point complexity and the similar camera points to obtain target camera points; Among them, a set of images captured by cameras at different points is obtained; a pre-trained feature extraction model is used to extract features of each image in the image set, wherein the feature extraction model is trained based on images marked with camera point information; the features are clustered into multiple first parent clustering clusters based on prototypes, and the number of images in each clustering cluster is not greater than a set value; and the features of each image in each of the first parent clustering clusters are clustered into multiple child clustering clusters based on density; the features corresponding to the center points of the features of each image in each of the child clustering clusters are determined, and the features corresponding to each of the center points are clustered into multiple second parent clustering clusters based on density; the multiple second parent clustering clusters and the child clustering clusters are fused to obtain the clustering results of the camera points of the image set.

8. The method according to claim 7, characterized in that Camera point selection is performed according to the point complexity and the similar camera points to obtain a target camera point, including: The similar camera points are sorted according to the point complexity, and the first N similar camera points are selected as the target camera points, where the value of N is determined according to the number of similar camera points and a preset selection ratio.

9. A camera point clustering device, characterized in that: include: A first acquisition module is used to acquire image sets collected by cameras at different locations; an extraction module, configured to extract features of each image in the image set using a pre-trained feature extraction model, wherein the feature extraction model is trained based on images annotated with camera point information; A first clustering module is configured to cluster the features into a plurality of first parent clusters based on prototypes, wherein the number of images in each cluster is not greater than a set value; and cluster the features of each image in each of the first parent clusters into a plurality of child clusters based on density; A second clustering module is used to determine the features corresponding to the center points of the features of each image in each of the child clusters, and cluster the features corresponding to each of the center points into multiple second parent clusters based on density; A fusion module is used to fuse the multiple second parent clusters and each of the child clusters to obtain a clustering result of the camera points of the image set.

10. A camera point selection device, characterized in that: include: a second acquisition module, configured to acquire features of each image in an image set and clustering results of camera positions in the image set, wherein the image set includes images captured by cameras at different positions; A third acquisition module is used to obtain the point center and point complexity of each camera point according to the characteristics of each image in the image set and the clustering result of the camera points in the image set; A third clustering module is used to cluster the point centers of the camera points to obtain similar camera points; A selection module, configured to select camera points according to the point complexity and the similar camera points to obtain a target camera point; The device is used to obtain a set of images collected by cameras at different points; use a pre-trained feature extraction model to extract features of each image in the image set, wherein the feature extraction model is trained based on images marked with camera point information; cluster the features into multiple first parent clustering clusters based on prototypes, with the number of images in each clustering cluster not exceeding a set value; and cluster the features of each image in each of the first parent clustering clusters into multiple child clustering clusters based on density; determine the features corresponding to the center points of the features of each image in each of the child clustering clusters, and cluster the features corresponding to each of the center points into multiple second parent clustering clusters based on density; and fuse the multiple second parent clustering clusters and the each of the child clustering clusters to obtain the clustering results of the camera points of the image set.

11. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the steps of the method according to any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the steps of the method according to any one of claims 1 to 8.

Images