A person image clustering method and device

Abstract

The application provides a person image clustering method and device, relates to the technical field of image clustering, and is used for improving the accuracy of person image clustering. In the person image clustering method, the self-organizing mapping network is used to process the multi-aspect attribute features of the lens, so as to determine the winning neuron corresponding to the lens, and the winning neurons to which the multiple lenses respectively belong are used to determine the cluster to which the multiple lenses belong, so that a spatial domain clustering manner for person images is provided. Since the clustering results of the multiple lenses are not affected by the quality of the person images, the reliability of the lens clustering is improved, and the reliability of the person image clustering is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image clustering, and particularly relates to a person image clustering method and device. BACKGROUND

[0002] Person image clustering is a process of classifying person images into different clusters (or classes). Person images in the same cluster have great similarity, while person images in different clusters have great dissimilarity. Clustering a large number of person images can provide data basis for later image retrieval and a large number of sample data for later training of a model for recognizing person images.

[0003] At present, a commonly used person image clustering method focuses on features of person images. In the person image clustering method, features of a large number of person images are extracted, and person images with similar features are classified into a cluster. However, in the person image clustering method, once the quality of a person image is poor, the clustering result of the person image is not reliable. SUMMARY

[0004] Embodiments of the present application provide a person image clustering method and device, which are used to improve the reliability of clustered person images.

[0005] In a first aspect, a person image clustering method is provided, including: obtaining a plurality of card slots each having a card slot feature, wherein the card slot feature of one card slot is used to describe a position of the one card slot itself and a position relationship between the one card slot and other card slots except the one card slot in the plurality of card slots; determining a corresponding winning neuron of each of the plurality of card slots in a self-organizing mapping network based on the obtained plurality of card slot features, wherein the corresponding winning neuron of one card slot is a neuron in a plurality of neurons contained in the self-organizing mapping network and having a maximum similarity with the card slot feature of the one card slot; determining a cluster to which each of the plurality of card slots belongs based on the corresponding winning neuron of each of the plurality of card slots; and clustering person images corresponding to card slots belonging to the same cluster within a preset time period to obtain a clustering result.

[0006] In this embodiment, the checkpoint features of multiple checkpoints can be extracted, and a self-organizing map network (SAMR) is used to process these features to determine the dominant neurons corresponding to each checkpoint. Then, based on these dominant neurons, the clusters corresponding to the checkpoints are determined. This provides a method for clustering checkpoints using a SAMR. Since the quality of the images of people does not need to be considered when clustering checkpoints, meaning the image quality does not affect the clustering effect, and the location of the checkpoints and the positional relationship between two checkpoints are relatively easy to collect and do not change, the reliability of the checkpoint clustering results is higher. Furthermore, the SAMR clustering process used in this embodiment results in high clustering efficiency. Moreover, images of people captured by cameras at the same checkpoint inherently share certain commonalities, such as the same capture location and similar camera parameters. Therefore, using the checkpoint clustering results as the image clustering results in this embodiment can also improve the accuracy of people image clustering.

[0007] In one possible implementation, obtaining the checkpoint features of multiple checkpoints includes: acquiring checkpoint data of each of the multiple checkpoints, the checkpoint data including checkpoint location data and path data between the checkpoint and other checkpoints among the multiple checkpoints; and performing the following steps on the acquired multiple checkpoint data respectively: performing principal component analysis on the checkpoint data of one of the multiple checkpoints to obtain the checkpoint features of the one checkpoint.

[0008] In the above implementation method, principal component analysis can be used to process the checkpoint data to obtain checkpoint features, which helps to reduce the data dimensionality of the checkpoint features and accelerate the efficiency of subsequent clustering.

[0009] In one possible implementation, acquiring the checkpoint data of each of the plurality of checkpoints includes: receiving the location data of each of the plurality of checkpoints; performing the following operations on each of the plurality of checkpoints: using the location of one of the checkpoints as the center and a preset radius as the search radius, searching for a walking path between the one checkpoint and each of the other checkpoints, wherein the other checkpoints refer to the checkpoints other than the one checkpoint; and determining the path data between the one checkpoint and the other checkpoints based on the searched walking paths.

[0010] In a possible implementation, the determination of the corresponding winning neuron of each of the plurality of card slots in the self-organizing mapping network based on the obtained plurality of card slot features respectively comprises: performing at least one iteration operation on the plurality of card slot features respectively, where one iteration operation comprises the following processes: determining the first similarity between the card slot feature of one of the plurality of card slots and the weight of each of the plurality of neurons respectively, and taking the neuron corresponding to the maximum first similarity as the winning neuron; determining at least one neuron within the neighborhood of the winning neuron based on the neighborhood radius; updating the weight of each of the at least one neuron based on the distance between the winning neuron and each of the at least one neuron and the learning rate respectively; updating the neighborhood radius and the learning rate; and until the self-organizing mapping network meets a preset termination condition, taking the winning neuron obtained in the last iteration operation as the corresponding winning neuron of the one card slot.

[0011] In the above implementation, the card slot feature can be iteratively processed until the self-organizing mapping network meets the preset termination condition, so that the determined winning neuron is more accurate.

[0012] In a possible implementation, the updating of the neighborhood radius and the learning rate comprises: updating the learning rate of the self-organizing mapping network at a preset first rate; and updating the neighborhood radius of the self-organizing mapping network at a preset second rate; and if the updated learning rate is less than or equal to a preset learning rate and / or the number of iteration operations on the card slot feature of the one card slot meets a preset number, it is determined that the self-organizing mapping network meets the preset termination condition.

[0013] In the above implementation, a plurality of possible preset termination conditions are provided, which enriches the preset termination condition and makes the determined winning neuron more accurate.

[0014] In a possible implementation, the determination of the cluster to which each of the plurality of card slots belongs based on the corresponding winning neuron of each of the plurality of card slots comprises: attributing the card slots corresponding to the same winning neuron to the same cluster.

[0015] In the above implementation, the card slots belonging to the same winning neuron can be directly attributed to the same cluster, and the way of determining the cluster to which the card slot belongs is more simple and direct.

[0016] In a possible implementation, the bayonet holes corresponding to the same winning neuron are attributed to the same cluster, including: if the total number of the bayonet holes corresponding to the same winning neuron is greater than or equal to a first preset number and less than or equal to a second preset number, the bayonet holes corresponding to the same winning neuron are attributed to the same cluster, where the first preset number is less than the second preset number.

[0017] In the implementation, the bayonet holes corresponding to the same winning neuron are attributed to the same cluster only when the total number of the bayonet holes corresponding to the same winning neuron meets a certain condition, so that the number of the bayonet holes belonging to the same cluster is more reasonable, and the number of the character images in the same cluster is prevented from being too large or too small.

[0018] In a possible implementation, the bayonet holes corresponding to the same winning neuron are attributed to the same cluster, including: if the total number of the bayonet holes corresponding to the same winning neuron is greater than or equal to a first preset number and less than or equal to a second preset number, the bayonet holes corresponding to the same winning neuron are attributed to the same cluster, where the first preset number is less than the second preset number.

[0019] In the implementation, if the total number of the bayonet holes corresponding to the same winning neuron is too small, the bayonet holes corresponding to adjacent winning neurons are combined, and if the total number of the bayonet holes corresponding to the same winning neuron is too large, the bayonet holes corresponding to adjacent winning neurons are split, so that the number of the bayonet holes in the same cluster is prevented from being too large or too small.

[0020] In a second aspect, a character image clustering apparatus is provided, including: an obtaining module, configured to obtain a plurality of bayonet hole features respectively corresponding to a plurality of bayonet holes, where a bayonet hole feature of one bayonet hole is used to describe a position of the one bayonet hole itself and a position relationship between the one bayonet hole and other bayonet holes except the one bayonet hole in the plurality of bayonet holes; a determining module, configured to determine a winning neuron corresponding to each of the plurality of bayonet holes in a self-organizing mapping network based on the obtained plurality of bayonet hole features respectively, where the winning neuron corresponding to one bayonet hole is a neuron in the self-organizing mapping network having the largest similarity with the bayonet hole feature of the one bayonet hole, and to determine a cluster to which each of the plurality of bayonet holes belongs based on the winning neuron corresponding to each of the plurality of bayonet holes respectively; and a clustering module, configured to cluster character images corresponding to the bayonet holes belonging to the same cluster within a preset time period to obtain a clustering result.

[0021] In a possible implementation, the obtaining module is specifically configured to: acquire the bay data of the plurality of bays respectively, the bay data comprising position data of a bay and path data between the bay and other bays in the plurality of bays; and perform the following steps on the plurality of acquired bay data respectively: performing principal component analysis on the bay data of one bay in the plurality of bays to obtain a bay feature of the one bay.

[0022] In a possible implementation, the obtaining module is specifically configured to: receive position data of the plurality of bays respectively; and perform the following operations on the plurality of bays respectively: taking a position of one bay in the plurality of bays as a center, taking a preset radius as a search radius, searching for a walking path between the one bay and each of other bays, the other bays referring to the plurality of bays except the one bay; and determining path data between the one bay and the other bays respectively based on the searched walking paths.

[0023] In a possible implementation, the determining module is specifically configured to: perform at least one iteration operation on the plurality of bay features respectively, wherein one iteration operation comprises the following processes: determining a first similarity between the bay feature of one bay in the plurality of bays and the weight of each neuron respectively, taking a neuron corresponding to a first similarity with a maximum value as a winning neuron; determining at least one neuron within a neighborhood of the winning neuron based on a neighborhood radius; updating the weight of the at least one neuron respectively based on a distance between the winning neuron and the at least one neuron respectively and a learning rate; updating the neighborhood radius and the learning rate; and taking the winning neuron obtained in the last iteration operation as the winning neuron corresponding to the one bay until the self-organizing mapping network meets a preset termination condition.

[0024] In a possible implementation, the determining module is specifically configured to: update the learning rate of the self-organizing mapping network at a preset first rate; and update the neighborhood radius of the self-organizing mapping network at a preset second rate; and determine that the self-organizing mapping network meets the preset termination condition if the updated learning rate is less than or equal to a preset learning rate and / or a number of times of performing the iteration operation on the bay feature of the one bay meets a preset number of times.

[0025] In a possible implementation, the clustering module is specifically configured to: attribute bays corresponding to a same winning neuron to a same cluster.

[0026] In a possible implementation, the clustering module is specifically configured to: if the total number of the bayonet sockets corresponding to the same winning neuron is greater than or equal to a first preset number and less than or equal to a second preset number, attribute the bayonet sockets corresponding to the same winning neuron to the same cluster, where the first preset number is less than the second preset number.

[0027] In a possible implementation, the clustering module is specifically configured to: if the total number of the bayonet sockets corresponding to the same winning neuron is less than the first preset number, attribute the bayonet sockets corresponding to the same winning neuron and the bayonet sockets corresponding to the adjacent winning neuron of the same winning neuron to the same cluster; and if the total number of the bayonet sockets corresponding to the same winning neuron is greater than the second preset number, divide and attribute the bayonet sockets corresponding to the same winning neuron to at least two clusters, where the total number of the bayonet sockets included in each cluster of the at least two clusters is greater than or equal to the first preset number and less than or equal to the second preset number.

[0028] In a third aspect, a person image clustering device is provided, including:

[0029] at least one processor, and

[0030] a memory connected with the at least one processor in communication;

[0031] The memory stores instructions executable by the at least one processor, and the at least one processor implements the method in any one of the first aspect and possible implementation manners thereof by executing the instructions stored in the memory.

[0032] In a fourth aspect, a computer readable storage medium is provided, which stores computer instructions, and when the computer instructions are run on a computer, the computer executes the method in any one of the first aspect and possible implementation manners thereof.

[0033] The beneficial effects of the above-mentioned second aspect to fourth aspect can refer to the beneficial effects discussed in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 An application scenario diagram suitable for the embodiments of the present application is provided;

[0035] Figure 2 A flowchart of a person image clustering method provided by the embodiments of the present application is provided;

[0036] Figure 3 A structure diagram of a self-organizing mapping network provided by the embodiments of the present application is provided;

[0037] Figure 4 The result output by the self-organizing network provided in the embodiment of the present application is shown in FIG. 3. Figure 3 The result output by the self-organizing network provided in the embodiment of the present application is shown in FIG. 3. Figure 1

[0038] Figure 5 The result output by the self-organizing network provided in the embodiment of the present application is shown in FIG. 3. Figure 3 The result output by the self-organizing network provided in the embodiment of the present application is shown in FIG. 3. Figure 2

[0039] Figure 6 The structure diagram of a person image clustering device provided in the embodiment of the present application is shown in FIG. 1.

[0040] Figure 7 The structure diagram of a person image clustering device provided in the embodiment of the present application is shown in FIG. 1. DETAILED DESCRIPTION

[0041] In order to better understand the technical solutions provided in the embodiments of the present application, the following will be described in detail in combination with the accompanying drawings and specific embodiments.

[0042] In order to better understand the technical terms related to the embodiments of the present application, the technical terms related to the embodiments of the present application will be introduced as follows.

[0043] 1. In the embodiment of the present application, the person image clustering is the process of grouping multiple person images to form multiple sets. The person image includes a face image or a body image.

[0044] 2. In the embodiment of the present application, the time-space domain division includes time and space, and the time-space domain is used to represent the subject world dimension of personnel existence and activity. A time-space domain is composed of a group of continuous time points and space points. Both time and space can be used as the main carrier of personnel appearing in a certain time-space domain. The space is the geographical interval of the person's activity in a continuous time. In the embodiment of the present application, the space point is discretized into a specific bayonet. The time is the time span representing the person's activity in a certain space range.

[0045] 3. In the embodiment of the present application, the cluster or class can represent the collection of person images in a certain time-space range in a certain time-space domain.

[0046] 4. In the embodiment of the present application, the topology mapping is a discipline that studies some properties of geometric figures or spaces that remain unchanged after continuous shape changes. It only considers the positional relationship between objects and does not consider their shape and size. Topology mapping refers to projecting and mapping various attribute features that determine the division of the space domain into points on a two-dimensional plane.

[0047] ​​5、The bayonet in the embodiment of the present application is used to erect the camera. The bayonet is generally installed at a fixed position, and the camera can be erected on the bayonet. The camera corresponding to the bayonet in the embodiment of the present application refers to the camera installed on the bayonet.

[0048] It should be noted that "multiple" in the embodiments of the present application refers to two or more than two, and "A and / or B" includes A, B, and the three cases of A and B.

[0049] In order to improve the reliability of the person image clustering, the embodiment of the present application provides a person image clustering method. In the person image clustering method, a self-organizing map (SOM) can be used to cluster multiple bays based on the bayonet features of the multiple bays to obtain clusters to which the multiple bays respectively belong, one bay corresponds to at least one camera, and the person image is collected by each camera, so clustering the multiple bays is equivalent to clustering the person image in the spatial domain. The person images corresponding to the bays belonging to one cluster can be clustered. Compared with the prior art which only uses image features for clustering, since the bay clustering is not affected by the quality of the person image, the way of clustering the person image based on the bay in the embodiment of the present application can reduce the influence of the person image on the clustering result, and further improve the reliability of the person image clustering. Moreover, the SOM can be used to cluster the multiple bays in the embodiment of the present application. The SOM can perform multiple iteration operation processing on the bayonet features of the multiple bays, which can improve the accuracy and efficiency of clustering the multiple bays. Moreover, the bayonet features in the embodiment of the present application are the positions of the bays and the positional relationship between the multiple bays. These features are easy to collect and not easy to change, so they are conducive to more accurate clustering of the bays.

[0050] The person image clustering method in the embodiment of the present application can be executed by a person image clustering device. The person image clustering device can be a device with computing function, for example, a server, a server cluster, or a personal computer, etc. The present application does not limit this. The server can be a physical server or a virtual server.

[0051] The deployment scenario of the person image clustering device will be described below with reference to the application scenario schematic diagram shown in Figure 1 Or, Figure 1 It can also be understood as the deployment schematic diagram of the person image clustering device in the embodiment of the present application.

[0052] As Figure 1 shown, the scenario schematic diagram includes a person image clustering device 110, multiple bays, and a camera 130 installed on each bay. The number of the multiple bays is greater than or equal to 2, Figure 1 The multiple bays include, for example, Figure 1The first, second, third and fourth eyelets 121, 122, 123 and 124 in the first embodiment are taken as examples for introduction.

[0053] It should be noted that one eyelet can be installed with one or more cameras 130, Figure 1 The first embodiment takes the case of installing one camera 130 on one eyelet as an example. Figure 1 The person image clustering device 110 in the first embodiment takes the case of taking the person image clustering device 110 as a server as an example, but does not limit the specific implementation manner of the person image clustering device 110.

[0054] The person image clustering device 110 can obtain eyelet data of eyelets from other devices, the eyelet data including position data of the eyelets and path data between the eyelets and other eyelets in the plurality of eyelets. The other devices refer to devices other than the person image clustering device 110. The plurality of eyelets are clustered according to the eyelet data, and then the person images collected by the cameras installed on the eyelets are clustered according to the clustering results of the plurality of eyelets. The specific process of clustering will be introduced below. For example, the person image clustering device 110 determines that the first eyelet 121 and the second eyelet 122 belong to the same cluster, and then the person images collected by the cameras corresponding to the first eyelet 121 and the second eyelet 122 in a preset time period can be aggregated. Similarly, if the third eyelet 123 and the fourth eyelet 124 belong to the same cluster, the person images collected by the cameras corresponding to the third eyelet 123 and the fourth eyelet 124 in a preset time period can be aggregated.

[0055] Please refer to Figure 2 The flowchart of a person image clustering method provided by the embodiment of the present application is shown in FIG. 2. It should be noted that, Figure 2 The person image clustering method in the embodiment of the present application is introduced taking the case of taking the person image clustering device as a server as an example.

[0056] In step 21, the person image clustering device 110 obtains the eyelet features of the plurality of eyelets.

[0057] Each eyelet corresponds to an eyelet feature, and the eyelet feature is used to describe the position of the eyelet itself and the position relationship between the eyelet and other eyelets. The other eyelets refer to the eyelets other than the eyelet in the plurality of eyelets.

[0058] There can be various ways for the server to obtain the plurality of eyelet features, and the case of obtaining the eyelet feature of one eyelet by the server is taken as an example for introduction.

[0059] The first way of obtaining the eyelet feature of one eyelet is as follows.

[0060] In step 1.1, the eyelet data of one eyelet is obtained.

[0061] The kiosk data includes location data of the kiosk and path data between the kiosk and other kiosks. The meanings of the respective data and the obtaining manners are exemplarily introduced below.

[0062] 1. The location data of the kiosk is used to represent the location where the kiosk is located. For example, the location data is longitude and latitude data of the kiosk.

[0063] Exemplarily, the server can connect to a maintenance platform corresponding to the kiosk, and obtain the location data of the kiosk from the maintenance platform.

[0064] Optionally, the location data further includes road data, inner and outer ring data, commercial district data, point of interest (POI) data and area of interest (AOI) data where the kiosk is located. In a geographic information system, a POI can be a house, a shop, a mailbox or a bus stop, etc. The AOI can be understood as a larger area where the POI is located, and can be understood as a block of position on a map. For example, the server can perform reverse geocoding on an electronic map, and thus obtain one or more of the road data, the inner and outer ring data, the commercial district data, the POI data and the AOI data where the kiosk is located.

[0065] 2. The path data between the kiosk and other kiosks includes path data between the kiosk and each of the other kiosks. For example, the path data includes one or more of shortest path distance, longest path distance, average path distance and reachable path quantity between two kiosks.

[0066] Exemplarily, the server can perform reverse geocoding on an electronic map, and take the location where the kiosk is located as the center and take a preset radius as the search radius, so as to crawl a walking path between the kiosk and any of the other kiosks. Based on the crawled walking path, the path data between the kiosk and the other kiosks is obtained.

[0067] In the embodiments of the present application, the path data between two kiosks is taken as a part of the kiosk data, and the path data can actually conform to human activities, so that the collected kiosk features are consistent with the human activity attributes, which is beneficial to more accurate clustering of the human images collected by the camera corresponding to the kiosk.

[0068] As an embodiment, the server can directly take the data obtained in step 1.1 as the kiosk features of the kiosk.

[0069] The second way of obtaining the kiosk features of a kiosk is specifically as follows.

[0070] The server can obtain the port feature of the port from other devices. The other devices are devices other than the server. The other devices can obtain the port feature of the port in the same way as the server obtains the port feature of the port, which will not be described here.

[0071] Similarly, the server can obtain a plurality of port features, one port feature corresponding to one port.

[0072] Step 1.2. Principal component analysis is performed on the port data of the port to obtain the port feature of the port.

[0073] To reduce the data processing dimension, in an embodiment of the present application, the path data between the port and other ports can be added, specifically, the path data of the same item of each item of the port and other ports is added, thereby reducing the processing dimension of the port data of the port. Of course, the server can also not process the path data obtained in step 1.1, which is not limited in the present application.

[0074] For example, the path data of the first port includes the shortest path and the number of reachable paths. The shortest path distance between the first port and the second port is 1 km, and the number of reachable paths between the first port and the second port is 3; the shortest path distance between the first port and the third port is 2 km, and the number of reachable paths between the first port and the third port is 4; the server can sum the shortest path distance between the first port and the second port and the shortest path distance between the first port and the third port, and the server can sum the number of reachable paths between the first port and the second port and the number of reachable paths between the first port and the third port.

[0075] The server can perform principal component analysis on the port data, or can add the port data and then perform principal component analysis on the added data. Principal component analysis can use orthogonal transformation to linearly transform the observed values of a series of possibly related variables, thereby projecting them into a series of linearly uncorrelated variables, which are called principal components. After principal component analysis, new features formed by linear combination of the port data can be obtained to reduce the data dimension. Principal component analysis can eliminate irrelevant components in part of the features, and can reduce the time complexity of algorithm operation and reduce the calculation cost.

[0076] Step 22, respectively based on the obtained plurality of port features, determine the corresponding winning neurons of the plurality of ports in the self-organizing mapping network.

[0077] In this embodiment, the server can use a Self-Organizing Map (SOM) network to cluster multiple checkpoints. For example, the server can input the features of multiple checkpoints into the SOM network to obtain the winning neuron for each checkpoint in the self-organizing map network. The winning neuron for a checkpoint is the neuron in the SOM network with the highest similarity to the checkpoint features of that checkpoint.

[0078] To illustrate the clustering process more clearly, the network structure of the SOM network is first described as an example. The SOM network can be a hierarchical structure, including an input layer and an output layer, the output layer also known as the competition layer. The output layer contains multiple neurons that can perform topological mapping, thereby outputting a plane. In this embodiment, rows in this plane are labeled with p. r This indicates that the columns of the plane are p c Indicate. For example, with Figure 3 Taking the schematic diagram of the self-organizing map network shown as an example, this SOM network includes an input layer 301 and an output layer 302. The output layer includes p... r *p c The SOM network has 10 neurons. The model parameters include the neighborhood radius R, the rate of reduction of the neighborhood radius α, the learning rate η, the rate of reduction of the learning rate γ, and the maximum number of iterations.

[0079] The method for determining the winning neuron for each of the multiple checkpoints is the same. The following is an example of determining the winning neuron for a single checkpoint.

[0080] Step 2.1: The server performs at least one iteration operation on the checkpoint features, wherein one iteration operation includes the processes shown in steps 2.1.1 to 2.1.4.

[0081] Step 2.1.1: Determine the first similarity between the checkpoint features and the weights of the multiple neurons, and select the neuron with the highest first similarity value as the winning neuron.

[0082] like Figure 3 As shown, the output layer of this example has a total of 18 neurons. Figure 3 In the example of 18 neurons (the actual number of neurons in the output layer is not limited), Pr is 3, Pc is 6, and the input layer has a total of D input neurons, where D can also represent the number of input features of the checkpoint.

[0083] N = Pr × Pc × D (1)

[0084] Each neuron has D connection weights, which can be represented as a weight vector W. Ci = [w1, w2, w3, ..., wD], where Ci represents the i-th neuron. Therefore, N represents the total number of connection weights in the SOM network.

[0085] Each socket feature is compared with all layer neurons for similarity, as an example, the Euclidean distance between the socket feature and the neuron can be taken as the similarity between the socket and the SOM network.

[0086] The formula for calculating the Euclidean distance is as follows.

[0087]

[0088] Where m represents the actual number of open portrait clusters of the socket, X j (j = 1, 2, 3, …, m represents the socket feature of the socket, xk represents the kth variable (dimension) in X j i (i = 1, 2, 3, …, Pr x Pc) represents the i-th neuron, wk represents the weight parameter W i ～ of the neuron, and the kth variable (dimension) in W

[0089] In a possible embodiment, the weight vector of each neuron in the SOM network can be directly taken as the weight parameter of each neuron. Alternatively, in order to reduce the amount of calculation, the weight vector of each neuron in the SOM network can also be normalized, and the result after normalization can be taken as the weight parameter of each neuron. The formula for normalization can be shown as follows.

[0090]

[0091] Where W i ～ represents the weight parameter in the SOM network, Wi represents the weight vector of the i-th neuron in the output layer, and || Wi || represents the length of Wi.

[0092] By analogy, the server can determine the similarity between the socket feature of the socket and multiple neurons in the SOM network, respectively, so as to obtain multiple similarities, which are referred to as multiple first similarities for the sake of distinction. The server can determine the maximum value of the multiple first similarities, and take the neuron corresponding to the maximum first similarity as the winning neuron.

[0093] If the SOM network meets the preset termination condition at this time, the determined winning neuron can be taken as the winning neuron corresponding to the socket. If the SOM network does not meet the preset termination condition, the steps of S2.1.2 are executed.

[0094] Step 2.1.2, based on the neighborhood radius, at least one neuron within the neighborhood of the winning neuron is determined.

[0095] ​The server can determine at least one neuron within the neighborhood of the winning neuron based on the neighborhood radius R. Continuing with the example shown in FIG. 2A, neuron A is the winning neuron corresponding to the card slot, R is 1, and the server determines that neurons B, C, D, F, and G are at least one neuron within the neighborhood of the winning neuron A. Figure 3 In the example shown in FIG. 2A, neuron A is the winning neuron corresponding to the card slot, R is 1, and the server determines that neurons B, C, D, F, and G are at least one neuron within the neighborhood of the winning neuron A. Figure 3 In the example shown in FIG. 2A, neuron A is the winning neuron corresponding to the card slot, R is 1, and the server determines that neurons B, C, D, F, and G are at least one neuron within the neighborhood of the winning neuron A.

[0096] Step 2.1.3. Update the weight parameter of each of the at least one neuron based on the distance between the winning neuron and each of the at least one neuron and the neighborhood radius.

[0097] The server can update the weight parameter of each of the at least one neuron based on the distance between the winning neuron and each of the at least one neuron and the neighborhood radius. An example of a formula for updating the weight parameter is shown below.

[0098]

[0099] where W k represents the current weight parameter of one of the at least one neuron; X k represents the card slot feature of the card slot; η represents the learning rate of the neuron; R represents the neighborhood radius; and dist represents the distance between one of the at least one neuron and the winning neuron determined in step 2.1.1.

[0100] Optionally, the distance between one of the at least one neuron and the winning neuron determined in step 2.1.1 can be represented by the Euclidean distance, which is not limited in the embodiments of the present application.

[0101] By analogy, the server can update the weight parameter of each of the at least one neuron according to the formula (4) above to obtain the updated at least one neuron.

[0102] Step 2.1.4. Update the neighborhood radius and the learning rate.

[0103] After the server updates the weight parameter of the at least one neuron according to step 2.1.3, the server can update the neighborhood radius and the learning rate. For example, the server can update the learning rate at a preset first rate. An example of a formula for updating the learning rate is shown below.

[0104] η k+1 = η k × α (5)

[0105] where η k represents the current value of the learning rate of the SOM network; and η k+1represents the updated value of the current learning rate of the SOM network; and a represents the first rate.

[0106] Similarly, the server can update the neighborhood radius of the SOM network according to a preset second rate. The first rate and the second rate can have the same value or different values.

[0107] R k+1 = R k × γ (5)

[0108] R k represents the value of the current neighborhood radius of the SOM network; and R k+1 represents the updated value of the current neighborhood radius of the SOM network; and γ represents the second rate.

[0109] Optionally, the first rate and the second rate can have the same value or different values. The first rate and the second rate have values greater than 0 and less than 1.

[0110] Step 2.2, until the SOM network satisfies a preset termination condition, the winning neuron obtained in the last iteration operation is taken as the winning neuron corresponding to the current card slot.

[0111] After step 2.1.4, if it is determined that the SOM network satisfies the preset termination condition, the winning neuron obtained in step 2.1.1 is taken as the winning neuron corresponding to the current card slot. If the SOM network does not satisfy the preset termination condition, the neuron updated in step 2.1.3 is used to continue the process of steps 2.1.1 to 2.1.4 until the SOM network satisfies the preset termination condition, and the winning neuron obtained in the last iteration operation is taken as the winning neuron corresponding to the current card slot.

[0112] For example, the preset termination condition is understood as that the updated learning rate is less than or equal to a preset learning rate, or the number of iterations of the card slot feature of the current card slot satisfies a preset number of times, or the updated learning rate is less than or equal to the preset learning rate and it is determined that the number of iterations of the card slot feature of the current card slot satisfies the preset number of times. The content of the preset termination condition is not limited in the embodiments of the present application.

[0113] In a possible embodiment, the server can save the position index of each winning neuron in the SOM network, and subsequently the server can determine which winning neurons correspond to which card slots according to the position index. Continue to refer to the example of Figure 3 For example, the server saves the position index of card slot 2, card slot 3 and card slot 4 as (0, 0), and the server can determine that card slot 2, card slot 3 and card slot 4 all correspond to the first neuron in the first row and the first column.

[0114] Step 23, based on the plurality of sockets each corresponding to a winning neuron, respectively determine the clusters to which the plurality of sockets belong.

[0115] By analogy, the server can determine the winning neuron corresponding to each of the plurality of sockets. Further, the server can determine the clusters (or also known as classes) to which the plurality of sockets belong based on the winning neuron of each of the plurality of sockets.

[0116] In a possible embodiment, the server can attribute the sockets corresponding to the same winning neuron to a cluster, thereby determining the cluster corresponding to each of the plurality of sockets. This way of determining the cluster corresponding to the socket is direct and simple.

[0117] For example, continuing with the example shown in Figure 3 , the server can regard the non-zero neurons in the matrix as a cluster respectively, and divide the sockets corresponding to the same neuron into a cluster.

[0118] For example, a schematic diagram of the result output by a SOM network is shown in, for example Figure 4 . The SOM network outputs a two-dimensional matrix. In Figure 4 , each position in the matrix corresponds to a socket, and the value at each position in the matrix represents the number of sockets whose winning neuron is the neuron at this position. For example, taking the first number "3" in the matrix, the first neuron in the first SOM network is the winning neuron of 3 sockets, thereby determining that the 3 sockets belong to the same cluster. In the matrix, "0" represents a dead node (the node can correspond to a neuron in the SOM network), indicating that the neuron does not correspond to any socket.

[0119] In a possible embodiment, if the server determines that the total number of sockets corresponding to the same winning neuron is greater than or equal to a first preset number and less than or equal to a second preset number, the first preset number is less than the second preset number, the server attributes the sockets corresponding to the same winning neuron to the same cluster. If the server determines that the total number of sockets corresponding to the same winning neuron is less than the first preset number, the server attributes the sockets corresponding to the same winning neuron and the sockets corresponding to the adjacent winning neuron of the same winning neuron to the same cluster. If the server determines that the total number of sockets corresponding to the same winning neuron is greater than the second preset number, the server divides the sockets corresponding to the same winning neuron into at least two clusters, each of the at least two clusters includes a total number of sockets greater than or equal to the first preset number and less than or equal to the second preset number.

[0120] In the embodiment of the present application, the server does not directly attribute the card holes belonging to the same winning neuron to the same cluster, but flexibly divides the clusters to which the card holes belong according to the number of card holes belonging to the same winning neuron, the first preset number and the second preset number, so as to avoid too many or too few card holes in the same cluster, which is not conducive to subsequent clustering of the person images.

[0121] For example, the result output by a SOM network is as shown in the following table. Figure 5 Figure 5 The following table shows the grid output result of the SOM network output containing 801 card holes. Figure 5 It can be seen from the matrix shown in the following table that there are many output nodes containing only one card hole, and these nodes should be merged with adjacent nodes. According to experience, the first preset number is set to 3, and the second preset number is set to 20. The total number of card holes contained in a node is lower than the first preset number, and the card holes contained in a node higher than the second preset number are split, so as to ensure that the number of card holes belonging to the same cluster is greater than or equal to the first preset number and less than or equal to the second preset number.

[0122] In this way, the server is equivalent to clustering the plurality of card holes. Since different card holes are distributed in different regions, the above process is equivalent to clustering the plurality of card holes in the spatial domain.

[0123] It should be noted that in actual operation, the server can directly input the card hole features of the plurality of card holes into the self-organizing mapping network, and the self-organizing mapping network can output the winning neurons corresponding to the plurality of card holes respectively, which is equivalent to obtaining the clusters corresponding to the plurality of card holes respectively.

[0124] In step 24, the person images corresponding to the card holes belonging to the same cluster in a preset time period are clustered to obtain a clustering result.

[0125] The server can collect the person images of the same preset time period according to the time stamps on the person images, so as to collect the person images of the same preset time period belonging to the card holes of the cluster at one time, thereby realizing preliminary clustering of the person images, which is equivalent to clustering the person images based on the space-time domain. The time stamp on the person image refers to the time when the person image is taken.

[0126] For example, assuming that the preset time period is from 9:00 on July 21, 2021 to 15:00 on July 21, 2021, and a certain spatial domain contains three card holes (card holes 7, 8 and 12), then all the snapshot person images of these three card holes between 9:00 on July 21, 2021 and 15:00 on July 21, 2021 belong to the same space-time domain.

[0127] ​Further, the server can further cluster the person images corresponding to the same preset time period of the cameras belonging to the same cluster according to the features of the person images.

[0128] In a possible implementation, the server can determine a clustering evaluation index of each spatio-temporal field, and re-cluster the person images of the spatio-temporal field if it is determined that the clustering evaluation index does not satisfy a preset evaluation index.

[0129] Based on the same inventive concept, the embodiments of the present application provide a person image clustering device, which can be used to implement the functions of the person image clustering device described above or the functions of the server described above. Please refer to the structural schematic diagram of the device shown in Figure 6 The device includes an obtaining module 601, a determining module 602, and a clustering module 603. The functions of each module are introduced as follows.

[0130] The obtaining module 601 is configured to obtain camera features of the plurality of cameras, wherein the camera feature of one camera is used to describe the position of the camera itself and the positional relationship between the camera and other cameras in the plurality of cameras except for the one camera.

[0131] The determining module 602 is configured to determine, respectively based on the obtained camera features of the plurality of cameras, corresponding winning neurons of the plurality of cameras in the self-organizing mapping network, wherein the corresponding winning neuron of one camera is a neuron in the plurality of neurons of the self-organizing mapping network that has the maximum similarity with the camera feature of the one camera, and determine, respectively based on the corresponding winning neurons of the plurality of cameras, clusters to which the plurality of cameras belong.

[0132] The clustering module 603 is configured to cluster person images corresponding to the cameras belonging to the same cluster within a preset time period, to obtain clustering results.

[0133] In a possible implementation, the obtaining module 601 is specifically configured to: acquire camera data of the plurality of cameras respectively, wherein the camera data includes position data of the cameras and path data between the cameras and other cameras in the plurality of cameras; and perform the following steps on the acquired camera data of the plurality of cameras respectively: perform principal component analysis on the camera data of one camera in the plurality of cameras to obtain the camera feature of the one camera.

[0134] In a possible implementation, the obtaining module 601 is specifically configured to: receive position data of the plurality of cameras respectively; perform the following operations on the plurality of cameras respectively: take the position of one camera in the plurality of cameras as the center, take a preset radius as the search radius, search for a walking path between the one camera and each of the other cameras, wherein the other cameras refer to the cameras in the plurality of cameras except for the one camera; and determine the path data between the one camera and the other cameras respectively based on the searched walking path.

[0135] In a possible implementation, the determining module 602 is specifically configured to: perform at least one iteration operation on the plurality of bay features respectively, where one iteration operation includes the following processes: determining a first similarity between the bay feature of one bay in the plurality of bays and the respective weight of each neuron, taking the neuron corresponding to the maximum first similarity as a winning neuron; determining at least one neuron within a neighborhood of the winning neuron based on a neighborhood radius; updating the respective weight of each of the at least one neuron based on the respective distance between the winning neuron and each of the at least one neuron and a learning rate; updating the neighborhood radius and the learning rate; and taking the winning neuron obtained in the last iteration operation as the winning neuron corresponding to the one bay until the self-organizing mapping network meets a preset termination condition.

[0136] In a possible implementation, the determining module 602 is specifically configured to: update the learning rate of the self-organizing mapping network according to a preset first descending rate; and update the neighborhood radius of the self-organizing mapping network according to a preset second descending rate.

[0137] If the updated learning rate is less than or equal to a preset learning rate, and / or the number of times of performing the iteration operation on the bay feature of one bay meets a preset number of times, it is determined that the self-organizing mapping network meets the preset termination condition.

[0138] In a possible implementation, the clustering module 603 is specifically configured to: attribute the bays corresponding to the same winning neuron to the same cluster.

[0139] In a possible implementation, the clustering module 603 is specifically configured to: if the total number of the bays corresponding to the same winning neuron is greater than or equal to a first preset number and less than or equal to a second preset number, attribute the bays corresponding to the same winning neuron to the same cluster, where the first preset number is less than the second preset number.

[0140] In a possible implementation, the clustering module 603 is specifically configured to: if the total number of the bays corresponding to the same winning neuron is less than the first preset number, attribute the bays corresponding to the same winning neuron and the bays corresponding to the adjacent winning neurons of the same winning neuron to the same cluster; and if the total number of the bays corresponding to the same winning neuron is greater than the second preset number, divide and attribute the bays corresponding to the same winning neuron to at least two clusters, where each cluster in the at least two clusters includes a total number of bays greater than or equal to the first preset number and less than or equal to the second preset number.

[0141] Based on the same inventive concept, the embodiment of the present application provides a person image clustering device, which refers to Figure 7 The computer device includes a processor 701 and a memory 702.

[0142] The processor 701 can be a central processing unit (CPU), or a digital processing unit, etc. The specific connection medium between the memory 702 and the processor 701 is not limited in the embodiments of the present application. In the embodiments of the present application, the memory 702 and the processor 701 are connected through a bus 703, and the bus 703 is represented by a thick line in the embodiments of the present application. The connection mode between other components is only schematically illustrated, and is not limited. The bus 703 can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one line is used to represent the bus 703 in the embodiments of the present application, but it does not mean that there is only one bus or only one type of bus. Figure 7 Figure 7 The bus 703 can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one line is used to represent the bus 703 in the embodiments of the present application, but it does not mean that there is only one bus or only one type of bus. Figure 7

[0143] The memory 702 can be a volatile memory, for example, a random-access memory (RAM). The memory 702 can also be a non-volatile memory, for example, a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or the memory 702 can be any other medium capable of carrying or storing desired program codes in the form of instructions or data structures and capable of being accessed by a computer, but is not limited to this. The memory 702 can be a combination of the above memories.

[0144] The processor 701 is configured to execute any of the above-mentioned person image clustering methods when invoking the computer program stored in the memory 702. Further, the processor 701 can also be configured to implement the functions of the device in the embodiments of the present application, or the functions of the person image clustering device in the foregoing. Figure 6

[0145] Based on the same inventive concept, the embodiments of the present application provide a computer storage medium, which stores computer instructions, and when the computer instructions run on a computer, the computer executes any of the above-mentioned person image clustering methods.

[0146] ​​​Those skilled in the art will appreciate that embodiments of the application can be devised for a method, a system, or a computer program product. Accordingly, the present application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, and the like) embodying computer readable program code.

[0147] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in one or more of the flowchart illustrations and / or block diagrams. Figure 1 means for performing the functions specified in one or more of the flowchart illustrations and / or block diagrams.

[0148] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in one or more of the flowchart illustrations and / or block diagrams. Figure 1 means for performing the functions specified in one or more of the flowchart illustrations and / or block diagrams.

[0149] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in one or more of the flowchart illustrations and / or block diagrams. Figure 1 means for performing the functions specified in one or more of the flowchart illustrations and / or block diagrams.

[0150] While preferred embodiments of the application have been described, modifications and alterations thereto can occur to those skilled in the art upon reading the preceding description. It is intended to include all such modifications and alterations insofar as they come within the scope of the basic inventive concepts disclosed herein.

[0151] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method of clustering person images, the method comprising: The method comprises the following steps: obtaining a plurality of card features of a plurality of cards, wherein the card features of one card are used to describe the position of the one card itself and the positional relationship between the one card and other cards in the plurality of cards except the one card, wherein the card features of the one card are determined based on card data of the one card, the card data comprising position data of the one card and path data between the one card and other cards in the plurality of cards, wherein the path data between the one card and other cards is obtained based on a walking path between the one card and any of the other cards, wherein the walking path between the one card and any of the other cards is obtained by inversely geocoding an electronic map and crawling within a preset radius centered at the position of the one card; determining a corresponding winning neuron of each of the plurality of cards in a self-organizing map network based on the obtained plurality of card features, wherein the corresponding winning neuron of one card is a neuron in the plurality of neurons of the self-organizing map network that has the maximum similarity with the card features of the one card; determining a cluster to which each of the plurality of cards belongs based on the corresponding winning neuron of each of the plurality of cards; 2. The method of claim 1, wherein, clustering the person images corresponding to the cards in the same cluster within a preset time period to obtain clustering results. Obtaining a plurality of card features of a plurality of cards comprises: obtaining card data of the plurality of cards respectively; performing the following steps on the obtained plurality of card data respectively:

3. The method of claim 2, wherein, performing principal component analysis on the card data of one card in the plurality of cards to obtain the card features of the one card. Obtaining card data of the plurality of cards respectively comprises: receiving position data of the plurality of cards respectively; performing the following operations on the plurality of cards respectively: taking the position of one card in the plurality of cards as the center and a preset radius as the search radius, searching for a walking path between the one card and each of the other cards, wherein the other cards refer to the cards in the plurality of cards except the one card; 4. The method of claim 1, wherein, determining path data between the one card and the other cards based on the searched walking path. Determining a corresponding winning neuron of each of the plurality of cards in a self-organizing map network based on the obtained plurality of card features comprises: performing at least one iteration operation on the plurality of card features respectively, wherein one iteration operation comprises the following processes: determining the first similarity between the card features of one card in the plurality of cards and the weights of the plurality of neurons respectively, and taking the neuron corresponding to the maximum first similarity as the winning neuron; determining at least one neuron within the neighborhood of the winning neuron based on the neighborhood radius; updating the weights of the at least one neuron respectively based on the distance between the winning neuron and the at least one neuron and the learning rate; updating the neighborhood radius and the learning rate; Until the self-organizing mapping network meets a preset termination condition, a winning neuron obtained in a last iteration operation is determined as a winning neuron corresponding to the one mount.

5. The method of claim 4, wherein, The neighborhood radius and the learning rate are updated, including: The learning rate of the self-organizing mapping network is updated at a preset first rate, and the neighborhood radius of the self-organizing mapping network is updated at a preset second rate; If the updated learning rate is less than or equal to a preset learning rate, and / or the number of times of iteration operations on the mount feature of the one mount meets a preset number of times, it is determined that the self-organizing mapping network meets the preset termination condition.

6. The method according to any one of claims 1 to 5, wherein, Based on the winning neurons corresponding to the plurality of mounts respectively, clusters to which the plurality of mounts belong are determined respectively, including: Mounts corresponding to the same winning neuron are attributed to the same cluster.

7. The method according to any one of claims 1 to 5, wherein Mounts corresponding to the same winning neuron are attributed to the same cluster, including: If the total number of mounts corresponding to the same winning neuron is greater than or equal to a first preset number and less than or equal to a second preset number, the mounts corresponding to the same winning neuron are attributed to the same cluster, wherein the first preset number is less than the second preset number.

8. The method of claim 7, wherein, Mounts corresponding to the same winning neuron are attributed to the same cluster, further including: If the total number of mounts corresponding to the same winning neuron is less than the first preset number, the mounts corresponding to the same winning neuron and the mounts corresponding to adjacent winning neurons of the same winning neuron are attributed to the same cluster; If the total number of mounts corresponding to the same winning neuron is greater than the second preset number, the mounts corresponding to the same winning neuron are divided and attributed to at least two clusters, wherein the total number of mounts included in each cluster of the at least two clusters is greater than or equal to the first preset number and less than or equal to the second preset number.

9. A person image clustering apparatus characterized by comprising: Including: An obtaining module is configured to obtain mount features of a plurality of mounts respectively, wherein a mount feature of one mount is used to describe a position of the one mount itself and a positional relationship between the one mount and other mounts of the plurality of mounts except the one mount, wherein the mount feature of the one mount is determined based on mount data of the one mount, the mount data including position data of the one mount and path data between the one mount and other mounts of the plurality of mounts, wherein the path data between the one mount and other mounts of the plurality of mounts is obtained based on a walking path between the one mount and any one of the other mounts, wherein the walking path between the one mount and any one of the other mounts is obtained by inversely geocoding an electronic map and crawling within a preset radius with a position of the one mount as a center; determine, respectively, a winning neuron corresponding to each of the plurality of viewports in a self-organizing mapping network, wherein the winning neuron corresponding to one of the viewports is one of a plurality of neurons included in the self-organizing mapping network that has the greatest similarity to the viewport feature of the one of the viewports, and determine, respectively, a cluster to which each of the plurality of viewports belongs based on the winning neuron corresponding to each of the plurality of viewports; cluster the person images corresponding to the viewports belonging to the same cluster within a preset time period to obtain a clustering result.

10. The apparatus of claim 9, wherein, The determining module is specifically configured to: perform at least one iteration operation on the plurality of viewport features, respectively, wherein one iteration operation includes the following processes: determine a first similarity between the viewport feature of one of the plurality of viewports and the weight of each of the plurality of neurons, and take the neuron corresponding to the maximum first similarity as the winning neuron; determine at least one neuron within a neighborhood of the winning neuron based on a neighborhood radius; update the weight of each of the at least one neuron based on the distance between the winning neuron and each of the at least one neuron and a learning rate, respectively; update the neighborhood radius and the learning rate; until the self-organizing mapping network meets a preset termination condition, take the winning neuron obtained in the last iteration operation as the winning neuron corresponding to the one of the viewports.

11. A person image clustering apparatus characterized by comprising: comprise: at least one processor, and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the at least one processor implements the method according to any one of claims 1-8 by executing the instructions stored in the memory.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, when the computer instructions run on the computer, make the computer execute the method according to any one of claims 1-8. The computer readable storage medium stores computer instructions, when the computer instructions run on the computer, make the computer execute the method according to any one of claims 1-8.

Images