A crowd positioning method, system and storage medium based on hybrid expert network
The hybrid expert network addresses the challenges of crowd localization in complex scenes by employing multiple expert models with a sparse gating mechanism to refine features, improving accuracy and efficiency in diverse crowd environments.
Patent Information
- Application Number
- CN202411298363.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2024-12-17
- Estimated Expiration
- 2044-09-18
AI Technical Summary
Existing crowd localization methods struggle to accurately predict the coordinates of individuals in complex crowd scenes due to high density, occlusion, and varying environmental factors, often requiring large amounts of labeled data and complex post-processing, and are inefficient in handling diverse crowd densities and lighting conditions.
A hybrid expert network architecture that integrates a sparse gating mechanism and multiple expert models to process different areas of input features, utilizing self-attention and cross-attention mechanisms to refine crowd location features and enhance prediction accuracy.
The hybrid expert network improves the model's ability to accurately identify and locate individuals in complex scenarios by specialized processing of input features, reducing the need for extensive labeled data and enhancing prediction efficiency.
Smart Images
Figure CN118823685B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision and deep learning technology, in particular to crowd analysis technology, with a specific application in accurate crowd positioning and counting in crowded scenes. The present invention relates to a crowd positioning method, system and storage medium based on a hybrid expert network. Background Art
[0002] With the rapid development of artificial intelligence and computer vision technology, crowd positioning technology plays an increasingly important role in intelligent monitoring, public security, smart city construction and other fields. The core goal of crowd positioning is to accurately locate and identify the position of each individual in a complex crowd image. The key challenge of this task is to accurately predict the specific coordinates of each person's head from a complex background.
[0003] Mainstream crowd localization methods can be classified into the following two categories: crowd localization based on target detection and crowd localization based on density maps. The crowd localization method based on target detection first roughly locates the potential head position by initializing a pseudo bounding box, and then continuously calibrates these bounding boxes by selecting finer prediction boxes through an optimization algorithm during the training process. Although this method is intuitive and easy to understand, it requires performing non-maximum suppression (NMS) in the detection stage to eliminate redundant bounding boxes. In addition, due to the high density and occlusion problems of the crowd, the target detection method often performs poorly in high-density crowd scenes. The crowd localization method based on density maps aims to construct a continuous density map to reflect the distribution of the crowd in the image by applying a Gaussian kernel to generate a density map that matches the position and size of the head. Although this method can overcome the limitations of the target detection method to a certain extent, especially when dealing with medium-density crowds, it is difficult to avoid overlap between density points in extremely crowded scenes even if a small-sized Gaussian kernel is used. In addition, the density map method often requires a large amount of labeled data, which increases the cost and complexity of data annotation.
[0004] In recent years, a crowd localization method based on regression prediction has begun to enter the field of vision of researchers. The essence of this method is that it enables the model to directly accurately predict the coordinates of the heads of people in the image. Compared with traditional methods based on target detection or density maps, its process is simpler and clearer, avoiding NMS and cumbersome post-processing steps, and significantly improving the accuracy and efficiency of prediction. The method based on regression prediction can not only handle high-density crowd scenes, but also maintain high positioning accuracy in low-density and medium-density scenes.
[0005] However, existing crowd localization models usually analyze image features at a single scale or use a single dense network to process all inputs. However, crowd image data in actual application scenarios are affected by many factors, such as different crowd densities, different scenes, different lighting conditions, etc. Existing methods find it difficult to use a single dense network to deal with these influences. Therefore, it is very difficult and of great significance to enable the model to understand and process these complex scenes well and accurately predict crowd location information. Summary of the invention
[0006] The present invention proposes a crowd positioning method, system and storage medium based on a hybrid expert network, which utilizes a sparse expert model and a gating mechanism to enable different expert models to process different areas of input features respectively, thereby improving the model's positioning accuracy for crowd positions in complex environments.
[0007] The method specifically comprises the following steps:
[0008] Step S1: obtaining a crowd data set containing images, dividing the crowd data set into a training set, a validation set, and a test set, and performing preprocessing on crowd images and corresponding crowd position labels in the crowd data set;
[0009] Step S2: construct a convolutional neural network encoder, input the crowd images in the training set into the encoder for encoding, extract deep position features from the crowd images in the training set, and convert the original images into high-dimensional feature vector representations;
[0010] Step S3: Constructing hybrid experts The module, combined with its self-attention mechanism and hybrid expert network, performs special processing on different regions of the input features, further refines the crowd location features, and obtains the predicted crowd coordinate information;
[0011] Step S4: Build Matcher, used to match the real crowd coordinates with the predicted crowd coordinates and supervise the model training;
[0012] Step S5: construct and input the crowd images of the validation set into the model for training to obtain a crowd localization model based on a hybrid expert network;
[0013] Step S6: input the crowd images in the test set into the trained crowd localization model based on the hybrid expert network, and output the predicted crowd location information.
[0014] Furthermore, step S1 comprises the following steps:
[0015] Step S11: Divide the data set into a training set, a validation set, and a test set in proportion;
[0016] Step S12: Perform data enhancement on the images in the training set;
[0017] Step S13: Randomly crop and normalize the image after data enhancement in step S12 to convert the image data into a standard normal distribution; in order to ensure that the distribution position of the crowd in the label is consistent with the processed image, the same operation is performed on the label while enhancing the image data and preprocessing the image in each step.
[0018] Furthermore, step S2 specifically includes the following steps:
[0019] Step S21: Set the dimension size to The crowd image is input into the convolutional neural network encoder, and the dimension size is Feature map ,in , respectively represent the number of channels, height and width of the feature map. The specific expression is:
[0020] ,
[0021] in, represents the input image, Indicates the use of convolutional neural network for encoding;
[0022] Step S22: The feature map Flattened to dimensions of The feature vector of is embedded with position information. The specific expression is:
[0023] ,
[0024] in, represents the feature sequence obtained after position encoding, Indicates the position encoding of the feature map. Indicates flattening the input feature map.
[0025] Furthermore, step S3 specifically includes the following steps:
[0026] Step S31: Constructing hybrid experts encoder, which is in the original A hybrid expert module is introduced based on the encoder, and a sparse gating network is used to select the expert network involved in the prediction. Input encoder, processed by multi-head attention module and residual mechanism, obtain features , the specific expression is:
[0027] ,
[0028] in, represents the multi-head attention module, Represents the hierarchical normalization operation;
[0029] Step S32: The features obtained in step S31 Input the mixed expert network, through the sparse gating network Mechanism selection involved in prediction expert networks and weight their outputs to obtain expert output results; the sparse gating network introduces adjustable noise , introduces randomness into the selection of the expert model; then, calculates the final encoding feature , the specific calculation process and formula are as follows:
[0030] ,
[0031]
[0032]
[0033]
[0034] ,
[0035] in, is the output of the hybrid expert network, represents a sparse gating network, represents the set of expert networks in the encoder, Indicates The calculation results of the expert model are is the number of expert models, Indicates the use The function converts the input into a real number representing a probability distribution. Indicates adding noise to the input features. k express The threshold of the algorithm, for example, when k=2, The algorithm will focus on the 2 largest elements in the input set; express The input set of the algorithm, yes The elements, represents the input features of the noise adding algorithm, represents the weight of the gating network, It is a standardized operation. is the activation function.
[0036] Step S33: Constructing hybrid experts Decoder, same as steps S31 and S32, in the original The hybrid expert layer is introduced on the basis of the decoder; Input the decoder to get the decoded crowd location features , output the predicted population coordinate information; the specific expression is:
[0037] ,
[0038] ,
[0039] ,
[0040] in, represents the cross attention mechanism, is a trainable query instance, Represents mixed expert In the decoder, the expert network An expert model.
[0041] Further, step S4 specifically includes: The matcher uses the matching loss based on the Kaiser-Meyer-Olkin coefficient as the matching function; let the coordinate point set predicted by the model be , the actual coordinate point set is ,in and Represent the number of coordinate points predicted by the model and the actual number of coordinate points, respectively. The matching function formula is as follows:
[0042] ,
[0043] ， ，
[0044] in, It is The real point coordinates, It is The coordinates of the predicted points, express L1 distance, represents the Euclidean distance, Indicates The actual point closest to it The average sum of the distances between actual points, Indicates The predicted point is closest to The average sum of the distances between the predicted points, represents the number of neighboring points considered, Representative The actual coordinate point is closest to the The distance of the actual point, Representative The predicted coordinate point is closest to the The distance between the actual points, It is The confidence of the predicted points; through this matching function, each predicted coordinate point is matched with the corresponding real coordinate point.
[0045] Furthermore, step S5 specifically includes the following steps:
[0046] Step S51: Convolutional neural network and mixed experts The network is used as the basic network architecture. The convolutional neural network encoder is used to extract features from the crowd image after preprocessing in step S1 to obtain the image features after position encoding, which is recorded as ;Will Input Mix Expert The encoder selects the expert model involved in feature extraction through the gating network to further extract deep coding features ;Will Input Mix Expert The decoder selects the expert model involved in decoding through the sparse gating network to obtain the decoded crowd image features ;Will The output module of the input model outputs the predicted crowd coordinate information by the coordinate regression head, and the confidence classification head calculates the probability of the prediction being valid; then, the predicted crowd coordinate information is input into Matcher, which performs one-to-one matching between the predicted coordinates and the real coordinates to obtain the <real coordinates, predicted coordinates> point pair;
[0047] Step S52: Calculate the positioning loss based on the point pairs obtained in step S51 ; Then calculate the classification loss and the balance loss of the expert network ; The overall loss calculation formula is as follows:
[0048]
[0049] in, and is a hyperparameter used to balance the weights between various losses;
[0050] Step S53: After several rounds of training, the crowd images of the validation set are input into the model to obtain the validation result, i.e., the predicted crowd coordinates of the validation set images, and it is determined whether the result is one of the three best results at present. If so, the current model is saved and the third saved result is replaced, otherwise no replacement is made;
[0051] Step S54: Repeat steps S51-S53 in batches until the overall loss value calculated in step S52 converges and stabilizes. At this time, save the network parameters of the best model obtained in step S53 to complete the training process of the crowd positioning model based on the hybrid expert network.
[0052] Furthermore, the positioning loss function in step S52 as follows:
[0053]
[0054] in, is a point pair of <real coordinates, predicted coordinates>, is through Matcher for real points Matching prediction points, express L1 distance;
[0055] The classification loss function as follows:
[0056] ,
[0057] in, is the class weight, is the adjustment parameter, The prediction result is probability;
[0058] The balance loss as follows:
[0059] ,
[0060]
[0061] in, is the importance weight, represents the coefficient of variation, represents the importance of the expert network;
[0062] Finally, the overall loss is calculated as follows:
[0063]
[0064] in, and is a hyperparameter used to balance the weights between the various losses.
[0065] Furthermore, step S6 specifically includes: inputting the crowd image in the test set into the trained crowd positioning model based on the hybrid expert network, and adjusting its size to the size of the original crowd image, so as to obtain the corresponding crowd coordinate map.
[0066] The present invention also provides a crowd positioning system based on a hybrid expert network, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above-mentioned crowd positioning method based on a hybrid expert network can be implemented.
[0067] The present invention also provides a computer-readable storage medium on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, the above-mentioned crowd positioning method based on a hybrid expert network can be implemented.
[0068] After adopting the above scheme, the beneficial effect of the present invention is that: the present invention integrates the hybrid expert network into In the architecture, multiple expert models are able to perform specialized processing on different areas of the input features, thereby enhancing the model's ability to identify and locate people in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0069] Figure 1 It is a flow chart for realizing the method of the present invention.
[0070] Figure 2 It is a diagram of the network model structure in an embodiment of the present invention.
[0071] Figure 3 Is the hybrid expert in this invention Structure diagram of .
[0072] Figure 4 It is a structural diagram of the hybrid expert module in the present invention. DETAILED DESCRIPTION
[0073] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.
[0074] The present invention provides a crowd positioning method based on a hybrid expert network, such as Figure 1 As shown, the following steps are included:
[0075] Step S1: Divide the crowd dataset into a training set, a validation set, and a test set, and perform preprocessing on the crowd images and corresponding crowd position labels in the crowd dataset, including but not limited to data enhancement (such as rotation, flipping, scaling, etc.), color space conversion, normalization, etc.;
[0076] Step S2: construct a convolutional neural network encoder, input the training image into the encoder for encoding, perform deep feature extraction on the training image, and convert the original image into a high-dimensional feature vector representation;
[0077] Step S3: Constructing hybrid experts The module, combined with its self-attention mechanism and hybrid expert network, specifically processes different regions of the input features to further refine the crowd location features;
[0078] Step S4: Build Matcher, used to match the real crowd coordinates with the predicted crowd coordinates and supervise the model training;
[0079] Step S5: construct and train a crowd localization model based on a hybrid expert network;
[0080] Step S6: input the crowd image into the trained crowd localization model based on the hybrid expert network, and output the predicted crowd location information.
[0081] In this embodiment, step S1 specifically includes the following steps:
[0082] Step S11: Using the NWPU-Crowd dataset, the crowd dataset is divided into a training set, a validation set, and a test set containing 3109, 500, and 1500 crowd images, respectively; image preprocessing includes but is not limited to data enhancement (such as rotation, flipping, scaling, etc.), color space conversion, normalization, etc.;
[0083] Step S12: Perform data enhancement on the images in the training set;
[0084] Step S13: Preprocess the images after data enhancement in step S12 to ensure that they are suitable for the crowd localization method based on hybrid expert network. First, randomly crop the images into Pixels are added, and then the cropped image is normalized to convert the image data into a standard normal distribution. In order to ensure that the distribution of people in the label is consistent with the processed image, the label is also subjected to the same operation while enhancing and preprocessing the image data in each step.
[0085] In this embodiment, step S2 specifically includes the following steps:
[0086] Pick ,
[0087] Step S21: Set the dimension size to The crowd image is input into the convolutional neural network encoder, and the dimension size is Feature map The specific expression is:
[0088] ,
[0089] in, represents the input image, Indicates the use of convolutional neural network for encoding;
[0090] Step S22: The feature map Flattened to dimensions of The feature vector of is embedded with position information. The specific expression is:
[0091] ,
[0092] in, represents the feature sequence obtained after position encoding, Indicates the position encoding of the feature map. Indicates flattening the input feature map.
[0093] In this embodiment, step S3 specifically includes the following steps:
[0094] Step S31: Constructing hybrid experts encoder, which is in the original A hybrid expert module is introduced based on the encoder, and a sparse gating network is used to select the expert network involved in the prediction. Input encoder, processed by multi-head attention module and residual mechanism, obtain features , the specific expression is:
[0095] ,
[0096] in, represents the multi-head attention module, Represents the hierarchical normalization operation;
[0097] Step S32: The features obtained in step S31 Input the mixed expert network, through the sparse gating network Mechanism selection involved in prediction expert networks and weight their outputs to obtain expert output results; the sparse gating network introduces adjustable noise , introduces randomness into the selection of the expert model; then, calculates the final encoding feature , the specific calculation process and formula are as follows:
[0098] ,
[0099]
[0100]
[0101]
[0102] ,
[0103] in, is the output of the hybrid expert network, represents a sparse gating network, represents the set of expert networks in the encoder, Indicates The calculation results of the expert model are is the number of expert models, Indicates the use The function converts the input into a real number representing a probability distribution. Indicates adding noise to the input features. k express The threshold of the algorithm, for example, when k=2, The algorithm will focus on the 2 largest elements in the input set; express The input set of the algorithm, yes The elements, represents the input features of the noise adding algorithm, represents the weight of the gating network, It is a standardized operation. is the activation function.
[0104] Step S33: Constructing hybrid experts Decoder, similar to steps S31 and S32, in the original A hybrid expert layer is introduced based on the decoder. Input the decoder to get the decoded crowd location features , output the predicted crowd coordinate information. The specific expression is:
[0105] ,
[0106] ,
[0107] ,
[0108] in, represents the cross attention mechanism, is a trainable query instance, Represents mixed expert In the decoder, the expert network An expert model.
[0109] In this embodiment, step S4 is specifically as follows: The matcher uses the matching loss based on the Kaiser-Meyer-Olkin coefficient as the matching function. Suppose the coordinate point set predicted by the model is , the actual coordinate point set is ,in and Represent the number of coordinate points predicted by the model and the actual number of coordinate points, respectively. The matching function formula is as follows:
[0110] ,
[0111] ， ，
[0112] in, It is The real point coordinates, It is The coordinates of the predicted points, express L1 distance, represents the Euclidean distance, Indicates The actual point closest to it The average sum of the distances between actual points, Indicates The predicted point is closest to The average sum of the distances between the predicted points, represents the number of neighboring points considered, Representative The actual coordinate point is closest to the The distance of the actual point, Representative The predicted coordinate point is closest to the The distance between the actual points, It is The confidence of the predicted points; through this matching function, each predicted coordinate point is matched with the corresponding real coordinate point.
[0113] In this embodiment, step S5 specifically includes the following steps:
[0114] Step S51: Figure 2 As shown, convolutional neural network and mixed expert The network is used as the basic network architecture. First, the convolutional neural network encoder is used to extract features from the crowd image after preprocessing in step S1 to obtain the image features after position encoding, which is recorded as .Will Input Mix Expert Encoder, structure as Figure 3 As shown, for the input features After adding the position encoding, after the self-attention / cross-attention mechanism and residual and normalization processing, it is input into the hybrid expert module. The structure is as follows Figure 4 As shown (Fig. represents the input of the hybrid expert module, Indicates The output of experts), the gating network selects the expert model involved in feature extraction, further extracts deep coding features, and finally outputs features after residual and normalization processing. ;Will Input Mix Expert The decoder selects the expert model involved in decoding through the sparse gating network to obtain the decoded crowd position features ;Will The output module of the input model outputs the predicted crowd coordinate information by the coordinate regression head, and the confidence classification head calculates the probability of the prediction being valid; then, the predicted crowd coordinate information is input into Matcher, which performs one-to-one matching between the predicted coordinates and the real coordinates to obtain the <real coordinates, predicted coordinates> point pair;
[0115] Step S52: Calculate the positioning loss based on the point pairs obtained in step S51 ; Then calculate the classification loss and the balance loss of the expert network The overall loss calculation formula is as follows:
[0116]
[0117] in, and is a hyperparameter used to balance the weights between the various losses. For reference, take 0.1. You can refer to 2.5. Hyperparameters are usually determined through experience or experiments. Here, you can refer to the existing hyperparameter value selection methods in the same field to select values.
[0118] Step S53: After several rounds of training, the crowd images of the validation set are input into the model to obtain the validation result, i.e., the predicted crowd coordinates of the validation set images, and it is determined whether the result is one of the three best results at present. If so, the current model is saved and the third saved result is replaced, otherwise no replacement is made;
[0119] Step S54: Repeat steps S51-S53 in batches until the overall loss value calculated in step S52 converges and stabilizes. At this time, save the network parameters of the best model obtained in step S53 to complete the training process of the crowd positioning model based on the hybrid expert network.
[0120] In this embodiment, the loss calculation described in step S5 specifically includes the following calculations:
[0121] The positioning loss function described in step S5 as follows:
[0122]
[0123] in, is a point pair of <real coordinates, predicted coordinates>, is through Matcher for real points Matching prediction points, express L1 distance;
[0124] The classification loss function as follows:
[0125] ,
[0126] in, is the class weight, is the adjustment parameter, The prediction result is probability;
[0127] The balance loss as follows:
[0128] ,
[0129]
[0130] in, is the importance weight, represents the coefficient of variation, represents the importance of the expert network;
[0131] Finally, the overall loss is calculated as follows:
[0132]
[0133] in, and is a hyperparameter used to balance the weights between the various losses.
[0134] In this embodiment, step S6 specifically includes: inputting the crowd image in the test set into the trained crowd positioning model based on the hybrid expert network, and adjusting its size to the size of the original crowd image, so as to obtain the corresponding crowd coordinate map.
[0135] The present invention introduces a hybrid expert network on the basis of a crowd positioning method based on regression prediction to address the problems of the prior art.
[0136] The hybrid expert network is similar to the ensemble learning method, which aims to combine multiple individual networks into a system and establish a supervision mechanism. In this system, each network processes a different subset of training samples, focusing on a specific area of the input space. It usually contains an expert network composed of expert models and a trainable sparse gating network. The gating network is used to select the expert model involved in the calculation. The output of the hybrid expert layer represents the weighted average sum of the outputs of all selected experts.
[0137] The present invention also provides a crowd positioning system based on a hybrid expert network, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above-mentioned crowd positioning method based on a hybrid expert network can be implemented.
[0138] The present invention also provides a computer-readable storage medium on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, the above-mentioned crowd positioning method based on a hybrid expert network can be implemented.
[0139] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0140] The present invention is described with reference to the flowcharts of the methods, devices (apparatus), and computer program products of the embodiments of the present invention. It should be understood that each process in the flowchart, as well as the combination of processes in the flowchart, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart. Figure 1 A device that specifies functions in a process or multiple processes.
[0141] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple functions specified in a flowchart.
[0142] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 The steps of a specified function in a process or multiple processes.
[0143] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.
[0144] This patent is not limited to the above-mentioned optimal implementation mode. Anyone can derive other forms of nighttime street scene semantic segmentation methods based on difficult category perception mechanisms under the inspiration of this patent. All equal changes and modifications made according to the scope of the patent application of the present invention should be covered by this patent.
Claims
1. A crowd positioning method based on a hybrid expert network, characterized in that: The method comprises the following steps: Step S1: obtaining a crowd data set containing images, dividing the crowd data set into a training set, a validation set, and a test set, and performing preprocessing on crowd images and corresponding crowd position labels in the crowd data set; Step S2: construct a convolutional neural network encoder, input the crowd images in the training set into the encoder for encoding, extract deep position features from the crowd images in the training set, and convert the original images into high-dimensional feature vector representations; Step S2 specifically includes the following steps: Step S21: Set the dimension size to The crowd image is input into the convolutional neural network encoder, and the dimension size is Feature map ,in Respectively represent the number of channels, height and width of the feature map. The specific expression is: , in, represents the input image, Indicates the use of convolutional neural network for encoding; Step S22: The feature map Flattened to dimensions of The feature vector of is embedded with position information. The specific expression is: , in, represents the feature sequence obtained after position encoding, Indicates the position encoding of the feature map. Indicates flattening the input feature map; Step S3: Constructing hybrid experts The module, combined with its self-attention mechanism and hybrid expert network, performs special processing on different regions of the input features, further refines the crowd location features, and obtains the predicted crowd coordinate information; Step S3 specifically includes the following steps: Step S31: Constructing hybrid experts encoder, which is in the original A hybrid expert module is introduced based on the encoder, and a sparse gating network is used to select the expert network involved in the prediction. Input encoder, processed by multi-head attention module and residual mechanism, obtain features , the specific expression is: , in, represents the multi-head attention module, Represents the hierarchical normalization operation; Step S32: The features obtained in step S31 Input the mixed expert network, through the sparse gating network Mechanism selection involved in prediction expert networks and weight their outputs to obtain expert output results; the sparse gating network introduces adjustable noise , introduces randomness into the selection of the expert model; then, calculates the final encoding feature , the specific calculation process and formula are as follows: , , in, is the output of the hybrid expert network, represents a sparse gating network, represents the set of expert networks in the encoder, Indicates The calculation results of the expert model are is the number of expert models, Indicates the use The function converts the input into a real number representing a probability distribution. Indicates adding noise to the input features. k express The threshold of the algorithm; express The input set of the algorithm, yes The The input features of the expert model are represents the input features of the noise adding algorithm, represents the weight of the gating network, It is a standardized operation. is the activation function; Step S33: Constructing hybrid experts Decoder, same as steps S31 and S32, in the original The hybrid expert layer is introduced on the basis of the decoder; Input the decoder to get the decoded crowd location features , output the predicted population coordinate information; the specific expression is: ， ， ， in, represents the cross attention mechanism, is a trainable query instance, Represents mixed expert In the decoder, the expert network Expert models; Step S4: Build Matcher, used to match the real crowd coordinates with the predicted crowd coordinates and supervise the model training; Step S5: construct and input the crowd images of the validation set into the model for training to obtain a crowd localization model based on a hybrid expert network; Step S6: input the crowd images in the test set into the trained crowd localization model based on the hybrid expert network, and output the predicted crowd location information.
2. The crowd positioning method based on hybrid expert network according to claim 1, characterized in that: Step S1 comprises the following steps: Step S11: Divide the data set into a training set, a validation set, and a test set according to a certain ratio; Step S12: Perform data enhancement on the images in the training set; Step S13: Randomly crop and normalize the image after data enhancement in step S12 to convert the image data into a standard normal distribution; in order to ensure that the distribution position of the crowd in the label is consistent with the processed image, the same operation is performed on the label while enhancing the image data and preprocessing the image in each step.
3. A crowd positioning method based on a hybrid expert network according to claim 1, characterized in that, step S4 specifically includes: The matcher uses the matching loss based on the Kaiser-Meyer-Olkin coefficient as the matching function; let the coordinate point set predicted by the model be , the actual coordinate point set is ,in and Represent the number of coordinate points predicted by the model and the actual number of coordinate points, respectively. The matching function formula is as follows: , ， ， in, It is The real point coordinates, It is The coordinates of the predicted points, express L1 distance, represents the Euclidean distance, Indicates The actual point closest to it The average sum of the distances between actual points, Indicates The predicted point is closest to The average sum of the distances between the predicted points, represents the number of neighboring points considered, Representative The actual coordinate point is closest to the The distance of the actual point, Representative The predicted coordinate point is closest to the The distance between the actual points, It is The confidence of the predicted points; through this matching function, each predicted coordinate point is matched with the corresponding real coordinate point.
4. The crowd positioning method based on hybrid expert network according to claim 1 is characterized by: Step S5 specifically includes the following steps: Step S51: Convolutional neural network and mixed experts The network is used as the basic network architecture. The convolutional neural network encoder is used to extract features from the crowd image after preprocessing in step S1 to obtain the image features after position encoding, which is recorded as ;Will Input Mix Expert The encoder selects the expert model involved in feature extraction through the sparse gating network to further extract deep coding features ;Will Input Mix Expert The decoder selects the expert model involved in decoding through the sparse gating network to obtain the decoded crowd image features ;Will The output module of the input model outputs the predicted crowd coordinate information by the coordinate regression head, and the confidence classification head calculates the probability of the prediction being valid; then, the predicted crowd coordinate information is input into Matcher, which performs one-to-one matching between the predicted coordinates and the real coordinates to obtain the <real coordinates, predicted coordinates> point pair; Step S52: Calculate the positioning loss based on the point pairs obtained in step S51 ; Then calculate the classification loss and the balance loss of the expert network ; The overall loss calculation formula is as follows: in, and is a hyperparameter used to balance the weights between various losses; Step S53: After several rounds of training, input the validation set images into the model to obtain the validation results, i.e., the predicted population coordinates of the validation set images, and determine whether the result is one of the three best results. If so, save the current model and replace the third saved result, otherwise do not replace it; Step S54: Repeat steps S51-S53 in batches until the overall loss value calculated in step S52 converges and stabilizes. At this time, save the network parameters of the best model obtained in step S53 to complete the training process of the crowd positioning model based on the hybrid expert network.
5. The crowd positioning method based on hybrid expert network according to claim 4 is characterized in that: The positioning loss as follows: in, is a point pair of <real coordinates, predicted coordinates>, is through Matcher for real points Matching prediction points, express L1 distance; The classification loss as follows: , in, is the class weight, is the adjustment parameter, The prediction result is probability; The balance loss as follows: , in, is the importance weight, represents the coefficient of variation, represents the importance of the expert network; Finally, the overall loss is calculated as follows: in, and is a hyperparameter used to balance the weights between the various losses.
6. The crowd positioning method based on hybrid expert network according to claim 1 is characterized in that: The step S6 specifically includes: inputting the crowd image in the test set into the trained crowd positioning model based on the hybrid expert network, and adjusting its size to the size of the original crowd image, so as to obtain the corresponding crowd coordinate map.
7. A crowd positioning system based on a hybrid expert network, characterized in that: The method comprises a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, the method according to any one of claims 1 to 6 can be implemented.
8. A computer-readable storage medium, characterized in that: Computer program instructions that can be executed by a processor are stored thereon, and when the processor executes the computer program instructions, the method according to any one of claims 1 to 6 can be implemented.