Pedestrian position auxiliary marking method and system in dense crowd image
By adopting two iterative training methods in dense crowd image processing, using the distillation technology of open source crowd positioning data set and the second network model, the inefficiency of manual labeling in dense crowd scenarios is solved, and the speed of pedestrian position labeling is achieved. Improved accuracy.
Patent Information
- Application Number
- CN202410938226.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-11-05
AI Technical Summary
In dense crowd scenarios, manual marking of pedestrian locations requires a lot of manpower, and the prior art is difficult to effectively improve the speed and accuracy of labeling.
Using two iterative training methods based on open source population positioning data sets, first pre-training is performed through the initial model, and then the second network model is introduced for distillation, using coarse image characterization and refined image characterization to construct a loss function, and the first network model is performed. Further training is generated to generate a stable annotation model for auxiliary annotation.
It effectively reduces the manpower investment in image labeling for dense crowds, improves the speed and accuracy of pedestrian position labeling, and the generated annotation model has strong annotation capabilities and stability.
Smart Images

Figure CN118898801A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to a method and system for auxiliary marking of pedestrian positions in dense crowd images. Background Art
[0002] With the development of productivity, people's overall living standards have improved. On some holidays and weekends, more and more people choose to travel. This "crowding" travel style has brought great pressure to public transportation and tourist attractions, and even brought potential safety risks. If it is not managed and supervised in some very crowded environments, it is likely to cause some safety accidents. For example, on New Year's Eve in 2015, 35 people were killed in a stampede due to overcrowding during the "New Year's Eve" event held in Shanghai; during the "Halloween" in 2022, 158 people were killed in a stampede in Itaewon, Seoul, South Korea. Therefore, how to effectively manage and conduct crowd detection and analysis is a work with social security significance.
[0003] At present, the detection of the number of people in a crowded crowd mainly relies on manual on-site supervision or monitoring assisted by manual supervision, but these methods require a lot of manpower. In addition, as the degree of crowding increases, it becomes more challenging to supervise and respond quickly through manual supervision. With the maturity of deep learning technology and image processing technology, artificial intelligence technology can be used to assist management in real-time monitoring of crowds, so as to prepare for the occurrence of some safety accidents.
[0004] However, the reason why artificial intelligence computer vision technology based on deep learning has a high accuracy rate is that it is built on a large-scale and high-quality data set, which requires a large number of labeled samples. But in practical applications, because the number of people in a crowd image can reach hundreds or thousands, it takes a lot of effort to manually annotate a very dense crowd. At present, no research has proposed an effective solution to the problem of high annotation costs for crowd images. Summary of the invention
[0005] In view of this, the present application proposes a method and system for auxiliary labeling of pedestrian positions in dense crowd images, aiming to solve the problem of excessive manpower investment in labeling pedestrian positions in dense crowd scenes, and effectively improve the speed and accuracy of pedestrian position labeling in dense crowd images.
[0006] In order to achieve the above-mentioned purpose, an embodiment of the present application proposes a method for auxiliary labeling of pedestrian positions in dense crowd images, including: performing a first iterative training on an initial model based on an open source crowd positioning data set, completing the first iterative training after reaching a preset first number of training rounds, and obtaining a first network model; inputting the training data in the open source crowd positioning data set into the first network model, obtaining a coarse image representation output by the first network model based on the training data, and obtaining an intermediate representation output by the penultimate layer of the first network model based on the training data; inputting the intermediate representation into a second network model, obtaining a refined image representation output by the second network model based on the intermediate representation; constructing a first loss function based on the coarse image representation, the refined image representation and a label map annotated on the training data, and performing a second iterative training on the first network model based on the first loss function, completing the second iterative training after reaching a preset second number of training rounds, and obtaining a labeling network; inputting the image to be labeled into the labeling network, obtaining the auxiliary label predicted by the labeling network, and annotating the auxiliary label to the image to be labeled.
[0007] The embodiment of the present application also proposes a pedestrian position auxiliary labeling system in a dense crowd image, the system comprising a first training module, a second training module and a model use module; the first training module is used to perform a first iterative training on the initial model based on an open source crowd positioning data set, complete the first iterative training after reaching a preset first training round number, and obtain a first network model; the second training module is used to input the training data in the open source crowd positioning data set into the first network model, obtain the coarse image representation output by the first network model based on the training data, and obtain the intermediate representation output by the penultimate layer of the first network model based on the training data, input the intermediate representation into the second network model, obtain the refined image representation output by the second network model based on the intermediate representation, and finally construct a first loss function based on the coarse image representation, the refined image representation and the label map annotated on the training data, and perform a second iterative training on the first network model based on the first loss function, complete the second iterative training after reaching a preset second training round number, and obtain the labeled network; the model use module is used to input the image to be labeled into the labeling network, obtain the auxiliary label predicted by the labeling network, and annotate the auxiliary label to the image to be labeled.
[0008] An embodiment of the present application also proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for auxiliary labeling of pedestrian positions in dense crowd images as described above.
[0009] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method for auxiliary marking of pedestrian positions in dense crowd images as described above.
[0010] The embodiment of the present application proposes a method and system for assisting the annotation of pedestrian positions in dense crowd images. First, the initial model is trained for the first iteration based on the open source crowd positioning data set to obtain a first network model. The first iteration training is a pre-training, which can enable the first network model to quickly obtain basic pedestrian position annotation capabilities. Considering that the amount of data in the open source crowd positioning data set is relatively small and the scenes included are not many, the first network model will produce some noise predictions when annotating, and the output image representation is relatively rough. Therefore, the present application introduces a second network model, and uses the second network model to distill the first network model to obtain a refined image representation with less noise. The first loss function is constructed using the coarse image representation and the refined image representation label map. The first loss function is used to perform a second iteration training on the first network model, and consistency constraints are imposed on it, so as to train a strong and stable annotation model. The trained annotation model is used to assist in the annotation of the image to be annotated, which effectively solves the problem of excessive manpower investment in annotating pedestrian positions in dense crowd scenes, and effectively improves the speed and accuracy of pedestrian position annotation in dense crowd images.
[0011] Optionally, the coarse image representation output by the first network model based on the training data is specifically a confidence map with the same resolution as the training data, and the refined image representation output by the second network model based on the intermediate representation is specifically a threshold map with the same resolution as the training data. The first loss function is constructed based on the coarse image representation, the refined image representation and the label map annotated on the training data, including: comparing the confidence map and the threshold map pixel by pixel, and determining the predicted value of the current pixel based on the value of the corresponding position of the current pixel in the confidence map and the value of the corresponding position in the threshold map; wherein, if the value of the corresponding position of the current pixel in the confidence map is greater than or equal to the value of the corresponding position in the threshold map, the predicted value of the current pixel is determined to be 1, otherwise, the predicted value of the current pixel is determined to be 0; based on the predicted value of each pixel, the predicted map corresponding to the training data is obtained; based on the predicted value of each pixel, the first loss function is constructed.
[0012] Optionally, the first loss function is constructed based on the prediction graph corresponding to the training data and the label graph marked on the training data, and is implemented by the following formula:
[0013]
[0014] Among them, y i represents the label value of the i-th pixel in the label map Y, Representing the prediction graph The predicted value of the i-th pixel in , ds represents the integration in the spatial dimension, represents the error between the label image and the prediction image, and L1 represents the first loss function constructed.
[0015] Optionally, the first network model consists of 25 convolution modules, and the second network model consists of 3 convolution modules; the second network model first uses 720 convolution layers with a convolution kernel size of 3×3, a convolution kernel channel of 256, a 0-padded length of 1, and a step size of 1 to compress the intermediate representation into a 256-dimensional representation matrix, and then successively uses a convolution layer with a convolution kernel size of 3×3, a convolution kernel channel of 64, and a convolution layer with a convolution kernel size of 1×1, and a convolution kernel channel of 1 to convert the 256-dimensional representation matrix into a 1-dimensional representation matrix, and finally upsamples the 1-dimensional representation matrix by 8 times to obtain a refined image representation.
[0016] Optionally, while constructing a first loss function based on the coarse image representation, the refined image representation and the label map annotated on the training data, and performing a second iterative training on the first network model based on the first loss function, the method further includes: constructing a second loss function based on the coarse image representation and the refined image representation; and performing consistency constraint iterative training on the second network model based on the second loss function. While performing the second iterative training on the first network model, the second network model may also be subjected to consistency constraint iterative training, thereby improving the performance of the second network model, and ultimately better improving the annotation accuracy and annotation speed of the trained annotation model.
[0017] Optionally, constructing a second loss function based on the coarse image representation and the refined image representation is implemented by the following formula:
[0018]
[0019] Among them, f j represents the j-th coarse image representation, represents the jth refined image representation, ds represents integration in the spatial dimension, Represents the coarse image representation F compared to the refined image representation The error between them, L2 represents the second loss function constructed.
[0020] Optionally, the preset first number of training rounds and the preset second number of training rounds are both 300 rounds. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] One or more embodiments are exemplarily described by the pictures in the corresponding drawings, and these exemplary descriptions do not constitute limitations on the embodiments.
[0022] Figure 1 is a flow chart of a method for assisting the marking of pedestrian positions in a dense crowd image proposed in one embodiment of the present application;
[0023] Figure 2 is a structural schematic diagram of a pedestrian position auxiliary annotation system in a dense crowd image proposed in another embodiment of the present application;
[0024] Figure 3 It is a schematic diagram of the structure of an electronic device provided by another embodiment of the present application. DETAILED DESCRIPTION
[0025] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not be construed as any limitation, and the various embodiments can be combined and referenced with each other under the premise of no contradiction.
[0026] An embodiment of the present application proposes a method for auxiliary labeling of pedestrian positions in dense crowd images, which is applied to an electronic device, wherein the electronic device can be a terminal or a server. In this embodiment and the following embodiments, the electronic device is described using a server as an example. The implementation details of the method for auxiliary labeling of pedestrian positions in dense crowd images proposed in this embodiment are specifically described below. The following content is only the implementation details provided for easy understanding and is not necessary for implementing this solution.
[0027] The process of the pedestrian position auxiliary labeling method in the dense crowd image proposed in this embodiment is as follows: Figure 1 As shown, including:
[0028] Step 101 , performing a first iterative training on the initial model based on an open source crowd positioning data set, completing the first iterative training after reaching a preset first training round number, and obtaining a first network model.
[0029] In a specific implementation, the server first constructs an initial model, and then performs a first iteration training on the initial model based on an open source crowd positioning data set, and completes the first iteration training after reaching a preset first training round number to obtain a first network model. The preset first training round number can generally be set to 300 rounds.
[0030] In one example, the open source crowd positioning dataset selected by the server may be ShanghaiTechPartA. Each element (training data) in the dataset is a single image, and the corresponding label image is a binary image, where the foreground of the binary image represents the position of the head in the crowd.
[0031] It is worth noting that the ShanghaiTechPartA dataset contains only about 300 images (training data) and contains relatively few scenes. For image annotation applications for open scenes, the information is relatively scarce. Therefore, if the trained first network model is used directly for label annotation, many incorrect annotations will occur. For this reason, the server introduces the second network model to participate in the second round of training to improve the annotation accuracy.
[0032] Step 102: input the training data in the open source crowd positioning data set into the first network model, obtain the coarse image representation output by the first network model based on the training data, and obtain the intermediate representation output by the penultimate layer of the first network model based on the training data.
[0033] In the specific implementation, after obtaining the first network model, the server can introduce the second network model for the second round of training. First, the training data in the open source crowd positioning dataset is input into the first network model to obtain the coarse image representation output by the first network model based on the training data. At the same time, the intermediate representation of the penultimate layer of the first network model based on the training data output is also obtained.
[0034] In one example, the first network model is composed of 25 convolution modules. The coarse image representation output by the first network model based on the training data is specifically a confidence map F with the same resolution as the training data. The process can be expressed by the formula:
[0035] F=NET1(X)
[0036] Among them, X represents the input training data, NET1(·) represents the first network model, and F represents the output confidence map.
[0037] In one example, the intermediate representation output by the penultimate layer of the first network model based on the training data is denoted by G.
[0038] Step 103: input the intermediate representation into the second network model to obtain a refined image representation output by the second network model based on the intermediate representation.
[0039] In a specific implementation, after obtaining the intermediate representation, the server can input the intermediate representation into the second network model to obtain the refined image representation output by the second network model based on the intermediate representation. The second network model converts the intermediate representation into a refined image representation with less noise, thereby distilling the first network model in the second round of training.
[0040] In one example, the second network model consists of three convolution modules. The second network model first uses 720 convolution layers with a convolution kernel size of 3×3, a convolution kernel channel of 256, a zero-padded length of 1, and a stride of 1 to compress the intermediate representation into a 256-dimensional representation matrix, and then successively uses a convolution layer with a convolution kernel size of 3×3, a convolution kernel channel of 64, and a convolution layer with a convolution kernel size of 1×1 and a convolution kernel channel of 1 to convert the 256-dimensional representation matrix into a 1-dimensional representation matrix, and finally upsamples the 1-dimensional representation matrix by 8 times to obtain a refined image representation.
[0041] In one example, the refined image representation output by the second network model based on the intermediate representation is specifically a threshold image with the same resolution as the training data. The process can be expressed by the formula:
[0042]
[0043] Where G represents the intermediate representation of the input, NET2(·) represents the second network model, Represents the threshold map of the output.
[0044] Step 104, based on the coarse image representation, the refined image representation and the label map annotated on the training data, construct a first loss function, and perform a second iterative training on the first network model based on the first loss function, and complete the second iterative training after reaching a preset second training round number to obtain a labeled network.
[0045] In a specific implementation, after obtaining the coarse image representation output by the first network model and the refined image representation output by the second network model, the server can construct a first loss function based on the coarse image representation, the refined image representation and the label map annotated on the training data, and perform a second iterative training on the first network model based on the first loss function, and complete the second iterative training after reaching a preset second training round number to obtain a labeled network. The preset second training round number can generally be set to 300 rounds.
[0046] In one example, when constructing the first loss function based on the coarse image representation (confidence map), the refined image representation (threshold map), and the label map annotated on the training data, the server first compares the confidence map and the threshold map pixel by pixel, and determines the predicted value of the current pixel based on the value of the corresponding position of the current pixel in the confidence map and the value of the corresponding position in the threshold map, wherein if the value of the corresponding position of the current pixel in the confidence map is greater than or equal to the value of the corresponding position in the threshold map, the predicted value of the current pixel is determined to be 1, otherwise, the predicted value of the current pixel is determined to be 0. Based on the predicted value of each pixel, the server can obtain the predicted map corresponding to the training data, and then construct the first loss function based on the predicted map corresponding to the training data and the label map annotated on the training data.
[0047] In one example, the server constructs a first loss function based on a prediction graph corresponding to training data and a label graph annotated on the training data, which can be implemented by the following formula:
[0048]
[0049] Among them, y i Represents the label value of the i-th pixel in the label map Y, Represents the prediction graph The predicted value of the i-th pixel in , ds represents the integration in the spatial dimension, represents the error between the label image and the predicted image, L1 represents the first loss function constructed, i∈[1,W×H], W represents the label image Y, the predicted image width, H represents the label map Y, the prediction map The height of W×H represents the label map Y and the prediction map The number of pixels in the image.
[0050] In one example, the server constructs a first loss function based on the coarse image representation (confidence map), the refined image representation (threshold map) and the label map annotated on the training data, and performs a second iterative training on the first network model based on the first loss function. At the same time, the server can also construct a second loss function based on the coarse image representation (confidence map) and the refined image representation (threshold map), and perform consistency constrained iterative training on the second network model based on the second loss function.
[0051] It is understandable that while performing the second iterative training on the first network model, the second network model can also be subjected to consistency constraint iterative training, thereby improving the performance of the second network model and ultimately improving the labeling accuracy and labeling speed of the trained labeling model.
[0052] In one example, the server constructs a second loss function based on the coarse image representation and the refined image representation, which can be implemented by the following formula:
[0053]
[0054] Among them, f j represents the j-th coarse image representation, represents the jth refined image representation, ds represents integration in the spatial dimension, Represents the coarse image representation map (confidence map) F compared to the refined image representation map (threshold map) The error between them, L2 represents the second loss function constructed.
[0055] Step 105: input the image to be annotated into the annotation network, obtain the auxiliary labels predicted by the annotation network, and annotate the auxiliary labels to the image to be annotated.
[0056] In a specific implementation, after obtaining the annotation network, the server can deploy the annotation model to the application scenario, obtain the image to be annotated in real time, input the image to be annotated into the annotation network, obtain the auxiliary labels predicted by the annotation network, and annotate the auxiliary labels to the image to be annotated.
[0057] In one example, after annotating the auxiliary label predicted by the network, a person skilled in the art may fine-tune the auxiliary label, that is, make a small-scale correction, and finally annotate the fine-tuned label to the image to be annotated.
[0058] In this embodiment, the first iterative training of the initial model is first performed based on the open source crowd positioning data set to obtain the first network model. The first iterative training is a pre-training, which can enable the first network model to quickly obtain basic pedestrian position annotation capabilities. Considering that the amount of data in the open source crowd positioning data set is relatively small and the scenes included are not many, the first network model will produce some noise predictions when annotating, and the output image representation is relatively coarse. Therefore, the second network model is introduced in this application, and the first network model is used to distill the first network model to obtain a refined image representation with less noise. The coarse image representation and the refined image representation label map are used to construct a first loss function, and the first loss function is used to perform a second iterative training on the first network model, that is, to perform consistency constraint training on it, so as to train a strong and stable annotation model, and use the trained annotation model to assist in the annotation of the image to be annotated, which effectively solves the problem of excessive manpower investment in annotating pedestrian positions in dense crowd scenes, and effectively improves the speed and accuracy of pedestrian position annotation in dense crowd images.
[0059] The step division of the above methods is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this application.
[0060] Another embodiment of the present application proposes a pedestrian position auxiliary labeling system in a dense crowd image. The implementation details of the pedestrian position auxiliary labeling system in a dense crowd image proposed in this embodiment are specifically described below. The following content is only for the convenience of understanding the implementation details provided, and is not necessary for the implementation of this solution. The pedestrian position auxiliary labeling system in a dense crowd image proposed in this embodiment can be as follows Figure 2 As shown, it includes: a first training module 201, a second training module 202 and a model using module 203.
[0061] The first training module 201 is used to perform a first iterative training on the initial model based on the open source crowd positioning data set, and complete the first iterative training after reaching a preset first training round number to obtain a first network model.
[0062] The second training module 202 is used to input the training data in the open source crowd positioning data set into the first network model, obtain the coarse image representation output by the first network model based on the training data, and obtain the intermediate representation output by the penultimate layer of the first network model based on the training data, input the intermediate representation into the second network model, obtain the refined image representation output by the second network model based on the intermediate representation, and finally construct a first loss function based on the coarse image representation, the refined image representation and the label map annotated on the training data, and perform a second iterative training on the first network model based on the first loss function, and complete the second iterative training after reaching a preset second training round number to obtain a labeled network.
[0063] The model using module 203 is used to input the image to be annotated into the annotation network, obtain the auxiliary labels predicted by the annotation network, and annotate the auxiliary labels to the image to be annotated.
[0064] It is not difficult to find that this embodiment is a system embodiment corresponding to the above method embodiment, and this embodiment can be implemented in conjunction with the above method embodiment. The relevant technical details and technical effects mentioned in the above embodiments are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiments.
[0065] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.
[0066] Another embodiment of the present application provides an electronic device, such as Figure 3 As shown, it includes: at least one processor 301; and a memory 302 that is communicatively connected to the at least one processor 301; wherein the memory 302 stores instructions that can be executed by the at least one processor 301, and the instructions are executed by the at least one processor 301 so that the at least one processor 301 can execute the pedestrian position auxiliary labeling method in the dense crowd image described in the above-mentioned method embodiments.
[0067] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be further described in this article. The bus interface is responsible for providing an interface between the bus and the transceiver. The transceiver can be one component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium through an antenna, and further, the antenna also receives data and transmits the data to the processor.
[0068] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.
[0069] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method for auxiliary labeling of pedestrian positions in dense crowd images described in the above method embodiments.
[0070] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as: ROM), random access memory (Random Access Memory, referred to as: RAM), disk or optical disk and other media that can store program codes.
[0071] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.
Claims
1. A method for assisting pedestrian position annotation in dense crowd images, characterized in that: include: Performing a first iteration training on the initial model based on the open source crowd positioning data set, completing the first iteration training after reaching a preset first training round number, and obtaining a first network model; Inputting the training data in the open source crowd positioning dataset into the first network model, obtaining a coarse image representation output by the first network model based on the training data, and obtaining an intermediate representation output by the penultimate layer of the first network model based on the training data; Inputting the intermediate representation into a second network model, and obtaining a refined image representation output by the second network model based on the intermediate representation; Based on the coarse image representation, the refined image representation and the label map annotated on the training data, a first loss function is constructed, and the first network model is subjected to second iterative training based on the first loss function, and the second iterative training is completed after reaching a preset second training round number to obtain a labeled network; The image to be labeled is input into the labeling network, the auxiliary label predicted by the labeling network is obtained, and the auxiliary label is labeled to the image to be labeled.
2. The method for assisting pedestrian position annotation in dense crowd images according to claim 1, characterized in that: The coarse image representation output by the first network model based on the training data is specifically a confidence map with the same resolution as the training data, the refined image representation output by the second network model based on the intermediate representation is specifically a threshold map with the same resolution as the training data, and the first loss function is constructed based on the coarse image representation, the refined image representation and the label map annotated on the training data, including: Comparing the confidence map and the threshold map pixel by pixel, and determining the predicted value of the current pixel based on the value of the corresponding position of the current pixel in the confidence map and the value of the corresponding position of the threshold map; wherein, if the value of the corresponding position of the current pixel in the confidence map is greater than or equal to the value of the corresponding position in the threshold map, then the predicted value of the current pixel is determined to be 1, otherwise, then the predicted value of the current pixel is determined to be 0; Based on the predicted value of each pixel point, a prediction graph corresponding to the training data is obtained; A first loss function is constructed based on a prediction graph corresponding to the training data and a label graph annotated on the training data.
3. The method for assisting pedestrian position labeling in dense crowd images according to claim 2, characterized in that: The first loss function is constructed based on the prediction graph corresponding to the training data and the label graph marked on the training data, and is implemented by the following formula: Among them, y i represents the label value of the i-th pixel in the label map Y, Representing the prediction graph The predicted value of the i-th pixel in , ds represents the integration in the spatial dimension, represents the error between the label image and the prediction image, and L1 represents the first loss function constructed.
4. The method for assisting pedestrian position annotation in dense crowd images according to claim 1, characterized in that: The first network model consists of 25 convolutional modules, and the second network model consists of 3 convolutional modules; The second network model first uses 720 convolution layers with a convolution kernel size of 3×3, a convolution kernel channel of 256, a 0-padded length of 1, and a stride of 1 to compress the intermediate representation into a 256-dimensional representation matrix, and then successively uses a convolution layer with a convolution kernel size of 3×3, a convolution kernel channel of 64, and a convolution layer with a convolution kernel size of 1×1, and a convolution kernel channel of 1 to convert the 256-dimensional representation matrix into a 1-dimensional representation matrix, and finally upsamples the 1-dimensional representation matrix by 8 times to obtain a refined image representation.
5. The method for assisting pedestrian position labeling in dense crowd images according to claim 4, characterized in that: While constructing a first loss function based on the coarse image representation, the refined image representation, and the label map annotated on the training data, and performing a second iterative training on the first network model based on the first loss function, the method further includes: Constructing a second loss function based on the coarse image representation and the refined image representation; Based on the second loss function, the second network model is iteratively trained with consistency constraints.
6. The method for assisting pedestrian position labeling in dense crowd images according to claim 5, characterized in that: The second loss function is constructed based on the coarse image representation and the refined image representation, and is implemented by the following formula: Among them, f j represents the j-th coarse image representation, represents the jth refined image representation, ds represents integration in the spatial dimension, Represents the coarse image representation F compared to the refined image representation The error between them, L2 represents the second loss function constructed.
7. The method for assisting pedestrian position labeling in a dense crowd image according to any one of claims 1 to 6, characterized in that: The preset first number of training rounds and the preset second number of training rounds are both 300 rounds.
8. A pedestrian position auxiliary labeling system in dense crowd images, characterized in that: The system comprises: a first training module, a second training module and a model use module; The first training module is used to perform a first iterative training on the initial model based on the open source crowd positioning data set, and complete the first iterative training after reaching a preset first training round number to obtain a first network model; The second training module is used to input the training data in the open source crowd positioning data set into the first network model, obtain the coarse image representation output by the first network model based on the training data, and obtain the intermediate representation output by the penultimate layer of the first network model based on the training data, input the intermediate representation into the second network model, obtain the refined image representation output by the second network model based on the intermediate representation, and finally construct a first loss function based on the coarse image representation, the refined image representation and the label map annotated on the training data, and perform a second iterative training on the first network model based on the first loss function, and complete the second iterative training after reaching a preset second training round number to obtain a labeled network; The model using module is used to input the image to be annotated into the annotation network, obtain the auxiliary label predicted by the annotation network, and annotate the auxiliary label to the image to be annotated.
9. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for auxiliary marking of pedestrian positions in dense crowd images according to any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for auxiliary marking of pedestrian positions in a dense crowd image according to any one of claims 1 to 7 is implemented.
