Image annotation method, device, electronic device and storage medium
Through the feature fusion network of the feature extraction network and the bidirectional feature pyramid network, the pedestrian image is automatically marked, which solves the problem of high workload in pedestrian image annotation and achieves efficient and accurate pedestrian annotation.
Patent Information
- Application Number
- CN202411365884.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-29
- Publication Date
- 2024-12-24
- Estimated Expiration
- 2044-09-29
AI Technical Summary
Currently, the applicability of pedestrian images is huge and time-consuming, and there is a lack of efficient automatic annotation methods.
A feature fusion network composed of feature extraction network and a bidirectional feature pyramid network is adopted to obtain a multi-scale feature map of pedestrian images through training models, calculate the distribution focus loss, cross-conversion loss and cross-entropy loss, generate the overall loss value, and automatically perform the pedestrian area. and label prediction and labeling.
The annotation process of pedestrian images is simplified, the labeling efficiency is improved, and more features are captured through multi-scale feature maps to output more accurate pedestrian annotation results.
Smart Images
Figure CN118887706B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of Internet technology, and in particular to an image annotation method, device, electronic device and storage medium. Background Art
[0002] With the rapid development of science and technology and the advent of the digital age, the number of pedestrian images has exploded. Whether it is in surveillance cameras in public places or on social media platforms, a large number of pedestrian images can be seen.
[0003] However, there are many pedestrian images at present. If the current pedestrian images are manually annotated one by one, the annotation workload will be very huge and the annotation time will be increased. Therefore, how to obtain the pedestrian annotation results of the current pedestrian images is an urgent problem to be solved. Summary of the invention
[0004] The embodiments of the present application provide an image annotation method, device, electronic device and storage medium to solve the above-mentioned technical problem of how to obtain the pedestrian annotation result of the current pedestrian image.
[0005] In a first aspect, an embodiment of the present application provides an image annotation method, which is applied to an electronic device, wherein the electronic device stores an annotation model, a feature fusion network consisting of a feature extraction network and a bidirectional feature pyramid network, and the image annotation method includes:
[0006] Obtain pedestrian annotation information of a preset pedestrian image, combine the preset pedestrian image and the pedestrian annotation information into a annotation sample, and combine a first number of different annotation samples into a training set of the annotation model, wherein the pedestrian annotation information includes a preset pedestrian region and a preset pedestrian label corresponding to the preset pedestrian image;
[0007] Acquire a first multi-scale feature map of the preset pedestrian image through the feature fusion network, and obtain a prediction result output by the annotation model based on the first multi-scale feature map, wherein the prediction result includes a predicted pedestrian area and a predicted pedestrian label;
[0008] Obtaining a distribution focus loss value and an intersection-over-union loss value between the predicted pedestrian region and the preset pedestrian region, and obtaining a cross entropy loss value between the predicted pedestrian label and the preset pedestrian label;
[0009] Generate an overall loss value of the prediction result according to the distribution focus loss value, the intersection-over-union loss value, the cross entropy loss value and a preset overall loss value generation model;
[0010] Determining whether the overall loss value is less than a preset loss value;
[0011] If the overall loss value is less than the preset loss value, the training of the labeling model is stopped, and the trained labeling model is output;
[0012] A current pedestrian image is obtained, a second multi-scale feature map of the current pedestrian image is obtained through the feature fusion network, and a pedestrian annotation result output by the annotation model based on the second multi-scale feature map is obtained, wherein the pedestrian annotation result includes a pedestrian annotation area and a pedestrian annotation label corresponding to the current pedestrian image.
[0013] Exemplarily, the overall loss value generation model is:
[0014] Loss=max(K1×Loss1,K2×Loss2) + K3×Loss3;
[0015] Wherein, the Loss is the overall loss value, the K1 is the weight coefficient of the distribution focus loss value, the K2 is the weight coefficient of the intersection-over-union loss value, the K3 is the weight coefficient of the cross entropy loss value, and the sum of the K1, the K2, and the K3 is 1;
[0016] Wherein, the Loss1 is the distribution focus loss value, and the Loss2 is the intersection-over-union loss value; the larger the Loss1 and the Loss2, the greater the mismatch between the predicted pedestrian area and the preset pedestrian area; the smaller the Loss1 and the Loss2, the smaller the mismatch between the predicted pedestrian area and the preset pedestrian area;
[0017] Among them, the larger the Loss1 and the Loss2 are, the greater the mismatch between the predicted pedestrian area and the preset pedestrian area; there are two reasons for this. On the one hand, the larger the Loss1 is, the greater the position difference between the bounding box position of the predicted pedestrian area and the bounding box position of the preset pedestrian area is; on the other hand, the larger the Loss2 is, the greater the difference in the degree of overlap between the bounding box position of the predicted pedestrian area and the bounding box position of the preset pedestrian area is;
[0018] The smaller the Loss1 and the Loss2 are, the smaller the mismatch between the predicted pedestrian area and the preset pedestrian area is. There are two reasons for this. On the one hand, the smaller the Loss1 is, the smaller the position difference between the bounding box position of the predicted pedestrian area and the bounding box position of the preset pedestrian area is. On the other hand, the smaller the Loss2 is, the smaller the difference in the degree of overlap between the bounding box position of the predicted pedestrian area and the bounding box position of the preset pedestrian area is.
[0019] Among them, the Loss3 is the cross entropy loss value; the larger the Loss3 is, the greater the difference between the predicted pedestrian label and the preset pedestrian label is; the smaller the Loss3 is, the smaller the difference between the predicted pedestrian label and the preset pedestrian label is.
[0020] In a possible implementation manner of the first aspect, extracting a first feature map of the preset pedestrian image of the training set by the feature extraction network;
[0021] Fusing the plurality of first feature maps through the bidirectional feature pyramid network to obtain the first multi-scale feature map of the preset pedestrian image;
[0022] The first multi-scale feature map is input into the annotation model, and the prediction result output by the annotation model based on the first multi-scale feature map is obtained.
[0023] In a possible implementation manner of the first aspect, if the overall loss value is less than a preset loss value, stopping training the labeling model and outputting the trained labeling model includes:
[0024] If the overall loss value is less than a preset loss value, a hyperparameter space of the annotation model is obtained, where the hyperparameter space includes a value range of a hyperparameter of the annotation model;
[0025] In the hyperparameter space, obtaining a hyperparameter combination consisting of a plurality of the hyperparameters;
[0026] A second different number of labeled samples are used to form a validation set of the labeling model, and the validation set is input into the labeling model corresponding to each of the hyperparameter combinations, and an evaluation index corresponding to each of the hyperparameter combinations is obtained, and the hyperparameter combination corresponding to the optimal evaluation index is set as the optimal hyperparameter combination, and the labeling model using the optimal hyperparameter combination is set as the trained labeling model.
[0027] In a possible implementation manner of the first aspect, acquiring a hyperparameter combination consisting of a plurality of the hyperparameters in the hyperparameter space includes:
[0028] In the hyperparameter space, using a preset Bayesian optimization algorithm, obtain a hyperparameter combination consisting of a plurality of the hyperparameters;
[0029] Alternatively, in the hyperparameter space, a preset random search algorithm is used to obtain a hyperparameter combination consisting of a plurality of the hyperparameters;
[0030] Alternatively, in the hyperparameter space, a preset grid search algorithm is used to obtain a hyperparameter combination consisting of a plurality of the hyperparameters.
[0031] In a possible implementation manner of the first aspect, acquiring the current pedestrian image, acquiring a second multi-scale feature map of the current pedestrian image through the feature fusion network, and acquiring a pedestrian annotation result output by the annotation model based on the second multi-scale feature map includes:
[0032] Acquire a current pedestrian image, and extract a second feature map of the current pedestrian image through the feature extraction network;
[0033] Fusing the plurality of second feature maps through the bidirectional feature pyramid network to obtain the second multi-scale feature map of the current pedestrian image;
[0034] The second multi-scale feature map is input into the annotation model, and the pedestrian annotation result output by the annotation model based on the second multi-scale feature map is obtained.
[0035] In a possible implementation manner of the first aspect, after acquiring the current pedestrian image, acquiring a second multi-scale feature map of the current pedestrian image through the feature fusion network, and acquiring a pedestrian annotation result output by the annotation model based on the second multi-scale feature map, the image annotation method includes:
[0036] The pedestrian marking result is sent to a preset screen, and the screen is controlled to display the pedestrian marking result.
[0037] In a possible implementation manner of the first aspect, the feature extraction network includes one or a combination of an Inception-Resnet network, a convolutional neural network, and a residual neural network.
[0038] In a second aspect, an embodiment of the present application provides an image annotation device, which is applied to an electronic device, wherein the electronic device stores an annotation model, a feature fusion network composed of a feature extraction network and a bidirectional feature pyramid network, including:
[0039] A first acquisition module is used to acquire pedestrian annotation information of a preset pedestrian image, combine the preset pedestrian image and the pedestrian annotation information into a annotation sample, and combine a first number of different annotation samples into a training set of the annotation model, wherein the pedestrian annotation information includes a preset pedestrian region and a preset pedestrian label corresponding to the preset pedestrian image;
[0040] A second acquisition module is used to acquire a first multi-scale feature map of the preset pedestrian image through the feature fusion network, and acquire a prediction result output by the annotation model based on the first multi-scale feature map, wherein the prediction result includes a predicted pedestrian area and a predicted pedestrian label;
[0041] A third acquisition module is used to obtain a distribution focus loss value and an intersection-over-union loss value between the predicted pedestrian area and the preset pedestrian area, and to obtain a cross entropy loss value between the predicted pedestrian label and the preset pedestrian label;
[0042] A generation module, used to generate a model according to the distribution focus loss value, the intersection-over-union loss value, the cross entropy loss value and a preset overall loss value, to generate an overall loss value of the prediction result;
[0043] A judgment module, used to judge whether the overall loss value is less than a preset loss value;
[0044] An output module, used for stopping training the annotation model and outputting the annotation model after training if the overall loss value is less than a preset loss value;
[0045] The annotation module is used to obtain a current pedestrian image, obtain a second multi-scale feature map of the current pedestrian image through the feature fusion network, and obtain a pedestrian annotation result output by the annotation model based on the second multi-scale feature map, wherein the pedestrian annotation result includes a pedestrian annotation area and a pedestrian annotation label corresponding to the current pedestrian image.
[0046] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image annotation method of the first aspect when executing the computer program.
[0047] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image annotation method of the first aspect is implemented.
[0048] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the image annotation method of the first aspect.
[0049] The beneficial effects of the embodiments of the present application lie in two aspects. On the one hand, a current pedestrian image is obtained, and a second multi-scale feature map of the current pedestrian image is obtained through the feature fusion network, and a pedestrian annotation result output by the annotation model based on the second multi-scale feature map is obtained. The pedestrian annotation result includes a pedestrian annotation area and a pedestrian annotation label corresponding to the current pedestrian image. Therefore, the electronic device can obtain the pedestrian annotation area and the pedestrian annotation label corresponding to the current pedestrian image, thereby simplifying the annotation process of the current pedestrian image and helping to improve the annotation efficiency of the current pedestrian image. On the other hand, the second multi-scale feature map can capture more features of the current pedestrian image, which helps the annotation model to output more accurate pedestrian annotation results. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0051] Figure 1 An application scenario diagram of the image annotation method provided in an embodiment of the present application;
[0052] Figure 2 is a flowchart of an image annotation method provided in an embodiment of the present application;
[0053] Figure 3 A flowchart for obtaining the optimal hyperparameter combination provided in an embodiment of the present application;
[0054] Figure 4 A schematic block diagram of an image annotation device provided in an embodiment of the present application;
[0055] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0056] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.
[0057] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0058] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.
[0059] In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.
[0060] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.
[0061] The image annotation method provided in the embodiments of the present application can be applied to electronic devices such as servers, mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.
[0062] See also Figure 1 , Figure 1 The application scenario diagram of the image annotation method provided in the embodiment of the present application is described in detail as follows:
[0063] The electronic device stores a labeling model, a feature fusion network consisting of a feature extraction network and a bidirectional feature pyramid network.
[0064] The feature extraction network includes one or a combination of an Inception-Resnet network, a convolutional neural network, and a residual neural network.
[0065] Wherein, a first multi-scale feature map of a preset pedestrian image is obtained through the feature fusion network.
[0066] The first multi-scale feature map includes features of different scales of a preset pedestrian image.
[0067] Wherein, a second multi-scale feature map of the current pedestrian image is obtained through the feature fusion network.
[0068] The second multi-scale feature map includes features of different scales of the current pedestrian image.
[0069] Among them, the Inception module is a component in a deep learning network structure.
[0070] Among them, the ResNet network is called the Residual Neural Network.
[0071] The Inception module is combined with the residual connection mechanism of ResNet to form the Inception-ResNet network.
[0072] The preset pedestrian image and the current pedestrian image may come from the same database or from different databases.
[0073] For the sake of explanation, the preset pedestrian image and the current pedestrian image are taken as an example from different databases, as detailed below:
[0074] The two databases include database A and database B. The electronic device is connected to database A and database B respectively to obtain the preset pedestrian image stored in database A and the current pedestrian image stored in database B.
[0075] In an embodiment of the present application, the electronic device can be connected to different databases at the same time to obtain preset pedestrian images and current pedestrian images from different databases.
[0076] See also Figure 2 , Figure 2 It is a flow chart of the image annotation method provided in an embodiment of the present application. The method can be applied to an electronic device, which stores an annotation model, a feature fusion network composed of a feature extraction network and a bidirectional feature pyramid network.
[0077] like Figure 2 As shown, the image annotation method provided in the embodiment of the present application includes the following steps, which are described in detail as follows:
[0078] S201, obtaining pedestrian annotation information of a preset pedestrian image, combining the preset pedestrian image and the pedestrian annotation information into a annotation sample, and combining a first number of different annotation samples into a training set of the annotation model, wherein the pedestrian annotation information includes a preset pedestrian region and a preset pedestrian label corresponding to the preset pedestrian image;
[0079] Among them, the preset pedestrian image is a preset pedestrian image.
[0080] The first number is set by the user or by the system by default and is not limited here.
[0081] For the sake of illustration, an example is given below:
[0082] For example, the first number is 10,000, and 10,000 different labeled samples form a training set for the labeled model;
[0083] For example, the first number is 100,000, and 100,000 different labeled samples form a training set for the labeled model.
[0084] The preset pedestrian region is used to identify the position of the pedestrian in the preset pedestrian image. The position of the pedestrian in the preset pedestrian image can be known through the preset pedestrian region.
[0085] Among them, the preset pedestrian labels are user-set or system default.
[0086] For ease of explanation, the preset pedestrian labels include the following:
[0087] Preset existence label: The preset existence label is used to indicate whether a pedestrian exists in a preset pedestrian image.
[0088] The default existence tag has two values, A1 and A2.
[0089] When the preset existence label is A1, it indicates that a pedestrian exists in the preset pedestrian image.
[0090] When the preset existence label is A2, it means that there is no pedestrian in the preset pedestrian image.
[0091] This kind of label is very useful in the pedestrian detection task because it can help the annotation model distinguish between the preset pedestrian images that contain pedestrians and the preset pedestrian images that do not contain pedestrians.
[0092] Preset occlusion degree label: The preset occlusion degree label is used to describe the degree to which pedestrians are obscured by other objects in the preset pedestrian image. The preset occlusion degree label can be a percentage value or a discrete level, indicating the proportion or degree of pedestrian obscuration. The preset occlusion degree label can be used to understand the visibility of pedestrians in the preset pedestrian image.
[0093] Preset posture label: The preset posture label is used to describe the body posture of pedestrians in the preset pedestrian image. Body postures include walking, standing, squatting and other different postures.
[0094] Preset attribute labels: Preset attribute labels are used to describe various attributes of pedestrians in preset pedestrian images, such as gender, age, clothing color, etc. These preset attribute labels can help the annotation model to more comprehensively understand the characteristics of pedestrians in preset pedestrian images.
[0095] Preset tracking tags: Preset tracking tags are used to identify and track the same pedestrian in the preset pedestrian image. When the pedestrian moves between consecutive preset pedestrian images, the preset tracking tags can ensure that the annotation model can continue to track and identify the pedestrian in the preset pedestrian image.
[0096] Among them, the labeling model can adopt a pre-trained model. Since the pre-trained model has been preliminarily learned on a large amount of data and has certain feature extraction and representation capabilities, the labeling model adopts the pre-trained model to reduce training time.
[0097] S202, obtaining a first multi-scale feature map of the preset pedestrian image through the feature fusion network, and obtaining a prediction result output by the annotation model based on the first multi-scale feature map, wherein the prediction result includes a predicted pedestrian area and a predicted pedestrian label;
[0098] The step of obtaining a first multi-scale feature map of the preset pedestrian image through the feature fusion network and obtaining a prediction result output by the annotation model based on the first multi-scale feature map includes:
[0099] Extracting a first feature map of the preset pedestrian image of the training set through the feature extraction network;
[0100] Fusing the plurality of first feature maps through the bidirectional feature pyramid network to obtain the first multi-scale feature map of the preset pedestrian image;
[0101] The first multi-scale feature map is input into the annotation model, and the prediction result output by the annotation model based on the first multi-scale feature map is obtained.
[0102] S203, obtaining a distribution focus loss value and an intersection-over-union loss value between the predicted pedestrian region and the preset pedestrian region, and obtaining a cross entropy loss value between the predicted pedestrian label and the preset pedestrian label;
[0103] S204, generating a model according to the distribution focus loss value, the intersection-over-union loss value, the cross entropy loss value and a preset overall loss value to generate an overall loss value of the prediction result;
[0104] Exemplarily, the overall loss value generation model is:
[0105] Loss=max(K1×Loss1,K2×Loss2) + K3×Loss3;
[0106] Wherein, the Loss is the overall loss value, the K1 is the weight coefficient of the distribution focus loss value, the K2 is the weight coefficient of the intersection-over-union loss value, the K3 is the weight coefficient of the cross entropy loss value, and the sum of the K1, the K2, and the K3 is 1;
[0107] Wherein, the Loss1 is the distribution focus loss value, and the Loss2 is the intersection-over-union loss value; the larger the Loss1 and the Loss2, the greater the mismatch between the predicted pedestrian area and the preset pedestrian area; the smaller the Loss1 and the Loss2, the smaller the mismatch between the predicted pedestrian area and the preset pedestrian area;
[0108] Among them, the larger the Loss1 and the Loss2 are, the greater the mismatch between the predicted pedestrian area and the preset pedestrian area; there are two reasons for this. On the one hand, the larger the Loss1 is, the greater the position difference between the bounding box position of the predicted pedestrian area and the bounding box position of the preset pedestrian area is; on the other hand, the larger the Loss2 is, the greater the difference in the degree of overlap between the bounding box position of the predicted pedestrian area and the bounding box position of the preset pedestrian area is;
[0109] The smaller the Loss1 and the Loss2 are, the smaller the mismatch between the predicted pedestrian area and the preset pedestrian area is. There are two reasons for this. On the one hand, the smaller the Loss1 is, the smaller the position difference between the bounding box position of the predicted pedestrian area and the bounding box position of the preset pedestrian area is. On the other hand, the smaller the Loss2 is, the smaller the difference in the degree of overlap between the bounding box position of the predicted pedestrian area and the bounding box position of the preset pedestrian area is.
[0110] Among them, the Loss3 is the cross entropy loss value; the larger the Loss3 is, the greater the difference between the predicted pedestrian label and the preset pedestrian label is; the smaller the Loss3 is, the smaller the difference between the predicted pedestrian label and the preset pedestrian label is.
[0111] Among them, through the max function, the labeling model can make a selection based on the maximum value of the distribution focus loss value and the intersection-over-union loss value, and the maximum value can be used to more easily determine the maximum mismatch between the predicted pedestrian area and the preset pedestrian area.
[0112] Among them, K1, K2, and K3 are set by the user or by system default, and are not limited here.
[0113] For the sake of illustration, an example is given below:
[0114] For example, K1 is 0.3, K2 is 0.35, and K3 is 0.35. The overall loss value generation model is:
[0115] Loss= max(0.3×Loss1, 0.35×Loss2) + 0.35×Loss3;
[0116] For example, K1 is 0.3, K2 is 0.32, and K3 is 0.38, and the overall loss value generation model is:
[0117] Loss= max(0.3×Loss1, 0.32×Loss2) + 0.38×Loss3;
[0118] Among them, the overall loss value reflects the degree of mismatch between the predicted pedestrian area and the preset pedestrian area, and the difference between the predicted pedestrian label and the preset pedestrian label. Therefore, the overall loss value can guide users to improve and optimize the model to achieve better prediction results.
[0119] S205, determining whether the overall loss value is less than a preset loss value;
[0120] Among them, the preset loss value is set by the user or by the system default, and is not restricted here.
[0121] S206, if the overall loss value is less than a preset loss value, stop training the annotation model and output the trained annotation model;
[0122] If the overall loss value is not less than the preset loss value, the process jumps to step 202 and steps 202 to S205 are repeated to iteratively train the labeling model.
[0123] S207, obtaining a current pedestrian image, obtaining a second multi-scale feature map of the current pedestrian image through the feature fusion network, and obtaining a pedestrian annotation result output by the annotation model based on the second multi-scale feature map, wherein the pedestrian annotation result includes a pedestrian annotation area and a pedestrian annotation label corresponding to the current pedestrian image.
[0124] Among them, obtaining the current pedestrian image includes:
[0125] Read the preset marking time and determine whether the current time is the marking time;
[0126] If the current time is the marked time, the preset monitoring system is connected via the preset network.
[0127] The current pedestrian image of the monitoring system is obtained, and the preset network includes one or a combination of Ethernet, 4G network, 5G network, and WIFI network.
[0128] Among them, WIFI network is a wireless fidelity network.
[0129] Among them, 4G network is the fourth generation mobile communication technology network.
[0130] Among them, 5G network is the fifth generation mobile communication technology network.
[0131] Among them, the current pedestrian image is the current pedestrian image.
[0132] The current pedestrian region is used to identify the position of the pedestrian in the current pedestrian image. The position of the pedestrian in the current pedestrian image can be known through the current pedestrian region.
[0133] Among them, the current pedestrian label is set by the user or by system default.
[0134] For ease of explanation, the current pedestrian labels include the following:
[0135] Current presence label: The current presence label is used to indicate whether there is a pedestrian in the current pedestrian image.
[0136] The current existence label has two values, namely B1 and B2. When the current existence label is B1, it means that there is a pedestrian in the current pedestrian image.
[0137] When the current existence label is B2, it means that there is no pedestrian in the current pedestrian image.
[0138] This kind of label is very useful in the pedestrian detection task because it can help the annotation model distinguish between the current pedestrian images that contain pedestrians and the current pedestrian images that do not contain pedestrians.
[0139] Current occlusion degree label: The current occlusion degree label is used to describe the degree to which the pedestrian is occluded by other objects in the current pedestrian image. The current occlusion degree label can be a percentage value or a discrete level, indicating the proportion or degree of occlusion of the pedestrian. The current occlusion degree label can be used to understand the visibility of the pedestrian in the current pedestrian image.
[0140] Current posture label: The current posture label is used to describe the body posture of the pedestrian in the current pedestrian image. Body postures include walking, standing, squatting and other different postures.
[0141] Current attribute labels: Current attribute labels are used to describe various attributes of pedestrians in the current pedestrian image, such as gender, age, clothing color, etc. These current attribute labels can help the annotation model to more comprehensively understand the characteristics of pedestrians in the current pedestrian image.
[0142] Current tracking label: The current tracking label is used to identify and track the same pedestrian in the current pedestrian image. When the pedestrian moves between consecutive current pedestrian images, the current tracking label can ensure that the annotation model can continue to track and identify the pedestrian in the current pedestrian image.
[0143] The step of acquiring the current pedestrian image, acquiring a second multi-scale feature map of the current pedestrian image through the feature fusion network, and acquiring a pedestrian annotation result output by the annotation model based on the second multi-scale feature map includes:
[0144] Acquire a current pedestrian image, and extract a second feature map of the current pedestrian image through the feature extraction network;
[0145] Fusing the plurality of second feature maps through the bidirectional feature pyramid network to obtain the second multi-scale feature map of the current pedestrian image;
[0146] The second multi-scale feature map is input into the annotation model, and the pedestrian annotation result output by the annotation model based on the second multi-scale feature map is obtained.
[0147] Wherein, after acquiring the current pedestrian image, acquiring the second multi-scale feature map of the current pedestrian image through the feature fusion network, and acquiring the pedestrian annotation result output by the annotation model based on the second multi-scale feature map, the image annotation method includes:
[0148] The pedestrian marking result is sent to a preset screen, and the screen is controlled to display the pedestrian marking result.
[0149] The beneficial effects of the embodiments of the present application lie in two aspects. On the one hand, a current pedestrian image is obtained, and a second multi-scale feature map of the current pedestrian image is obtained through the feature fusion network, and a pedestrian annotation result output by the annotation model based on the second multi-scale feature map is obtained. The pedestrian annotation result includes a pedestrian annotation area and a pedestrian annotation label corresponding to the current pedestrian image. Therefore, the electronic device can obtain the pedestrian annotation area and the pedestrian annotation label corresponding to the current pedestrian image, thereby simplifying the annotation process of the current pedestrian image and helping to improve the annotation efficiency of the current pedestrian image. On the other hand, the second multi-scale feature map can capture more features of the current pedestrian image, which helps the annotation model to output more accurate pedestrian annotation results.
[0150] See also Figure 3 , Figure 3 The flowchart for obtaining the optimal hyperparameter combination provided in the embodiment of the present application is described in detail as follows:
[0151] S301, if the overall loss value is less than a preset loss value, obtaining a hyperparameter space of the annotation model, wherein the hyperparameter space includes a value range of a hyperparameter of the annotation model;
[0152] The value range of the hyperparameters is user-defined or system-default, and is not restricted here.
[0153] For ease of explanation, the range of values for hyperparameters includes the following:
[0154] Learning rate: 0.001 to 0.1;
[0155] Batch size: 32 to 256;
[0156] Regularization parameter: 0.0001 to 0.1;
[0157] Number of hidden layers: 1 to 3;
[0158] Hidden layer size: 64 to 512 neurons.
[0159] S302, in the hyperparameter space, obtaining a hyperparameter combination consisting of a plurality of the hyperparameters;
[0160] Wherein, in the hyperparameter space, obtaining a hyperparameter combination consisting of a plurality of the hyperparameters comprises:
[0161] In the hyperparameter space, using a preset Bayesian optimization algorithm, obtain a hyperparameter combination consisting of a plurality of the hyperparameters;
[0162] Alternatively, in the hyperparameter space, a preset random search algorithm is used to obtain a hyperparameter combination consisting of a plurality of the hyperparameters;
[0163] Alternatively, in the hyperparameter space, a preset grid search algorithm is used to obtain a hyperparameter combination consisting of a plurality of the hyperparameters.
[0164] S303, forming a verification set of the annotation model with a second different number of the annotation samples, inputting the verification set into the annotation model corresponding to each of the hyperparameter combinations, obtaining an evaluation index corresponding to each of the hyperparameter combinations, setting the hyperparameter combination corresponding to the best evaluation index as the best hyperparameter combination, and setting the annotation model using the best hyperparameter combination as the trained annotation model.
[0165] The second number is set by the user or by the system by default and is not limited here.
[0166] For the sake of illustration, an example is given below:
[0167] For example, the second number is 2,000, and 2,000 different labeled samples form a validation set of the labeled model;
[0168] For example, the second number is 20,000, and 20,000 different labeled samples constitute a validation set of the labeled model.
[0169] The evaluation indicators corresponding to the hyperparameter combinations are set by the user or by the system by default, and are not restricted here.
[0170] For ease of explanation, the evaluation indicators corresponding to the hyperparameter combinations include the following:
[0171] Accuracy: The ratio of correctly predicted samples to the total number of samples.
[0172] Precision: The proportion of samples predicted to be positive that are actually positive.
[0173] Recall rate: The proportion of samples that are predicted to be positive among samples that are actually positive.
[0174] F1 score: The harmonic mean of precision and recall, used to measure the accuracy of model predictions.
[0175] The optimal hyperparameter combination corresponding to the optimal evaluation index is set as the optimal hyperparameter combination. For ease of explanation, an example is given below:
[0176] There are 3 hyperparameter combinations: hyperparameter combination 1, hyperparameter combination 2, and hyperparameter combination 3.
[0177] Evaluation indicators of hyperparameter combination 1:
[0178] The accuracy is 0.85; the precision is 0.88; the recall is 0.80; and the F1 score is 0.84;
[0179] Evaluation indicators of hyperparameter combination 2:
[0180] The accuracy is 0.90; the precision is 0.95; the recall is 0.91; the F1 score is 0.90;
[0181] Evaluation indicators of hyperparameter combination 3:
[0182] The accuracy is 0.88; the precision is 0.90; the recall is 0.85; and the F1 score is 0.87.
[0183] Among them, the optimal evaluation index is the evaluation index of hyperparameter combination 2, so the annotation model using hyperparameter combination 2 is set as the trained annotation model.
[0184] In the embodiments of the present application, by using the optimal hyperparameter combination, the performance of the labeling model can be improved, thereby bringing greater value to users and enterprises.
[0185] For the image annotation method described in the above embodiment, please refer to Figure 4 , Figure 4 A schematic block diagram of an image annotation device provided in an embodiment of the present application. Figure 4 The image annotation apparatus 400 shown can be applied to Figure 1 The electronic device in the application scenario diagram shown below takes the electronic device as an example. Figure 4 The image annotation device 400 shown in the figure is described in detail. The image annotation device 400 may include a first acquisition module 401, a second acquisition module 402, a third acquisition module 403, a generation module 404, a judgment module 405, an output module 406, and a annotation module 407.
[0186] A first acquisition module 401 is used to acquire pedestrian annotation information of a preset pedestrian image, combine the preset pedestrian image and the pedestrian annotation information into a annotation sample, and combine a first number of different annotation samples into a training set of the annotation model, wherein the pedestrian annotation information includes a preset pedestrian region and a preset pedestrian label corresponding to the preset pedestrian image;
[0187] A second acquisition module 402 is used to acquire a first multi-scale feature map of the preset pedestrian image through the feature fusion network, and acquire a prediction result output by the annotation model based on the first multi-scale feature map, wherein the prediction result includes a predicted pedestrian area and a predicted pedestrian label;
[0188] The third acquisition module 403 is used to obtain the distribution focus loss value and the intersection-over-union loss value between the predicted pedestrian area and the preset pedestrian area, and obtain the cross entropy loss value between the predicted pedestrian label and the preset pedestrian label;
[0189] A generation module 404 is used to generate a model according to the distribution focus loss value, the intersection-over-union loss value, the cross entropy loss value and a preset overall loss value to generate an overall loss value of the prediction result;
[0190] A judgment module 405 is used to judge whether the overall loss value is less than a preset loss value;
[0191] An output module 406 is used to stop training the annotation model and output the annotation model after training if the overall loss value is less than a preset loss value;
[0192] The annotation module 407 is used to obtain the current pedestrian image, obtain the second multi-scale feature map of the current pedestrian image through the feature fusion network, and obtain the pedestrian annotation result output by the annotation model based on the second multi-scale feature map, wherein the pedestrian annotation result includes the pedestrian annotation area and the pedestrian annotation label corresponding to the current pedestrian image.
[0193] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0194] The beneficial effects of the embodiments of the present application lie in two aspects. On the one hand, a current pedestrian image is obtained, and a second multi-scale feature map of the current pedestrian image is obtained through the feature fusion network, and a pedestrian annotation result output by the annotation model based on the second multi-scale feature map is obtained. The pedestrian annotation result includes a pedestrian annotation area and a pedestrian annotation label corresponding to the current pedestrian image. Therefore, the electronic device can obtain the pedestrian annotation area and the pedestrian annotation label corresponding to the current pedestrian image, thereby simplifying the annotation process of the current pedestrian image and helping to improve the annotation efficiency of the current pedestrian image. On the other hand, the second multi-scale feature map can capture more features of the current pedestrian image, which helps the annotation model to output more accurate pedestrian annotation results.
[0195] See also Figure 5 , Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
[0196] like Figure 5 As shown, Figure 5 The electronic device 2 includes: at least one processor 20, a memory 21, and a computer program 22 stored in the memory 21 and executable on the at least one processor 20, and when the processor 20 executes the computer program 22, the steps in any of the above-mentioned method embodiments are implemented.
[0197] The electronic device 2 may include, but is not limited to, a processor 20 and a memory 21. Those skilled in the art will appreciate that Figure 5 It is only an example of the electronic device 2 and does not constitute a limitation on the electronic device 2. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.
[0198] The processor 20 is used to run the computer program 22 stored in the memory 21, and implement the following steps when executing the computer program 22:
[0199] Obtain pedestrian annotation information of a preset pedestrian image, combine the preset pedestrian image and the pedestrian annotation information into a annotation sample, and combine a first number of different annotation samples into a training set of the annotation model, wherein the pedestrian annotation information includes a preset pedestrian region and a preset pedestrian label corresponding to the preset pedestrian image;
[0200] Acquire a first multi-scale feature map of the preset pedestrian image through the feature fusion network, and obtain a prediction result output by the annotation model based on the first multi-scale feature map, wherein the prediction result includes a predicted pedestrian area and a predicted pedestrian label;
[0201] Obtaining a distribution focus loss value and an intersection-over-union loss value between the predicted pedestrian region and the preset pedestrian region, and obtaining a cross entropy loss value between the predicted pedestrian label and the preset pedestrian label;
[0202] Generate an overall loss value of the prediction result according to the distribution focus loss value, the intersection-over-union loss value, the cross entropy loss value and a preset overall loss value generation model;
[0203] Determine whether the overall loss value is less than a preset loss value; if the overall loss value is less than the preset loss value, stop training the annotation model and output the trained annotation model; obtain a current pedestrian image, obtain a second multi-scale feature map of the current pedestrian image through the feature fusion network, and obtain a pedestrian annotation result output by the annotation model based on the second multi-scale feature map, wherein the pedestrian annotation result includes a pedestrian annotation area and a pedestrian annotation label corresponding to the current pedestrian image.
[0204] In some embodiments, the processor 20 is configured to implement:
[0205] Extracting a first feature map of the preset pedestrian image of the training set through the feature extraction network;
[0206] Fusing the plurality of first feature maps through the bidirectional feature pyramid network to obtain the first multi-scale feature map of the preset pedestrian image;
[0207] The first multi-scale feature map is input into the annotation model, and the prediction result output by the annotation model based on the first multi-scale feature map is obtained.
[0208] In some embodiments, the processor 20 is configured to implement:
[0209] If the overall loss value is less than a preset loss value, a hyperparameter space of the annotation model is obtained, where the hyperparameter space includes a value range of a hyperparameter of the annotation model;
[0210] In the hyperparameter space, obtaining a hyperparameter combination consisting of a plurality of the hyperparameters;
[0211] A second different number of labeled samples are used to form a validation set of the labeling model, and the validation set is input into the labeling model corresponding to each of the hyperparameter combinations, and an evaluation index corresponding to each of the hyperparameter combinations is obtained, and the hyperparameter combination corresponding to the optimal evaluation index is set as the optimal hyperparameter combination, and the labeling model using the optimal hyperparameter combination is set as the trained labeling model.
[0212] In some embodiments, the processor 20 is configured to implement:
[0213] In the hyperparameter space, using a preset Bayesian optimization algorithm, obtain a hyperparameter combination consisting of a plurality of the hyperparameters;
[0214] Alternatively, in the hyperparameter space, a preset random search algorithm is used to obtain a hyperparameter combination consisting of a plurality of the hyperparameters;
[0215] Alternatively, in the hyperparameter space, a preset grid search algorithm is used to obtain a hyperparameter combination consisting of a plurality of the hyperparameters.
[0216] In some embodiments, the processor 20 is configured to implement:
[0217] Acquire a current pedestrian image, and extract a second feature map of the current pedestrian image through the feature extraction network;
[0218] Fusing the plurality of second feature maps through the bidirectional feature pyramid network to obtain the second multi-scale feature map of the current pedestrian image;
[0219] The second multi-scale feature map is input into the annotation model, and the pedestrian annotation result output by the annotation model based on the second multi-scale feature map is obtained.
[0220] In some embodiments, the processor 20 is configured to implement:
[0221] The pedestrian marking result is sent to a preset screen, and the screen is controlled to display the pedestrian marking result.
[0222] In some embodiments, the processor 20 is configured to implement:
[0223] The feature extraction network includes one of an Inception-Resnet network, a convolutional neural network, and a residual neural network, or a combination thereof.
[0224] The processor 20 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0225] In some embodiments, the memory 21 may be an internal storage unit of the electronic device 2, such as a hard disk or memory of the electronic device 2. In other embodiments, the memory 21 may also be an external storage device of the electronic device 2, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 2. Further, the memory 21 may also include both an internal storage unit and an external storage device of the electronic device 2. The memory 21 is used to store an operating system, an application program, a boot loader, data, and other programs, such as the program code of the computer program. The memory 21 may also be used to temporarily store data that has been output or is to be output.
[0226] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.
[0227] An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.
[0228] The computer-readable storage medium stores program codes, which can be called by a processor to execute the image annotation method described in the above method embodiment.
[0229] The computer-readable storage medium has a storage space for program codes.
[0230] The program code includes the code of any step in the image annotation method described in the above method embodiment.
[0231] For example, the program code is called by the processor and can execute the following steps:
[0232] Obtain pedestrian annotation information of a preset pedestrian image, combine the preset pedestrian image and the pedestrian annotation information into a annotation sample, and combine a first number of different annotation samples into a training set of the annotation model, wherein the pedestrian annotation information includes a preset pedestrian region and a preset pedestrian label corresponding to the preset pedestrian image;
[0233] Acquire a first multi-scale feature map of the preset pedestrian image through the feature fusion network, and obtain a prediction result output by the annotation model based on the first multi-scale feature map, wherein the prediction result includes a predicted pedestrian area and a predicted pedestrian label;
[0234] Obtaining a distribution focus loss value and an intersection-over-union loss value between the predicted pedestrian region and the preset pedestrian region, and obtaining a cross entropy loss value between the predicted pedestrian label and the preset pedestrian label;
[0235] Generate an overall loss value of the prediction result according to the distribution focus loss value, the intersection-over-union loss value, the cross entropy loss value and a preset overall loss value generation model;
[0236] Determining whether the overall loss value is less than a preset loss value;
[0237] If the overall loss value is less than the preset loss value, the training of the labeling model is stopped, and the trained labeling model is output;
[0238] A current pedestrian image is obtained, a second multi-scale feature map of the current pedestrian image is obtained through the feature fusion network, and a pedestrian annotation result output by the annotation model based on the second multi-scale feature map is obtained, wherein the pedestrian annotation result includes a pedestrian annotation area and a pedestrian annotation label corresponding to the current pedestrian image.
[0239] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.
[0240] The computer-readable storage medium may also be an external storage device of the image annotation device or electronic device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), a non-transitory computer-readable storage medium, etc., equipped on the image annotation device or electronic device.
[0241] Since the computer program stored in the computer-readable storage medium can execute any one of the image annotation methods provided in the embodiments of the present application, the computer-readable storage medium can achieve the beneficial effects that can be achieved by any one of the image annotation methods provided in the embodiments of the present application. Please refer to the previous embodiments for details and will not be repeated here.
[0242] An embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the above-mentioned image annotation method.
[0243] The computer program product is loaded into the electronic device and can execute the following steps:
[0244] Obtain pedestrian annotation information of a preset pedestrian image, combine the preset pedestrian image and the pedestrian annotation information into a annotation sample, and combine a first number of different annotation samples into a training set of the annotation model, wherein the pedestrian annotation information includes a preset pedestrian region and a preset pedestrian label corresponding to the preset pedestrian image;
[0245] Acquire a first multi-scale feature map of the preset pedestrian image through the feature fusion network, and obtain a prediction result output by the annotation model based on the first multi-scale feature map, wherein the prediction result includes a predicted pedestrian area and a predicted pedestrian label;
[0246] Obtaining a distribution focus loss value and an intersection-over-union loss value between the predicted pedestrian region and the preset pedestrian region, and obtaining a cross entropy loss value between the predicted pedestrian label and the preset pedestrian label;
[0247] Generate an overall loss value of the prediction result according to the distribution focus loss value, the intersection-over-union loss value, the cross entropy loss value and a preset overall loss value generation model;
[0248] Determining whether the overall loss value is less than a preset loss value;
[0249] If the overall loss value is less than the preset loss value, the training of the labeling model is stopped, and the trained labeling model is output;
[0250] A current pedestrian image is obtained, a second multi-scale feature map of the current pedestrian image is obtained through the feature fusion network, and a pedestrian annotation result output by the annotation model based on the second multi-scale feature map is obtained, wherein the pedestrian annotation result includes a pedestrian annotation area and a pedestrian annotation label corresponding to the current pedestrian image.
[0251] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
[0252] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0253] Based on such understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to an electronic device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0254] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. An image annotation method, characterized in that: Applied to an electronic device, the electronic device stores a labeling model, a feature fusion network composed of a feature extraction network and a bidirectional feature pyramid network, and the image labeling method includes: Obtain pedestrian annotation information of a preset pedestrian image, combine the preset pedestrian image and the pedestrian annotation information into a annotation sample, and combine a first number of different annotation samples into a training set of the annotation model, wherein the pedestrian annotation information includes a preset pedestrian region and a preset pedestrian label corresponding to the preset pedestrian image; Acquire a first multi-scale feature map of the preset pedestrian image through the feature fusion network, and obtain a prediction result output by the annotation model based on the first multi-scale feature map, wherein the prediction result includes a predicted pedestrian area and a predicted pedestrian label; Obtaining a distribution focus loss value and an intersection-over-union loss value between the predicted pedestrian region and the preset pedestrian region, and obtaining a cross entropy loss value between the predicted pedestrian label and the preset pedestrian label; Generate an overall loss value of the prediction result according to the distribution focus loss value, the intersection-over-union loss value, the cross entropy loss value and a preset overall loss value generation model; Determining whether the overall loss value is less than a preset loss value; If the overall loss value is less than the preset loss value, the training of the labeling model is stopped, and the trained labeling model is output; Acquire a current pedestrian image, acquire a second multi-scale feature map of the current pedestrian image through the feature fusion network, and acquire a pedestrian annotation result output by the annotation model based on the second multi-scale feature map, wherein the pedestrian annotation result includes a pedestrian annotation area and a pedestrian annotation label corresponding to the current pedestrian image; The overall loss value generation model is: Loss=max(K1×Loss1, K2×Loss2)+K3×Loss3; Wherein, the Loss is the overall loss value, the K1 is the weight coefficient of the distribution focus loss value, the K2 is the weight coefficient of the intersection-over-union loss value, the K3 is the weight coefficient of the cross entropy loss value, and the sum of the K1, the K2, and the K3 is 1; Wherein, the Loss1 is the distribution focus loss value, and the Loss2 is the intersection-over-union loss value; the larger the Loss1 and the Loss2, the greater the mismatch between the predicted pedestrian area and the preset pedestrian area; the smaller the Loss1 and the Loss2, the smaller the mismatch between the predicted pedestrian area and the preset pedestrian area; The current pedestrian labels include the following: Current existence label: The current existence label is used to indicate whether there is a pedestrian in the current pedestrian image; The current existence tag has two values, B1 and B2. When the current existence label is B1, it means that there is a pedestrian in the current pedestrian image; When the current existence label is B2, it means that there is no pedestrian in the current pedestrian image; Current pose label: The current pose label is used to describe the body pose of the pedestrian in the current pedestrian image.
2. The image annotation method according to claim 1, characterized in that: The step of obtaining a first multi-scale feature map of the preset pedestrian image through the feature fusion network, and obtaining a prediction result output by the annotation model based on the first multi-scale feature map, includes: Extracting a first feature map of the preset pedestrian image of the training set through the feature extraction network; Fusing the plurality of first feature maps through the bidirectional feature pyramid network to obtain the first multi-scale feature map of the preset pedestrian image; The first multi-scale feature map is input into the annotation model, and the prediction result output by the annotation model based on the first multi-scale feature map is obtained.
3. The image annotation method according to claim 1, characterized in that: If the overall loss value is less than a preset loss value, the training of the labeling model is stopped, and the trained labeling model is outputted, including: If the overall loss value is less than a preset loss value, a hyperparameter space of the annotation model is obtained, where the hyperparameter space includes a value range of a hyperparameter of the annotation model; In the hyperparameter space, obtaining a hyperparameter combination consisting of a plurality of the hyperparameters; A second different number of labeled samples are used to form a validation set of the labeling model, and the validation set is input into the labeling model corresponding to each of the hyperparameter combinations, and an evaluation index corresponding to each of the hyperparameter combinations is obtained, and the hyperparameter combination corresponding to the optimal evaluation index is set as the optimal hyperparameter combination, and the labeling model using the optimal hyperparameter combination is set as the trained labeling model.
4. The image annotation method according to claim 3, characterized in that: The step of obtaining a hyperparameter combination consisting of a plurality of the hyperparameters in the hyperparameter space includes: In the hyperparameter space, using a preset Bayesian optimization algorithm, obtain a hyperparameter combination consisting of a plurality of the hyperparameters; Alternatively, in the hyperparameter space, a preset random search algorithm is used to obtain a hyperparameter combination consisting of a plurality of the hyperparameters; Alternatively, in the hyperparameter space, a preset grid search algorithm is used to obtain a hyperparameter combination consisting of a plurality of the hyperparameters.
5. The image annotation method according to claim 1, characterized in that: The step of acquiring a current pedestrian image, acquiring a second multi-scale feature map of the current pedestrian image through the feature fusion network, and acquiring a pedestrian annotation result output by the annotation model based on the second multi-scale feature map includes: Acquire a current pedestrian image, and extract a second feature map of the current pedestrian image through the feature extraction network; Fusing the plurality of second feature maps through the bidirectional feature pyramid network to obtain the second multi-scale feature map of the current pedestrian image; The second multi-scale feature map is input into the annotation model, and the pedestrian annotation result output by the annotation model based on the second multi-scale feature map is obtained.
6. The image annotation method according to claim 1, characterized in that: After obtaining the current pedestrian image, obtaining the second multi-scale feature map of the current pedestrian image through the feature fusion network, and obtaining the pedestrian annotation result output by the annotation model based on the second multi-scale feature map, the image annotation method includes: The pedestrian marking result is sent to a preset screen, and the screen is controlled to display the pedestrian marking result.
7. The image annotation method according to any one of claims 1 to 6, characterized in that: The feature extraction network includes one of an Inception-Resnet network, a convolutional neural network, and a residual neural network, or a combination thereof.
8. An image annotation device, characterized in that: Applied to an electronic device, the electronic device stores a labeling model, a feature fusion network consisting of a feature extraction network and a bidirectional feature pyramid network, including: A first acquisition module is used to acquire pedestrian annotation information of a preset pedestrian image, combine the preset pedestrian image and the pedestrian annotation information into a annotation sample, and combine a first number of different annotation samples into a training set of the annotation model, wherein the pedestrian annotation information includes a preset pedestrian region and a preset pedestrian label corresponding to the preset pedestrian image; A second acquisition module is used to acquire a first multi-scale feature map of the preset pedestrian image through the feature fusion network, and acquire a prediction result output by the annotation model based on the first multi-scale feature map, wherein the prediction result includes a predicted pedestrian area and a predicted pedestrian label; A third acquisition module is used to obtain a distribution focus loss value and an intersection-over-union loss value between the predicted pedestrian area and the preset pedestrian area, and to obtain a cross entropy loss value between the predicted pedestrian label and the preset pedestrian label; A generation module, used to generate a model according to the distribution focus loss value, the intersection-over-union loss value, the cross entropy loss value and a preset overall loss value, to generate an overall loss value of the prediction result; A judgment module, used to judge whether the overall loss value is less than a preset loss value; An output module, used for stopping training the annotation model and outputting the annotation model after training if the overall loss value is less than a preset loss value; A labeling module, used to obtain a current pedestrian image, obtain a second multi-scale feature map of the current pedestrian image through the feature fusion network, and obtain a pedestrian labeling result output by the labeling model based on the second multi-scale feature map, wherein the pedestrian labeling result includes a pedestrian labeling area and a pedestrian labeling label corresponding to the current pedestrian image; The overall loss value generation model is: Loss=max(K1×Loss1, K2×Loss2)+K3×Loss3; Wherein, the Loss is the overall loss value, the K1 is the weight coefficient of the distribution focus loss value, the K2 is the weight coefficient of the intersection-over-union loss value, the K3 is the weight coefficient of the cross entropy loss value, and the sum of the K1, the K2, and the K3 is 1; Wherein, the Loss1 is the distribution focus loss value, and the Loss2 is the intersection-over-union loss value; the larger the Loss1 and the Loss2, the greater the mismatch between the predicted pedestrian area and the preset pedestrian area; the smaller the Loss1 and the Loss2, the smaller the mismatch between the predicted pedestrian area and the preset pedestrian area; The current pedestrian labels include the following: Current existence label: The current existence label is used to indicate whether there is a pedestrian in the current pedestrian image; The current existence tag has two values, B1 and B2. When the current existence label is B1, it means that there is a pedestrian in the current pedestrian image; When the current existence label is B2, it means that there is no pedestrian in the current pedestrian image; Current pose label: The current pose label is used to describe the body pose of the pedestrian in the current pedestrian image.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the image annotation method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image annotation method according to any one of claims 1 to 7 is implemented.