A method, apparatus, device and medium for extracting image feature points

Abstract

The application discloses a method, device, equipment and medium for extracting image feature points. A fixed-point detection model is iterated based on a training image to obtain an output feature point and a first loss value. In a first training stage, a pre-set attention point is selected as a target attention point. When the first training stage reaches a corresponding iteration number, a second training stage is entered. The second training stage selects the target attention point based on a detection effect. A second loss value is determined based on a target weight matched by the target attention point, a target channel output feature and a label feature corresponding to the target attention point. The weight of the fixed-point detection model is updated by using the first loss value and the second loss value. The latest fixed-point detection model is iteratively trained until the training result of the latest fixed-point detection model meets a set requirement. The fixed-point detection model is used to detect a to-be-detected image to obtain a feature point of the to-be-detected image, thereby improving the fixed-point detection effect of the image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a method and device for extracting feature points of an image, equipment and a computer readable storage medium. BACKGROUND

[0002] OpenPose human posture recognition is a kind of pedestrian positioning method based on convolutional neural network and supervised learning, which can realize posture estimation of human action, facial expression, finger movement, etc.

[0003] Under the influence of different scenes and various factors, the detection effect of OpenPose will be affected to a certain extent. For example, in the street scene, if it is a hot day, more women will wear hats, carry umbrellas and wear skirts. In such a scene, due to the obstruction of hats, skirts, etc., the positioning effect of OpenPose on the head, shoulders and lower limbs is poor. In cold weather, more pedestrians will put their hands in their pockets, which will cause OpenPose to have poor detection effect on the upper limbs.

[0004] It can be seen that how to improve the positioning detection effect of the image is a problem to be solved by those skilled in the art. SUMMARY

[0005] The purpose of the embodiments of the present application is to provide a method and device for extracting feature points of an image, equipment and a computer readable storage medium, which can improve the positioning detection effect of the image.

[0006] To solve the above technical problems, the embodiments of the present application provide a method for extracting feature points of an image, comprising:

[0007] iterating the positioning detection model based on the training image to obtain output feature points and a first loss value;

[0008] selecting a target focus point from the output feature points based on the focus point selection rule corresponding to the training stage; wherein the target focus point of the first training stage is a pre-set focus point; the target focus point of the second training stage is a target focus point selected based on the detection effect of the positioning detection model completing the first training stage on each output feature point; the first training stage and the second training stage have their respective iteration numbers;

[0009] determining a second loss value based on the target weight matched by the target focus point, the target channel output feature and the label feature corresponding to the target focus point;

[0010] updating the weights of the fixed-point detection model by using the first loss value and the second loss value, taking the updated fixed-point detection model as the latest fixed-point detection model, and returning to the step of iteratively training the fixed-point detection model based on the training image to obtain the output feature point and the first loss value;

[0011] When the first training phase reaches its corresponding number of iterations, the second training phase is entered, and the training result of the latest fixed-point detection model is used to detect the to-be-detected image to obtain the feature points of the to-be-detected image until the set requirement is met.

[0012] Optionally, the second loss value is determined based on the target weight matched with the target focus point, the target channel output feature, and the label feature corresponding to the target focus point.

[0013] The target weight and the target channel output feature matched with the target focus point are extracted from the fixed-point detection model.

[0014] The target weight is dot multiplied with the label feature corresponding to the target focus point to obtain a first output result.

[0015] The target weight is dot multiplied with the target channel output feature to obtain a second output result.

[0016] The second loss value is generated according to the Euclidean distance between the first output result and the second output result.

[0017] Optionally, the method comprises the following steps for determining the label feature corresponding to the target focus point:

[0018] The training image is processed according to the position information of the set detection points to generate a hot map corresponding to each detection point.

[0019] Each hot map is converted into a label feature map.

[0020] The target label feature map corresponding to the target focus point is selected from all the label feature maps, and the features contained in the target label feature map are taken as the label feature corresponding to the target focus point.

[0021] Optionally, the target focus point is selected based on the detection effect of the fixed-point detection model that has completed the first training phase on each output feature point.

[0022] It is judged whether the output feature point contains a pre-set focus point.

[0023] In a case where the output feature points contain all the preset attention points, selecting a preset proportion of the feature points from the output feature points as target attention points;

[0024] In a case where the output feature points do not contain all the preset attention points, taking the attention points not contained as target attention points.

[0025] Optionally, in a case where the training result of the latest point detection model meets a preset requirement, detecting the to-be-detected image by using the point detection model meeting the preset requirement to obtain feature points of the to-be-detected image comprises:

[0026] determining whether the number of the currently screened target attention points is less than a preset number;

[0027] In a case where the number of the currently screened target attention points is less than the preset number, ending the iteration operation and detecting the to-be-detected image by using the currently latest point detection model to obtain feature points of the to-be-detected image.

[0028] Optionally, for the determination of the number of the output feature points, the method comprises:

[0029] screening, according to a data distribution feature of the training image, a target detection point meeting a correlation requirement with the data distribution feature from preset detection points;

[0030] taking the number of the target detection point as the number of the output feature points of the point detection model.

[0031] Optionally, after determining the second loss value based on the target weight matched by the target attention point, the target channel output feature and the label feature corresponding to the target attention point, the method further comprises:

[0032] determining a loss value corresponding to each of the detection points based on the weight, the channel output feature and the label feature matched by each of the detection points;

[0033] selecting a preset number of target loss values with maximum values from all the loss values;

[0034] taking a sum value of all the target loss values as a third loss value;

[0035] Correspondingly, the updating of the weight of the point detection model by using the first loss value and the second loss value comprises:

[0036] updating the weight of the point detection model by using the first loss value, the second loss value and the third loss value.

[0037] The embodiment of the application further provides a device for extracting feature points of an image, comprising an analysis unit, a screening unit, a determination unit, an updating unit, a serving unit and a detection unit.

[0038] The analysis unit is configured to iteratively train the point detection model based on the training images to obtain output feature points and a first loss value.

[0039] The screening unit is configured to screen target focus points from the output feature points based on focus point selection rules corresponding to a training phase; wherein the target focus points in the first training phase are preset focus points; the target focus points in the second training phase are target focus points screened based on the detection effect of the point detection model that has completed the first training phase on the output feature points; and the first training phase and the second training phase have their respective iteration numbers.

[0040] The determination unit is configured to determine a second loss value based on the target weight matched with the target focus point, the target channel output feature and the label feature corresponding to the target focus point.

[0041] The updating unit is configured to update the weight of the point detection model by using the first loss value and the second loss value.

[0042] The serving unit is configured to use the updated point detection model as the latest point detection model, and trigger the analysis unit to execute the step of iteratively training the point detection model based on the training images to obtain output feature points and a first loss value.

[0043] The detection unit is configured to enter the second training phase when the first training phase reaches its corresponding iteration number, and to detect a to-be-detected image by using the point detection model that meets the set requirement until the training result of the latest point detection model meets the set requirement, so as to obtain the feature points of the to-be-detected image.

[0044] Optionally, the determination unit comprises an extraction subunit, a first operation subunit, a second operation subunit and a generation subunit.

[0045] The extraction subunit is configured to extract the target weight matched with the target focus point and the target channel output feature from the point detection model.

[0046] The first operation subunit is configured to perform a point multiplication operation on the target weight and the label feature corresponding to the target focus point to obtain a first output result.

[0047] The second operation subunit is configured to perform a point multiplication operation on the target weight and the target channel output feature to obtain a second output result.

[0048] The generating sub-unit is configured to generate a second loss value according to the Euclidean distance between the first output result and the second output result.

[0049] Optionally, for the determination process of the label feature corresponding to the target focus point, the device comprises a generating unit, a converting unit and a selecting unit.

[0050] The generating unit is configured to process the training image according to the position information of the set detection points to generate a hotspot map corresponding to each detection point.

[0051] The converting unit is configured to convert each hotspot map into a label feature map.

[0052] The selecting unit is configured to select a target label feature map corresponding to the target focus point from all the label feature maps, and take the feature contained in the target label feature map as the label feature corresponding to the target focus point.

[0053] Optionally, the screening unit comprises a judging sub-unit, a selecting sub-unit and a taking sub-unit.

[0054] The judging sub-unit is configured to judge whether the output feature point contains a pre-set focus point.

[0055] The selecting sub-unit is configured to select a feature point with a set proportion from the output feature point as a target focus point in the case that the output feature point contains all the pre-set focus points.

[0056] The taking sub-unit is configured to take an uncontained focus point as a target focus point in the case that the output feature point does not contain all the pre-set focus points.

[0057] Optionally, the detecting unit comprises a number judging unit and an analyzing sub-unit.

[0058] The number judging unit is configured to judge whether the number of the currently screened target focus points is less than a set number; and in the case that the number of the currently screened target focus points is less than the set number, the iteration operation is ended, and the analyzing sub-unit is triggered.

[0059] The analyzing sub-unit is configured to detect a to-be-detected image by using a current latest point detection model to obtain feature points of the to-be-detected image.

[0060] Optionally, for the determination process of the number of the output feature points, the device comprises an output feature point screening unit.

[0061] The output feature point screening unit is configured to screen target detection points that meet a correlation requirement with the data distribution feature from the set detection points according to the data distribution feature of the training image; and take the number of the target detection points as the output feature point number of the fixed-point detection model.

[0062] Optionally, the method further includes a loss value determining unit, a loss value selecting unit and a loss value summing unit.

[0063] The loss value determining unit is configured to determine a loss value corresponding to each detection point based on the matched weight, the channel output feature and the label feature of each detection point.

[0064] The loss value selecting unit is configured to select a preset number of target loss values with the largest values from all the loss values.

[0065] The loss value summing unit is configured to take the sum of all the target loss values as a third loss value.

[0066] Correspondingly, the updating unit is configured to update the weight of the fixed-point detection model by using the first loss value, the second loss value and the third loss value.

[0067] Embodiments of the present application further provide an electronic device, which includes:

[0068] A memory configured to store a computer program;

[0069] A processor configured to execute the computer program to implement the steps of the method for extracting image feature points.

[0070] Embodiments of the present application further provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the method for extracting image feature points.

[0071] It can be seen from the technical solution that the fixed-point detection model is iterated based on the training image to obtain output feature points and a first loss value. The first loss value reflects the detection effect of the model, and the better the detection effect, the smaller the loss value. In order to improve the accuracy of some points with poor detection effect, the target attention point can be selected from the output feature points based on the attention point selection rule corresponding to the training stage. The target attention point in the first training stage is a pre-set attention point, and the target attention point in the second training stage is a target attention point selected based on the detection effect of the fixed-point detection model in the first training stage on each output feature point. The second loss value is determined based on the target weight matched with the target attention point, the target channel output feature, and the label feature corresponding to the target attention point. The weight of the fixed-point detection model is updated using the first loss value and the second loss value, the updated fixed-point detection model is used as the latest fixed-point detection model, and the step of iterating the fixed-point detection model based on the training image to obtain the output feature points and the first loss value is returned. The first training stage and the second training stage have their respective iteration numbers. When the first training stage reaches its corresponding iteration number, it enters the second training stage. Through continuous iterative training, when the training result of the latest fixed-point detection model meets the set requirement, it means that the latest fixed-point detection model has reached a relatively optimal detection effect, and at this time, the update of the weight of the fixed-point detection model can be stopped. When it is necessary to detect the feature points of the to-be-detected image, the to-be-detected image can be detected using the fixed-point detection model meeting the set requirement to obtain the feature points of the to-be-detected image. In the technical solution, the corresponding attention point selection rule is set for different training stages, the detection points with poor effect can be focused on, and the second loss value corresponding to the target attention point is combined with the first loss value to update the weight of the model, thereby improving the detection effect of the model on the target attention point, so that the updated model has a good fixed-point detection effect. BRIEF DESCRIPTION OF DRAWINGS

[0072] In order to more clearly illustrate the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0073] Figure 1 A flowchart of a method for extracting image feature points provided by an embodiment of the present application;

[0074] Figure 2 A schematic diagram of calculating a first loss value and a second loss value provided by an embodiment of the present application;

[0075] Figure 3A flowchart of a method for updating a fixed-point detection model weight provided by an embodiment of the present application is shown in FIG. 1.

[0076] Figure 4 A detection effect comparison diagram of a fixed-point detection model provided by an embodiment of the present application is shown in FIG. 2.

[0077] Figure 5 A structural diagram of a device for extracting image feature points provided by an embodiment of the present application is shown in FIG. 3.

[0078] Figure 6 A structural diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 4. DETAILED DESCRIPTION

[0079] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, any other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0080] The terms “include” and “have”, and any variations of the terms “include” and “have” in the specification and claims of the present application and the above-mentioned drawings, are intended to cover the non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can include steps or units not listed.

[0081] In order to enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0082] Next, a method for extracting image feature points provided by an embodiment of the present application will be described in detail. Figure 1 A flowchart of a method for extracting image feature points provided by an embodiment of the present application is shown in FIG. 1, which includes the following steps.

[0083] S101: Iteratively training the fixed-point detection model based on the training image to obtain output feature points and a first loss value.

[0084] The training image is an image used for training the fixed-point detection model.

[0085] The fixed-point detection model can be used to detect feature points on an image. By detecting the feature points on the image, different processing tasks such as object type recognition, face recognition, etc. can be performed based on the image features contained in the feature points. The feature points needed to be focused on for different processing tasks will be different.

[0086] In the embodiment of the present application, the user can set the detection points needed to be focused based on the actual detection requirements. For example, when performing fixed-point detection on pedestrians, the user can focus on the head, right shoulder, right elbow, right hand, left shoulder, left elbow, left hand, right crotch, right knee, right foot, left crotch, left knee, left foot, and center point of the pedestrian. In this case, the detection points can be set as the 14 positions, and the number of detection points is 14.

[0087] When constructing the fixed-point detection model, the number of output channels of the model can be determined based on the number of detection points. For example, if there are 14 detection points, the number of output channels of the fixed-point detection model is set to 14, and one channel corresponds to one output feature point.

[0088] Based on the training image, the fixed-point detection model is iterated to obtain the output feature points and the first loss value.

[0089] The output feature points are the feature points on the training image detected when the fixed-point detection model is iterated and trained using the training image.

[0090] Each time the fixed-point detection model performs image feature point extraction, the corresponding loss value can be calculated based on the pre-labeled feature points in the training image and the feature points detected by the model. In order to distinguish from the loss value mentioned later, the loss value calculated at present can be referred to as the first loss value.

[0091] S102: Based on the attention point selection rule corresponding to the training stage, the target attention points are selected from the output feature points.

[0092] Taking the case where the number of output channels is 14 as an example, in the traditional way, the loss value is the first loss value obtained by comprehensively evaluating the output features of the 14 output channels. The first loss value is used for back propagation to adjust the weights of the fixed-point detection model. However, this adjustment method is a whole adjustment of the weights of the 14 output channels.

[0093] In actual application, some output channels can accurately detect feature points, while some output channels detect feature points with poor effect or cannot detect feature points. In order to improve the detection effect of the fixed-point detection model, the present application proposes to focus on some points with poor detection effect, and to focus on these points with poor effect when adjusting the weights, so that the output channels with poor detection effect can also have a great improvement in detection effect.

[0094] Taking 14 detection points as an example, the target attention points can be the 14 points that are difficult to detect or prone to false detection, so the detection points that need to be focused on are, for example, the number of target attention points in the initial state can be 6 points.

[0095] In practical applications, the specific detection points used as target focus points can be set based on actual needs.

[0096] In an embodiment of the present application, in order to improve the training effect of the fixed-point detection model, the training of the fixed-point detection model can be divided into different training stages.

[0097] The target focus point in the first training phase can be a pre-set focus point. This means that the desired focus point can be manually set as the target focus point. During the first training phase, the weights of the fixed-point detection model are initially updated based on the first loss value corresponding to the output feature point and the second loss value corresponding to the target focus point. The first and second training phases each have their own corresponding number of iterations. When the first training phase reaches its corresponding number of iterations, the second training phase begins.

[0098] The target points of interest in the second training phase can be selected based on the detection results of the fixed-point detection model at each output feature point after completing the first training phase. The second training phase has a set number of iterations, and a round of iteration is completed when the set number of iterations is reached. The second training phase can include multiple rounds of iterations. After each round of iteration, the number of target points of interest can be further reduced, allowing for a new round of iterations to be performed until the latest fixed-point detection model training results meet the set requirements.

[0099] Taking the example of 6 feature points as the target focus points in the first training stage, in the second training stage, 4 feature points can be selected as target focus points based on the detection effect when performing the first round of iteration. After completing one round of iteration, 2 feature points can be selected as the target focus points for the next round of iteration.

[0100] In an embodiment of the present application, in the second training phase, the fixed point detection model that has completed the first training phase can be iterated based on the training image to obtain output feature points of the training image. After obtaining the output feature points, it can be determined whether the output feature points contain a pre-set focus point. The pre-set focus point is the point that the user wants to focus on.

[0101] If the output feature points include all pre-set focus points, you can select a set percentage of feature points from the output feature points as target focus points. If the output feature points do not include all pre-set focus points, it means that the fixed point detection model failed to detect the points you want to focus on. In this case, you can select the missing focus points as target focus points.

[0102] Considering that in actual applications, the number of focus points that the fixed-point detection model fails to detect may be small, resulting in a small number of target focus points, in addition to taking the non-included focus points as target focus points, a part of the output feature points can also be selected from the output feature points as target focus points.

[0103] When selecting target focus points, the specific value of the percentage can be set based on actual needs. For example, the percentage can be set to 2 / 3. Considering that the number of target focus points calculated according to the percentage may have decimals, in the specific implementation, the integer part can be directly taken as the number of target focus points.

[0104] For example, if there are 6 target focus points in the first training phase, 6*2 / 3=4 target focus points can be selected in the first iteration of the second training phase. In the next iteration, Floor(4*2 / 3)=2 points can be selected as target focus points, where Floor means rounding down.

[0105] Of course, in the second training phase, in addition to setting the ratio, you can also subtract 2 from the number of target focus points in the previous iteration as the number of target focus points in the new iteration.

[0106] S103: Determine a second loss value based on the target weight of the target focus point matching, the target channel output feature, and the label feature corresponding to the target focus point.

[0107] In an embodiment of the present application, the label feature corresponding to the target focus point may be a label feature. In a specific implementation, the training image may be processed according to the position information of the set detection points to generate a heat map corresponding to each detection point; each heat map is converted into a label feature map; and a target label feature map corresponding to the target focus point is selected from all label feature maps, and the features contained in the target label feature map are used as the label feature corresponding to the target focus point.

[0108] For example, if there are 14 detection points, after acquiring the training image, we can generate a heat map corresponding to each detection point based on the location information of these 14 detection points. By performing network processing on each heat map, we can obtain the corresponding label feature map.

[0109] In actual application, the input training image is resized to 184*184, or other sizes, and the size of the size depends on the input size of the fixed point detection model. Taking 184*184 as an example, according to the 184*184 size of the Resize, 14 heat maps can be generated corresponding to 14 detection points, and different heat maps correspond to different positions of the detection points on the 184*184 image. The front and back positions of the 14 heat maps and the positions of the 14 detection points are one-to-one corresponding.

[0110] Corresponding to the input image of 184*184, the final model output is 23*23 after model processing. In order to correspond to 14 detection points one by one, the output of the model needs to be 23*23*14, which means that the output channel of the model corresponds to the number of detection points.

[0111] Corresponding to the number of detection points and the output of the model, the size of laebl is 14*23*23, and corresponding to the heat map of 184*184, the pixel value size of the detection point on the 23*23 label feature map is 1, and the distribution of the points around the detection point conforms to the Gauss distribution.

[0112] At the same time, in order to calculate the loss function, the weight corresponding to each channel needs to be designed. The design of the weight is a 14*23*23 kernel with all 1s.

[0113] In the embodiment of the present application, after the target attention point is determined, the target weight and the target channel output feature matched with the target attention point can be extracted from the fixed point detection model; the target weight and the label feature corresponding to the target attention point are multiplied to obtain a first output result; the target weight and the target channel output feature are multiplied to obtain a second output result; and a second loss value is generated according to the Euclidean distance between the first output result and the second output result.

[0114] Figure 2 A schematic diagram for calculating the first loss value and the second loss value provided in the embodiment of the present application, Figure 2 Taking 14 detection points as an example, the fixed point detection model includes 14 output channels, and each output channel corresponds to a detection point. Figure 2 It is divided into two parts, the upper half is a schematic diagram for calculating the first loss value (Loss), and the lower half is a schematic diagram for calculating the second loss value (Loss_2). Figure 2 In the weight, weight represents weight, each output channel has its corresponding weight, so the weight corresponding to the fixed point detection model includes 14 parts, and in actual application, the 14 parts of the weight can be represented in the form of a weight matrix. Label represents label feature, and Output represents channel output feature.

[0115] In the calculation of the first loss value, the weight matrix can be point multiplied with the label features contained in the label feature map to obtain Result_1. The weight matrix is point multiplied with the channel output features output by the fixed-point detection model to obtain Result_2. The euclidean loss of Result_1 and Result_2 is calculated as the first loss value.

[0116] It is known that each weight, Label and Output has a corresponding detection point. When the target attention point is confirmed, the Output, Label and weight corresponding to the target attention point can be determined. Therefore, the slice layer is used for the Output, Label and weight to re-disassemble and re-combine (concat layer) the relevant channels corresponding to the target attention point, and then Loss_2 is calculated. Taking the last four detection points as the target attention point as an example, the operation for the target attention point is as follows Figure 2 The lower half.

[0117] In the calculation of the second loss value, the selected last four weight matrices can be point multiplied with the label features contained in the last four label feature maps to obtain Result_3. The selected last four weight matrices are point multiplied with the last four channel output features output by the fixed-point detection model to obtain Result_4. The euclidean loss of Result_3 and Result_4 is calculated as the second loss value.

[0118] S104: The weights of the fixed-point detection model are updated using the first loss value and the second loss value, the updated fixed-point detection model is taken as the latest fixed-point detection model, and the step of iteratively training the fixed-point detection model based on the training image to obtain the output feature point and the first loss value is returned.

[0119] In the embodiments of the present application, the weights of the fixed-point detection model can be updated using the first loss value and the second loss value. The way of updating the model weights using the loss value is relatively common, and will not be introduced here.

[0120] S105: When the first training phase reaches its corresponding number of iterations, the second training phase is entered, and the training result of the latest fixed-point detection model meets the set requirement. The fixed-point detection model that meets the set requirement is used to detect the to-be-detected image to obtain the feature points of the to-be-detected image.

[0121] In the embodiments of the present application, the fixed-point detection model is divided into a training stage and an application stage. The training stage is a stage of updating the weights of the fixed-point detection model, and the application stage is a stage of performing fixed-point detection on images using the trained fixed-point detection model. In order to distinguish the images used in the two stages, the images used for updating the weights of the fixed-point detection model can be referred to as training images, and the images subjected to fixed-point detection analysis using the trained fixed-point detection model can be referred to as to-be-detected images.

[0122] In the first training stage, the feature points selected by humans are used as target focus points, and the weights of the fixed-point detection model are updated through continuous iterative training. When the number of iterations of the first training stage reaches the corresponding number of iterations, the second training stage can be entered.

[0123] In the second training stage, the number of target focus points that need to be focused on can be further reduced after each iteration, so as to perform a new round of iteration to continuously optimize the detection effect of the fixed-point detection model.

[0124] When the training result of the latest fixed-point detection model meets the set requirement, it means that the fixed-point detection model has achieved good detection effect, and at this time the latest fixed-point detection model can be used for feature point extraction of subsequent images.

[0125] The set requirement can be a requirement for the detection effect of the fixed-point detection model, or a requirement for the number of target focus points.

[0126] Taking the detection effect as an example, the detection effect can be evaluated by the number of feature points that can be actually detected by the fixed-point detection model. For example, an image needs to focus on 14 feature points, and at least 9 points need to be detected to meet the requirement. Using the trained fixed-point detection model to detect the image can detect 10 points, which means that the trained fixed-point detection model meets the set requirement.

[0127] Taking the number of target focus points as an example, considering that in actual application, in the second training stage, the number of target focus points is reduced after each iteration, there may be a case that the number of output feature points detected by the fixed-point detection model does not meet the detection effect of the output feature points corresponding to the training image, and the number of output feature points is already too small to participate in the calculation of the second loss value. Therefore, in the embodiments of the present application, after the target focus points are selected according to the detection effect, it is determined whether the number of the currently selected target focus points is less than a set number.

[0128] The value of the set number can be set based on actual needs, for example, it can be set to 1.

[0129] If the number of output feature points is less than the set number, then the updating of the weight of the fixed-point detection model is ended, and a step of detecting the to-be-detected image by using the latest fixed-point detection model to obtain feature points of the to-be-detected image is performed.

[0130] It should be noted that, in the case that the training result of the latest fixed-point detection model meets the set requirement, the fixed-point detection model can be directly called for detection of the to-be-detected image in the subsequent process, and there is no need to perform the operations of S101-S104 each time the to-be-detected image is detected.

[0131] As can be seen from the above technical solution, the fixed-point detection model is iterated based on the training image to obtain output feature points and a first loss value. The first loss value reflects the detection effect of the model, and the better the detection effect, the smaller the loss value. In order to improve the accuracy of some detection points with poor detection effect, the target attention point can be selected from the output feature points based on the attention point selection rule corresponding to the training stage. The target attention point in the first training stage is a pre-set attention point, and the target attention point in the second training stage is a target attention point selected based on the detection effect of the fixed-point detection model in the first training stage on each output feature point. The second loss value is determined based on the target weight matched with the target attention point, the target channel output feature, and the label feature corresponding to the target attention point. The weight of the fixed-point detection model is updated by using the first loss value and the second loss value. The updated fixed-point detection model is used as the latest fixed-point detection model, and the step of iterating the fixed-point detection model based on the training image to obtain output feature points and a first loss value is returned. The first training stage and the second training stage have their respective iteration numbers. When the first training stage reaches its corresponding iteration number, the second training stage is entered. Through continuous iterative training, until the training result of the latest fixed-point detection model meets the set requirement, it is indicated that the latest fixed-point detection model has reached a relatively optimal detection effect. At this time, the updating of the weight of the fixed-point detection model can be stopped. When the detection of feature points of the to-be-detected image is needed, the to-be-detected image can be detected by using the fixed-point detection model that meets the set requirement to obtain the feature points of the to-be-detected image. In the technical solution, the corresponding attention point selection rule is set for different training stages, the detection points with poor effect can be focused on, and the second loss value corresponding to the target attention point is combined with the first loss value to update the weight of the model, thereby improving the detection effect of the model on the target attention point, so that the updated model has a good fixed-point detection effect.

[0132] For the determination process of the number of output feature points, the target detection point that meets the correlation requirement with the data distribution feature of the training image can be selected from the set detection points according to the data distribution feature of the training image. The number of target detection points is used as the number of output feature points of the fixed-point detection model.

[0133] The data distribution feature reflects characteristics of the training images, for example, a time of acquisition of the training images, a climate in which a scene contained in the training images is located, a location, and the like. In embodiments of the present application, a specific form of the data distribution feature is not limited.

[0134] In order to distinguish different detection points, the detection point that meets the correlation requirement of the data distribution feature can be referred to as a target detection point. The feature points detected by the point detection model under different data distribution features will be different,

[0135] The detection point that meets the correlation requirement of the data distribution feature can be a detection point that needs to be focused on under the data distribution feature. The detection point that needs to be focused on can be a detection point with poor detection effect.

[0136] For example, if the training images are mainly acquired in summer, it can be considered that a skirt worn by a woman is a reason for causing difficulty in point detection, and therefore, the lower limbs of the pedestrian in the training images can be taken as the output feature points; if the training images are mainly acquired in winter, because of cold, the head and upper limbs of the pedestrian are easily covered, and therefore, the upper limbs and head of the target can be focused on, and the detection points belonging to the upper limbs and head of the pedestrian in the training images can be taken as the output feature points; if most of the training images show that the pedestrians wear masks, the head can be focused on, and the detection points belonging to the head of the pedestrian in the training images can be taken as the output feature points. Any form of data distribution can be taken as a condition for selecting the output feature points of the model.

[0137] In the above introduction, the target focus point is selected from the output feature points to realize the focus on the detection point with poor detection effect. In embodiments of the present application, in order to further improve the detection effect of the point detection model, the detection effect of each detection point can be evaluated on the basis of the first loss value and the second loss value, a few detection points with the worst detection effect can be selected, the loss value of the few detection points is taken as a third loss value, and the weight of the point detection model is updated. Figure 3 A flowchart of a method for updating the weight of the point detection model provided in embodiments of the present application, the method comprising:

[0138] S301: iteratively training the point detection model based on the training images to obtain output feature points and a first loss value.

[0139] S302: selecting a target focus point from the output feature points based on a focus point selection rule corresponding to the training phase.

[0140] S303: determining a second loss value based on a target weight matched with the target focus point, a target channel output feature, and a label feature corresponding to the target focus point.

[0141] The implementation manners of S301 to S303 can refer to the introduction of S101 to S103, and will not be described here again.

[0142] S304: Based on the weight matched by each detection point, the channel output feature, and the label feature, the loss value corresponding to each detection point is determined.

[0143] In combination with Figure 2 The schematic diagram shows that each weight, Label, and Output has a corresponding detection point. The Output, Label, and weight corresponding to each detection point can be determined, so we use the slice layer for Output, Label, and weight to separate and recombine (concat layer) the relevant channels corresponding to each detection point, and then calculate the loss value.

[0144] S305: Select the target loss value with the maximum value from a preset number of loss values, and take the sum of all target loss values as the third loss value.

[0145] The preset number of values can be set based on actual needs. Taking 14 detection points as an example, the preset number can be set to 3. After calculating the loss value corresponding to each detection point, the first 3 loss values with the maximum value can be selected from the 14 loss values and added as the third loss value.

[0146] S306: Update the weight of the fixed-point detection model using the first loss value, the second loss value, and the third loss value, and take the updated fixed-point detection model as the latest fixed-point detection model.

[0147] After updating the weight of the fixed-point detection model using the first loss value, the second loss value, and the third loss value, the updated fixed-point detection model can be taken as the latest fixed-point detection model, the target focus point can be taken as the output feature point, and the step of S301 can be returned. Until the output feature point meets the output feature point detection effect corresponding to the training image, the process of updating the weight of the fixed-point detection model is ended. At this time, the latest fixed-point detection model can be used to detect the to-be-detected image to obtain the feature point of the to-be-detected image.

[0148] Figure 4 A detection effect comparison diagram of a fixed-point detection model provided by an embodiment of the present application, Figure 4 The image on the left side of the diagram is the detection point detected by the fixed-point detection model after updating the weight of the fixed-point detection model based on the traditional method. Figure 4 The image on the right side is the detection point detected by the fixed-point detection model after updating the weight of the fixed-point detection model based on the method provided by the present application. Figure 3 FromFigure 4 As can be seen, the fixed-point detection model updated based on the weight updating manner of the present application can detect more detection points.

[0149] In the embodiments of the present application, by comprehensively considering all output feature points, target attention points of focus, and loss values corresponding to several detection points with the worst detection effect in all output feature points, the weight of the fixed-point detection model is updated using the three types of loss values, so that the fixed-point detection model can achieve better detection effect more quickly.

[0150] Figure 5 A structural schematic diagram of a device for extracting image feature points provided in an embodiment of the present application includes an analysis unit 51, a screening unit 52, a determination unit 53, an updating unit 54, an as unit 55, and a detection unit 56.

[0151] The analysis unit 51 is configured to iteratively train the fixed-point detection model based on training images to obtain output feature points and a first loss value.

[0152] The screening unit 52 is configured to screen target attention points from the output feature points based on an attention point selection rule corresponding to a training stage; wherein the target attention points in the first training stage are pre-set attention points; the target attention points in the second training stage are target attention points screened based on the detection effect of the fixed-point detection model completing the first training stage on each output feature point; the first training stage and the second training stage have their respective iteration numbers.

[0153] The determination unit 53 is configured to determine a second loss value based on the target weight matched by the target attention points, the target channel output feature, and the label feature corresponding to the target attention points.

[0154] The updating unit 54 is configured to update the weight of the fixed-point detection model using the first loss value and the second loss value.

[0155] The as unit 55 is configured to use the updated fixed-point detection model as the latest fixed-point detection model, and trigger the analysis unit to perform the step of iteratively training the fixed-point detection model based on training images to obtain output feature points and a first loss value.

[0156] The detection unit 56 is configured to enter the second training stage when the first training stage reaches its corresponding iteration number, and continue until the training result of the latest fixed-point detection model meets the set requirement, and then use the fixed-point detection model meeting the set requirement to detect the to-be-detected image to obtain the feature points of the to-be-detected image.

[0157] Optionally, the determination unit includes an extraction subunit, a first operation subunit, a second operation subunit, and a generation subunit.

[0158] The extraction subunit is configured to extract a target weight and a target channel output feature matching the target attention point from the fixed-point detection model;

[0159] The first operation subunit is configured to perform a point multiplication operation on the target weight and a label feature corresponding to the target attention point to obtain a first output result;

[0160] The second operation subunit is configured to perform a point multiplication operation on the target weight and the target channel output feature to obtain a second output result;

[0161] The generation subunit is configured to generate a second loss value according to an Euclidean distance between the first output result and the second output result.

[0162] Optionally, for the determination process of the label feature corresponding to the target attention point, the device comprises a generation unit, a conversion unit and a selection unit;

[0163] The generation unit is configured to process the training image according to the position information of the set detection point to generate a hot map corresponding to each detection point;

[0164] The conversion unit is configured to convert each hot map into a label feature map;

[0165] The selection unit is configured to select a target label feature map corresponding to the target attention point from all label feature maps, and take the features contained in the target label feature map as the label feature corresponding to the target attention point.

[0166] Optionally, the screening unit comprises a judgment subunit, a selection subunit and an as subunit;

[0167] The judgment subunit is configured to judge whether the output feature point contains a pre-set attention point;

[0168] The selection subunit is configured to select a set proportion of feature points from the output feature points as the target attention point in the case that the output feature points contain all the pre-set attention points;

[0169] The as subunit is configured to take the uncontained attention point as the target attention point in the case that the output feature points do not contain all the pre-set attention points.

[0170] Optionally, the detection unit comprises a number judgment unit and an analysis subunit;

[0171] The number judgment unit is configured to judge whether the number of the currently screened target attention points is less than a set number; in the case that the number of the currently screened target attention points is less than the set number, the iteration operation is ended, and the analysis subunit is triggered;

[0172] The analysis subunit is configured to detect the to-be-detected image by using the latest fixed-point detection model to obtain feature points of the to-be-detected image.

[0173] Optionally, for the determination of the number of output feature points, the device comprises an output feature point screening unit.

[0174] The output feature point screening unit is configured to screen target detection points that satisfy a correlation requirement with a data distribution feature from the set detection points according to the data distribution feature of the training image; and take the number of the target detection points as the number of output feature points of the fixed-point detection model.

[0175] Optionally, the device further comprises a loss value determination unit, a loss value selection unit and a loss value summation unit.

[0176] The loss value determination unit is configured to determine a loss value corresponding to each detection point based on the matched weight, the channel output feature and the label feature of each detection point.

[0177] The loss value selection unit is configured to select a target loss value with a maximum value from a preset number of loss values.

[0178] The loss value summation unit is configured to take a sum value of all target loss values as a third loss value.

[0179] Correspondingly, the updating unit is configured to update the weight of the fixed-point detection model by using the first loss value, the second loss value and the third loss value.

[0180] Figure 5 The description of the features in the corresponding embodiments can be referred to Figure 1 and Figure 3 the related description of the corresponding embodiments, which will not be repeated here.

[0181] It can be seen from the above technical solution that the fixed point detection model is iterated based on the training image to obtain output feature points and a first loss value. The first loss value reflects the detection effect of the model, and the better the detection effect, the smaller the loss value. In order to improve the accuracy of some points with poor detection effect, the target attention point can be selected from the output feature points based on the attention point selection rule corresponding to the training stage. The target attention point in the first training stage is a pre-set attention point, and the target attention point in the second training stage is a target attention point selected based on the detection effect of the fixed point detection model in the first training stage on each output feature point. The second loss value is determined based on the target weight matched with the target attention point, the target channel output feature, and the label feature corresponding to the target attention point. The weight of the fixed point detection model is updated using the first loss value and the second loss value, the updated fixed point detection model is used as the latest fixed point detection model, and the step of iterating the fixed point detection model based on the training image to obtain the output feature points and the first loss value is returned. The first training stage and the second training stage have their respective iteration numbers. When the first training stage reaches its corresponding iteration number, it enters the second training stage. Through continuous iterative training, when the training result of the latest fixed point detection model meets the set requirement, it means that the latest fixed point detection model has reached a relatively optimal detection effect, and at this time, the update of the weight of the fixed point detection model can be stopped. When it is necessary to detect the feature points of the to-be-detected image, the to-be-detected image can be detected by using the fixed point detection model meeting the set requirement to obtain the feature points of the to-be-detected image. In the technical solution, the corresponding attention point selection rule is set for different training stages, the detection points with poor effect can be focused on, and the second loss value corresponding to the target attention point is combined with the first loss value to update the weight of the model, thereby improving the detection effect of the model on the target attention point, so that the updated model has a good fixed point detection effect.

[0182] Figure 6 A structural diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 1. Figure 6 The electronic device includes a memory 20 for storing a computer program.

[0183] A processor 21 is configured to execute the computer program to implement the steps of the method for extracting image feature points according to the above embodiments.

[0184] The electronic device provided by the embodiment can include, but is not limited to, a smart phone, a tablet computer, a notebook computer, or a desktop computer, etc.

[0185] The processor 21 can include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 21 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), a PLA (Programmable Logic Array). The processor 21 can also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also referred to as a CPU (Central Processing Unit). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 21 can be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing of content to be displayed on the display screen. In some embodiments, the processor 21 can further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

[0186] The memory 20 can include one or more computer-readable storage media that can be non-transitory. The memory 20 can further include a high-speed random access memory, and a nonvolatile memory such as one or more disk storage devices, flash storage devices. In the present embodiment, the memory 20 is at least used to store the following computer program 201, wherein the computer program is loaded and executed by the processor 21, and can implement the related steps of the method for extracting image feature points disclosed in any of the preceding embodiments. In addition, the resources stored by the memory 20 can further include an operating system 202 and data 203, and the storage mode can be temporary storage or permanent storage. The operating system 202 can include Windows, Unix, Linux, and the like. The data 203 can include but is not limited to output feature points corresponding to training images, and the like.

[0187] In some embodiments, the electronic device can further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.

[0188] Those skilled in the art can understand that, Figure 6 The structure shown in the figure does not constitute a limitation on the electronic device, and can include more or fewer components than those shown.

[0189] It can be understood that if the method for extracting image feature points in the above embodiments is implemented in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and performs all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), an electrically erasable programmable ROM, a register, a hard disk, a removable magnetic disk, a CD-ROM, a magnetic disk or an optical disk, and various media that can store program codes.

[0190] Based on this, the embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the method for extracting image feature points are implemented.

[0191] The above describes in detail the method, device, equipment and computer readable storage medium for extracting image feature points provided by the embodiments of the present application. The embodiments in the specification are described in a progressive manner, and each embodiment mainly describes the difference from other embodiments. The same or similar parts of each embodiment can be referred to. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0192] The skilled person can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of the two. In order to clearly show the interchangeability of hardware and software, the components and steps of each example have been described in the above description. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solutions. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0193] The above describes in detail the method, device, equipment and computer readable storage medium for extracting image feature points provided by the present application. The principles and implementation manners of the present application are described by applying specific examples, and the above description of the embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that, for those skilled in the art, some improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A method of extracting feature points of an image, characterized by, The method comprises the following steps: iterating a point detection model based on training images to obtain output feature points and a first loss value; selecting target attention points from the output feature points based on attention point selection rules corresponding to a training stage; wherein, the target attention points in the first training stage are pre-set attention points; the target attention points in the second training stage are target attention points selected based on the detection effect of the point detection model completing the first training stage on each output feature point; the first training stage and the second training stage have their respective iteration numbers; determining a second loss value based on target weights matched with the target attention points, target channel output features, and label features corresponding to the target attention points; updating the weights of the point detection model using the first loss value and the second loss value, taking the updated point detection model as the latest point detection model, and returning to the step of iterating the point detection model based on training images to obtain output feature points and a first loss value; entering the second training stage when the first training stage reaches its corresponding iteration number, and continuing until the training result of the latest point detection model meets the set requirements, then using the point detection model meeting the set requirements to detect the to-be-detected image to obtain the feature points of the to-be-detected image.

2. The method for extracting image feature points according to claim 1, characterized in that, The step of determining a second loss value based on target weights matched with the target attention points, target channel output features, and label features corresponding to the target attention points comprises the following steps: extracting target weights and target channel output features matched with the target attention points from the point detection model; performing a dot product operation on the target weights and the label features corresponding to the target attention points to obtain a first output result; performing a dot product operation on the target weights and the target channel output features to obtain a second output result; generating a second loss value according to the Euclidean distance between the first output result and the second output result.

3. The method for extracting image feature points according to claim 1, characterized in that, For the determination process of the label features corresponding to the target attention points, the method comprises the following steps: processing the training images according to the position information of the set detection points to generate a hot map corresponding to each detection point; converting each hot map into a label feature map; selecting a target label feature map corresponding to the target attention points from all the label feature maps, and taking the features contained in the target label feature map as the label features corresponding to the target attention points.

4. The method for extracting image feature points according to claim 1, characterized in that, The step of selecting target attention points based on the detection effect of the point detection model completing the first training stage on each output feature point comprises the following steps: determining whether the output feature points contain pre-set attention points; in the case that the output feature points contain all pre-set attention points, selecting feature points with a set proportion from the output feature points as target attention points; in the case that the output feature points do not contain all pre-set attention points, taking the uncontained attention points as target attention points.

5. The method for extracting image feature points according to claim 1, characterized in that, The training result of the latest fixed-point detection model meets the set requirement, and then the fixed-point detection model meeting the set requirement is used to detect the to-be-detected image to obtain the feature points of the to-be-detected image, which comprises: Determine whether the number of the current selected target attention points is less than a set number; If the number of the current selected target attention points is less than the set number, then end the iteration operation, and use the current latest fixed-point detection model to detect the to-be-detected image to obtain the feature points of the to-be-detected image.

6. The method for extracting image feature points according to claim 1, characterized in that, For the determination process of the output feature point number, the method comprises: According to the data distribution characteristics of the training image, the target detection points meeting the correlation requirement with the data distribution characteristics are selected from the set detection points; The number of the target detection points is used as the output feature point number of the fixed-point detection model.

7. The method for extracting image feature points according to any one of claims 1 to 6, characterized in that, After determining the second loss value based on the target weight matched by the target attention point, the target channel output feature and the label feature corresponding to the target attention point, it further comprises: Determine the loss value corresponding to each detection point based on the weight, channel output feature and label feature matched by each detection point; Select a preset number of target loss values with the maximum value from all the loss values; The sum of all the target loss values is used as the third loss value; Correspondingly, the use of the first loss value and the second loss value to update the weight of the fixed-point detection model comprises: The first loss value, the second loss value and the third loss value are used to update the weight of the fixed-point detection model.

8. An apparatus for extracting feature points of an image, the apparatus comprising: It comprises an analysis unit, a screening unit, a determination unit, an updating unit, an as unit and a detection unit; The analysis unit is used to iteratively train the fixed-point detection model based on the training image to obtain the output feature points and the first loss value; The screening unit is used to select the target attention point from the output feature points based on the attention point selection rule corresponding to the training stage; wherein the first training stage target attention point is a pre-set attention point; the second training stage target attention point is a target attention point selected based on the detection effect of the fixed-point detection model on each output feature point after completing the first training stage; the first training stage and the second training stage have their respective iteration times; The determination unit is used to determine the second loss value based on the target weight matched by the target attention point, the target channel output feature and the label feature corresponding to the target attention point; The updating unit is used to update the weight of the fixed-point detection model using the first loss value and the second loss value; The as unit is used to use the updated fixed-point detection model as the latest fixed-point detection model, and trigger the analysis unit to execute the step of iteratively training the fixed-point detection model based on the training image to obtain the output feature points and the first loss value. The detection unit is configured to enter a second training stage when the first training stage reaches a corresponding iteration number, and to use the fixed-point detection model meeting the set requirement to detect the to-be-detected image until the training result of the latest fixed-point detection model meets the set requirement, so as to obtain the feature points of the to-be-detected image.

9. An electronic device, comprising: The method comprises the following steps: a memory for storing a computer program; a processor for executing the computer program to implement the steps of the method for extracting feature points of an image according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium and is executed by the processor to implement the steps of the method for extracting feature points of an image according to any one of claims 1 to 7.

Images