A gait recognition model establishment method and a gait recognition method
By using a skinned multi-person linear model and graph network to extract gait features, the accuracy problem of gait recognition under changes in viewpoint and clothing is solved, achieving faster model convergence and higher recognition accuracy, which is suitable for video surveillance and security fields.
Patent Information
- Application Number
- CN202310374003.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-10
- Publication Date
- 2026-02-27
- Estimated Expiration
- 2043-04-10
AI Technical Summary
Existing gait recognition methods are not very accurate when the viewpoint and clothing change, and they are also expensive to train, slow to converge, and time-consuming.
A skinned multi-person linear model generation module is used to calculate the target video segment frame by frame to generate a skinned multi-person linear model. The shape and posture features are extracted by the skinned multi-person linear model feature extraction module and the graph pooling module based on human body parts. The posture parameter features are extracted by combining the graph network to construct a gait recognition model.
It improves the visual robustness and clothing robustness of gait recognition, with faster model convergence, higher recognition accuracy, and applicability to real-world scenarios.
Smart Images
Figure CN116434337B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer vision, in particular to a gait recognition model establishing method and a gait recognition method. BACKGROUND
[0002] Gait recognition is a kind of biometric recognition technology. Compared with other biometric recognition technologies such as face recognition, fingerprint recognition and iris recognition, gait recognition has three advantages: first, it does not require the cooperation of the target person, it can recognize in the most natural state of the target person, and the user experience is more friendly. Second, the recognition distance is far, and the gait recognition technology can recognize at a distance of dozens of meters or even hundreds of meters, while other feature recognition methods are mostly at the level of several meters. Third, the anti-counterfeiting performance is high, because everyone's walking posture is difficult to fake. Therefore, this technology can be widely applied in video monitoring and security fields.
[0003] The current mainstream gait recognition method first extracts the contour from the image of the person, and then uses a deep learning method to extract features from the contour to obtain the feature representation of the person. However, the contour-based method has the following problems: first, when the viewing angle changes, the contour recognition performance is easily affected. Second, when the target person changes clothes, or wears a coat, or carries a backpack, the contour will change greatly, resulting in poor performance of the gait recognition method. SUMMARY
[0004] The purpose of the present application is to solve the problem of improving the accuracy of gait recognition, and to provide a gait recognition model establishing method and a gait recognition method.
[0005] To achieve the above purpose, the present application adopts the following technical scheme:
[0006] A gait recognition method, comprising the following steps:
[0007] S1: obtaining a target video segment, generating a skinning multi-person linear model frame by frame through a skinning multi-person linear model generating module, and obtaining shape parameters and pose parameters;
[0008] S2: inputting the shape parameters and the pose parameters into a skinning multi-person linear model feature extraction module to obtain shape features of each frame and pose features of each frame;
[0009] S3: processing the shape features and the pose features through an aggregation module respectively to obtain time-aggregated shape features and time-aggregated pose features.
[0010] S4: inputting the time-aggregated pose features into a human body part-based graph pooling module to obtain human body part features, and adding the human body part features to the time-aggregated shape features;
[0011] S5: input the added features into the gait representation module to obtain the gait representation of the target person, and obtain the gait recognition result according to the similarity with the known video clips.
[0012] In some embodiments, step S2 specifically comprises the following steps:
[0013] S21: input the shape parameters into a multi-layer perception network to obtain the shape features of each frame, the shape parameters being represented by a vector;
[0014] S22: normalize the pose parameters and then input them into a graph network to obtain the pose features of each frame, the pose parameters being represented by a set of skeleton graphs.
[0015] In some embodiments, in step S21, the multi-layer perception network comprises three units, each unit comprising a linear calculation layer, a batch normalization layer and a random loss layer.
[0016] In some embodiments, in step S22, the graph network comprises three units, each unit having the following calculation formula:
[0017] X (l+1) = ReLU[X (l) + GConv(X (l) )]
[0018] wherein X (l) is the input of the lth unit, ReLU is the ReLU activation function, and GConv is the graph convolution operator.
[0019] In some embodiments, in step S4, inputting the time-aggregated pose features into a human body part-based graph pooling module to obtain the human body part features specifically comprises the following steps:
[0020] S41: construct a transverse feature list, wherein the transverse features can capture the human transverse features and multi-scale features;
[0021] S42: construct a limb feature list, wherein the limb features can focus on the limb features when the human walks;
[0022] S43: construct a torso list, the torso list encoding the human orientation information;
[0023] S44: respectively perform global pooling and addition on the transverse feature list, the limb feature list and the torso list to obtain the human body part features.
[0024] In some embodiments, in step S3, the aggregation module aggregates the shape features and the pose features by set pooling processing to obtain the time-aggregated shape features and pose features.
[0025] In some embodiments, in step S5, the gait representation module obtains the gait representation of the target person by processing the result of step S4 through a linear layer and batch normalization.
[0026] The application further provides a method for establishing a gait recognition model, comprising the following steps:
[0027] A1: constructing a gait recognition network, wherein the gait recognition network comprises
[0028] a skin multi-person linear model generation module, configured to calculate a target video segment frame by frame and generate a skin multi-person linear model, and obtain shape parameters and posture parameters;
[0029] a skin multi-person linear model feature extraction module, configured to receive the shape parameters and the posture parameters and obtain shape features of each frame and posture features of each frame
[0030] an aggregation module, configured to process the shape features and the posture features respectively, and obtain time-aggregated shape features and time-aggregated posture features;
[0031] a human body part-based graph pooling module, configured to obtain human body part features according to the time-aggregated posture features, and add the human body part features to the time-aggregated shape features;
[0032] a gait representation module, configured to obtain a gait representation of a target person according to the added features, and obtain a gait recognition result according to the similarity with known video segments;
[0033] A2: training the above gait recognition network with person video data to obtain a gait recognition model.
[0034] In some embodiments, in step A2, during the training process, a batch is formed by sampling from all video segments in the training set at each iteration, and a triplet loss and a cross-entropy loss are calculated between the gait representations obtained between the video segments in the batch, the triplet loss and the cross-entropy loss are calculated with a weighting sum in a ratio of 9:1 to obtain a total loss, and the total loss is back-propagated back to the network to update the network parameters.
[0035] The application further provides a computer readable medium having a computer program stored thereon, wherein the program is executed by a processor to implement the steps of the above method.
[0036] The application has the following beneficial effects:
[0037] The skin multi-person linear model generation module frame by frame calculates a target video segment, generates a skin multi-person linear model, and processes to obtain three-dimensional information representation, the skin multi-person linear model is not affected by the change of the visual angle, the application has visual robustness; the skin multi-person linear model is low-dimensional in the description of the shape, and focuses on the description of the human body shape rather than the clothing information, so that the application has clothing robustness. The skin multi-person linear model feature extraction module and the human body part-based graph pooling module can make the model converge faster, reduce the training cost, and have higher performance.
[0038] In some embodiments, the following benefits are also achieved: the embodiment models the posture parameters as a graph structure, extracts the posture parameter features by using a graph network, and combines the human body prior to construct a horizontal feature list, a limb feature list and a trunk list, so that the gait information is fully learned in the human body part-based graph pooling module, and the gait recognition accuracy in the embodiment can be improved.
[0039] Other benefits of the embodiment will be further described below. BRIEF DESCRIPTION OF DRAWINGS
[0040] Figure 1 is a flow chart of the gait recognition model establishment method in the embodiment of the application;
[0041] Figure 2 is a flow chart of the gait recognition method in the embodiment of the application;
[0042] Figure 3 is a schematic diagram of the specific steps of the gait recognition method in the embodiment of the application;
[0043] Figure 4 is a human body posture skeleton structure diagram modeled in the embodiment of the application;
[0044] Figure 5 is a skin multi-person linear model feature extraction module in the embodiment of the application;
[0045] Figure 6 is a human body part-based graph pooling module in the embodiment of the application;
[0046] Figure 7 is a convergence curve of the cross-entropy loss when the method in the experimental example is trained;
[0047] Figure 8 is a convergence curve of the ternary loss when the method in the experimental example is trained. DETAILED DESCRIPTION
[0048] The embodiments of the application are described in detail below. It should be emphasized that the following description is only exemplary and is not intended to limit the scope of the application and its applications.
[0049] Furthermore, the terms "first", "second", etc. are used only for descriptive purposes and do not connote or imply relative importance or a quantity of the indicated technical features. Thus, features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of embodiments of the present application, the meaning of "a plurality" is two or more, unless otherwise explicitly and specifically defined.
[0050] Embodiments
[0051] As Figure 1 shown, the present application provides a method for establishing a gait recognition model GaitPS (Gait Recognition with Pure SMPL representation), comprising the following steps:
[0052] S1: constructing a gait recognition network, the gait recognition network comprising a SMPL (Skin Measured Person Linear Model) generation module, a SMPL feature extraction module, an aggregation module, a human body part-based graph pooling module, and a gait representation module; wherein the SMPL generation module is configured to calculate a target video segment frame by frame and generate a SMPL, to obtain shape parameters and pose parameters; the SMPL feature extraction module is configured to receive the shape parameters and the pose parameters and obtain shape features of each frame and pose features of each frame; the aggregation module is configured to process the shape features and the pose features respectively, to obtain time-aggregated shape features and time-aggregated pose features; the human body part-based graph pooling module is configured to obtain human body part features according to the time-aggregated pose features, and add the human body part features to the time-aggregated shape features; and the gait representation module is configured to obtain a gait representation of a target person according to the added features, and obtain a gait recognition result according to a similarity with a known video segment;
[0053] S2: training the above gait recognition network with person video data to obtain a gait recognition model, and in each iteration process, inputting N groups of different persons, each person having M video segments.
[0054] Specifically, in each iteration, a batch is formed by sampling from all video segments of the training set, and a triplet loss and a cross entropy loss are calculated between the gait representations obtained between the video segments in the batch. The two losses are calculated with a weighting sum in a ratio of 9:1 to obtain a total loss, and the total loss is back-propagated to the network to update the network parameters.
[0055] As Figure 2As shown, the present invention also provides a gait recognition method, which involves inputting a target video segment into the aforementioned gait recognition model. The method includes the following steps:
[0056] S1: Obtain the target video segment, calculate and generate a skinned multi-person linear model frame by frame through the skinned multi-person linear model generation module, and obtain the shape parameters and pose parameters;
[0057] S2: Input the shape parameters and the pose parameters into the skinned multi-person linear model feature extraction module to obtain the shape features and pose features of each frame;
[0058] S3: The shape features and the pose features are processed by the aggregation module to obtain time-aggregated shape features and time-aggregated pose features;
[0059] S4: Input the pose features of the time aggregation into the graph pooling module based on human body parts to obtain human body part features, and add the human body part features to the shape features of the time aggregation;
[0060] S5: Input the summed features into the gait representation module to obtain the gait representation of the target person, and obtain the gait recognition result based on the similarity with known video clips.
[0061] refer to Figure 3 The specific steps of the gait recognition method in this embodiment are as follows:
[0062] In step S1, for each target video segment, the SMPL model (skinned multi-person linear model) is calculated and generated frame by frame to obtain shape parameters and pose parameters. The shape parameters are then represented by a vector, referencing... Figure 4 The attitude parameters are represented by a set of skeleton diagrams;
[0063] like Figure 5 As shown, in this embodiment, step S2 involves inputting the shape parameters and pose parameters into the skinned multi-person linear model feature extraction module for SMPL feature extraction, specifically including the following steps:
[0064] B1: Input the shape parameters into a multilayer perceptron network. Preferably, the multilayer perceptron network contains three units, each of which contains a linear computation layer, a batch normalization layer, and a dropout layer to obtain the shape features of each frame.
[0065] B2: The pose parameters are normalized and then input into the graph network to obtain the pose features for each frame. Preferably, the graph network contains three units, and the calculation formula for each unit is as follows:
[0066] X (l+1) =ReLU[X(l) + GConv(X (l) )] (1)
[0067] wherein, X (l) is the l-th unit input, ReLU is a ReLU activation function, and GConv is a graph convolution operator;
[0068] In step S3, the shape features and the pose features of each video are processed by an aggregation module respectively. In the embodiment, the aggregation module includes a set pooling unit which takes the maximum value of the input tensor in the time dimension to obtain the time-aggregated shape features and the time-aggregated pose features.
[0069] Referring to Figure 6 , in step S4, the time-aggregated pose features are input into a human body part-based graph pooling module to obtain human body part features, including the following steps:
[0070] C1: A transverse feature list is constructed. In the human body skeleton graph, the nodes are divided into four parts transversely, and the node features of each part are concatenated and input into the transverse feature list. Then, the features of the four adjacent parts are combined to obtain two features, which are input into the transverse feature list. Finally, the features of the four parts are combined as a whole and input into the transverse feature list; these features can capture the transverse features and multi-scale features of the human body;
[0071] C2: A limb feature list is constructed. The limbs of the human body skeleton graph are divided into left upper arm, left lower arm, right upper arm, right lower arm, left upper leg, left lower leg, right upper leg, and right lower leg, and the obtained eight groups of features are input into the limb feature list. Then, the eight groups of features are grouped into two groups, and the features of the left arm, right arm, left leg, right leg, upper arm, lower arm, upper leg, and lower leg are calculated and input into the list. The eight groups of features are grouped into four groups to form the features of the left arm, right arm, left leg, and right leg, which are input into the list; these features can focus on the limb features during walking;
[0072] C3: A torso list is constructed. The torso nodes in the human body skeleton graph are extracted, and their features are input into the torso list. The torso list encodes the orientation information of the human body. We assume that the orientation of the human body torso represents the camera perspective. According to the characteristics of the SMPL model, encoding the shape parameter features can alleviate the influence of perspective changes;
[0073] C4: The three lists in C1-C3 are respectively globally pooled and added to obtain the human body part features.
[0074] In step S5, the added features are input to a gait representation module, the gait representation module includes a linear layer and a batch normalization layer, and a gait representation of the target person is obtained; for a target video segment and a known video segment of a given person, features of the two video segments are calculated by the network respectively, similarity between the gait representation of the target video segment and the gait representation of the known video segment is calculated, and whether the two video segments belong to the same person is determined according to the similarity.
[0075] The embodiment of the present application solves the pain points in the prior art.
[0076] 1. Low robustness, especially sensitive to clothing and viewing angle in the real world.
[0077] 2. Similar methods lack detailed modeling of the SMPL model, resulting in high training cost, large model, slow convergence, and long algorithm time.
[0078] The method of the embodiment uses the SMPL model, i.e., the skinning multi-person linear model, for modeling, so that the gait features have high robustness to viewing angle changes and clothing changes: 2D pictures present different contents at different viewing angles, while the SMPL models are 3D models, and the models are not affected by viewing angle changes, so that the method of the embodiment has visual robustness; 2D images / 2D contours model clothing details clearly, and the edges of the person are depicted in detail, so that the contours will change greatly when the clothing deforms or is changed, which will greatly interfere with the performance, while the SMPL describes the shape in a low dimension and focuses on describing the human shape, so that the method of the embodiment has clothing robustness. The SMPL pose parameters are modeled as a graph structure in the embodiment, the pose parameter features are extracted by a graph network, and the gait information is fully learned in the graph pooling module in combination with human prior. Compared with existing similar methods, the model converges faster and has higher accuracy.
[0079] The embodiment of the present application also provides a computer readable medium having a computer program stored thereon, the program being capable of implementing the steps of the above method when executed by a processor.
[0080] Experimental example
[0081] In the experimental example, the loss function uses a ternary loss and a cross-entropy loss, and the weights of the two losses are 0.09. During training, the ADAM optimizer is used for optimization. The gait recognition network is trained on a server composed of two RTX 2080Ti (a kind of graphics card chip). The experimental accuracy results are shown in Table 1 below. The scores of the gait recognition method of the experimental example of the application on four indicators R-1 (used to evaluate the accuracy of the system), R-5 (used to evaluate the accuracy of the system), mAP (used to evaluate the accuracy of the system), and mINP (used to evaluate the ability of a model to search for the most difficult to find samples) all exceed the most advanced algorithms of the same kind. Among them, the R-1, mAP, and mINP indicators rank first, and the R-5 indicator ranks second, indicating that the method of the experimental example has high recognition accuracy.
[0082] Table 1
[0083]
[0084]
[0085] The speed results are shown in Figures 7-8 Compared with the baseline method, the method of the experimental example converges faster during training and can converge in a smaller number of iterations, greatly reducing the training time.
[0086] Comparing the test results with the same gait recognition method in the prior art, it can be seen that the method of the embodiment has high recognition accuracy and fast convergence speed.
[0087] Compared with the prior art, the experimental example of the application has the following advantages:
[0088] Good robustness, especially low sensitivity to clothing and viewing angle, more suitable for real-world scenarios.
[0089] Through the design of the graph module, the accuracy can be improved and the computational cost can be reduced, the model is small and the training time is short.
[0090] The experimental example of the application can be applied to the following scenarios:
[0091] 1. Security monitoring: Install a camera with gait recognition function in a factory, hospital, residential building or even open environment, which can play the functions of theft prevention, etc. like face recognition, through comprehensive and effective security control to ensure life and property safety.
[0092] 2. Criminal investigation monitoring: In most cases, the video often does not show the face or has low clarity, making it difficult to obtain effective information. However, after the application of gait recognition technology, the above phenomenon will be significantly improved, reducing the tracking difficulty and saving the time and cost of arrest.
[0093] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, a system or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.
[0094] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for carrying out each of the one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks.
[0095] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for carrying out each of the one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks.
[0096] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for carrying out each of the one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks.
[0097] The above further describes the present application in conjunction with specific / preferred embodiments, and cannot be deemed to limit the specific implementation of the present application to these descriptions. For those skilled in the art to which the present application belongs, without departing from the concept of the present application, they can make several substitutions or variations to the described embodiments, and these substitutions or variations shall be deemed to fall within the protection scope of the present application. In the description of the present application, the description of the terms "an embodiment", "some embodiments", "a preferred embodiment", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are contained in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In the case of no mutual contradiction, those skilled in the art can combine and combine the different embodiments or examples described in the present application and the features of the different embodiments or examples. Although the embodiments of the present application and their advantages have been described in detail, it should be understood that various changes, substitutions and modifications can be made herein without departing from the scope of protection of the patent application.
Claims
1. A gait recognition method characterized by comprising: The method comprises the following steps: S1: obtaining a target video segment, calculating and generating a skinned multi-person linear model frame by frame by a skinned multi-person linear model generation module to obtain shape parameters and pose parameters; S2: inputting the shape parameters and the pose parameters into a skinned multi-person linear model feature extraction module to obtain shape features of each frame and pose features of each frame; S3: performing set pooling processing on the shape features and the pose features respectively by an aggregation module, aggregating the shape features and the pose features to obtain time-aggregated shape features and time-aggregated pose features; S4: inputting the time-aggregated pose features into a human body part-based graph pooling module to obtain human body part features, and adding the human body part features to the time-aggregated shape features; S5: inputting the added features into a gait representation module to obtain a gait representation of a target person, and obtaining a gait recognition result according to a similarity with a known video segment; In step S4, inputting the time-aggregated pose features into the human body part-based graph pooling module to obtain human body part features specifically comprises the following steps: S41: constructing a horizontal feature list, wherein the horizontal features can capture human horizontal features and multi-scale features; S42: constructing a limb feature list, wherein the limb features can focus on limb features when a person walks; S43: constructing a torso list, which encodes human orientation information; S44: performing global pooling on the horizontal feature list, the limb feature list and the torso list respectively and adding them to obtain the human body part features; In step S5, the gait representation module processes the result of step S4 through a linear layer and batch normalization to obtain the gait representation of the target person.
2. The method of claim 1, wherein, Step S2 specifically comprises the following steps: S21: inputting the shape parameters into a multi-layer perception network to obtain shape features of each frame, wherein the shape parameters are represented by a vector; S22: normalizing the pose parameters and then inputting them into a graph network to obtain pose features of each frame, wherein the pose parameters are represented by a group of skeleton graphs.
3. The method of claim 2, wherein, In step S21, the multi-layer perception network comprises three units, each unit comprising a linear calculation layer, a batch normalization layer and a random loss layer.
4. The method of claim 2, wherein, In step S22, the graph network comprises three units, and each unit calculates the following formula: ； wherein, is the first unit input, ReLU is the ReLU activation function, and GConv is the graph convolution operator.
5. A method of establishing a gait recognition model, characterized by, The method comprises the following steps: A1: constructing a gait recognition network, wherein the gait recognition network comprises a skinned multi-person linear model generation module for calculating a target video segment frame by frame and generating a skinned multi-person linear model to obtain shape parameters and pose parameters; a skinned multi-person linear model feature extraction module for receiving the shape parameters and the pose parameters and obtaining shape features of each frame and pose features of each frame; an aggregation module for performing set pooling processing on the shape features and the pose features respectively, aggregating the shape features and the pose features to obtain time-aggregated shape features and time-aggregated pose features; a human body part based graph pooling module configured to obtain human body part features from the time-aggregated pose features, and to add the human body part features to the time-aggregated shape features; a gait representation module configured to obtain a gait representation of the target person from the added features, and to obtain a gait recognition result according to a similarity to known video clips; A2: training the gait recognition network with the person video data to obtain a gait recognition model; wherein the inputting of the time-aggregated pose features into the human body part based graph pooling module comprises the following steps: A11: constructing a horizontal feature list, wherein the horizontal features in the list are capable of capturing horizontal features and multi-scale features of the human body; A12: constructing a limb feature list, wherein the limb features in the list are capable of focusing on the features of the limbs when the person is walking; A13: constructing a torso list, wherein the torso list encodes the orientation information of the human body; A14: performing global pooling on the horizontal feature list, the limb feature list and the torso list respectively and adding them together to obtain the human body part features; the gait representation module obtains the gait representation of the target person by processing the result of step A14 through a linear layer and batch normalization.
6. The method of claim 5, wherein, In step A2, during the training process, a batch is formed by sampling from all video clips in the training set at each iteration, and a triplet loss and a cross-entropy loss are calculated between the gait representations obtained from the video clips in the batch, the triplet loss and the cross-entropy loss are calculated with a weight ratio of 9:1 to obtain a total loss, and the total loss is back-propagated to the network to update the network parameters.
7. A computer readable medium having stored thereon a computer program, characterized in that The program is executed by the processor to realize the steps of the method of any one of claims 1-4. The program is executed by the processor to realize the steps of the method of any one of claims 1-4.