Pedestrian trajectory generation method, device, equipment and storage medium

Abstract

The present application discloses a method, apparatus, device, and storage medium for generating pedestrian trajectories. The method comprises: acquiring at least one face image frame and at least one body image frame within a target acquisition range; determining at least one face region and a face identifier for each face region in each face image frame and generating a first pedestrian trajectory corresponding to the face identifier; determining at least one body region and a body identifier for each body region in each body image frame and generating a second pedestrian trajectory corresponding to the body identifier; establishing a relationship matching set between the face identifier and the body identifier; and fusing and correcting the first pedestrian trajectory and the second pedestrian trajectory based on the relationship matching set to obtain trajectory information of pedestrians within the target acquisition range. This method solves the problem of low accuracy of the determined pedestrian trajectory information.

Description

Technical Field

[0001] The present application generally relates to the field of intelligent monitoring technology, and specifically relates to a pedestrian trajectory generation method, device, equipment and storage medium. Background Art

[0002] For the activity trajectory of pedestrians, pedestrian trajectory information can generally be generated in the following two ways. One is a pedestrian trajectory information generation method based on facial information. This method processes facial images collected by cameras in different areas, arranges multiple facial images corresponding to the same face in chronological order, and finally generates pedestrian trajectory information on the map; the other method is a human body trajectory generation method. This method processes human body images collected by cameras in different areas, arranges multiple human body images corresponding to the same person in chronological order, and finally generates pedestrian trajectory information on the map.

[0003] However, in the process of generating pedestrian trajectory information based on facial information, face capture and recognition have high requirements on camera angles, which can easily lead to missed detections and missing trajectories; in the process of generating pedestrian trajectory information based on human body information, when a person's state changes suddenly, such as disappearing for a long time or being obscured, it is easy to misjudge the human body and the accuracy rate is low.

[0004] For example, Figure 1 As shown in the figure, the pedestrian trajectory based on facial information is shown as F_1. This is because when the pedestrian moves to area 3, the camera in area 3 does not capture the pedestrian's face due to occlusion, resulting in a missing portion of the trajectory. The pedestrian trajectory based on body information has two segments, B_1 and B_2. This is because the pedestrian's body identifier may switch from B_1 to B_2 due to rapid movement or severe occlusion for a long time during movement, resulting in an inaccurate trajectory. Summary of the Invention

[0005] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide a pedestrian trajectory generation method, apparatus, device and storage medium that can improve the accuracy of the generated pedestrian trajectory.

[0006] In a first aspect, the present application provides a method for generating pedestrian trajectories, comprising:

[0007] Acquire at least one face image frame and at least one body image frame within a target acquisition range;

[0008] Determine at least one face region in each face image frame and a face identifier for each face region and generate a first pedestrian trajectory corresponding to the face identifier;

[0009] Determine at least one human body region in each human body image frame and a human body identifier for each human body region and generate a second pedestrian trajectory corresponding to the human body identifier;

[0010] Establishing a relationship matching set between the face identifier and the body identifier, the relationship matching set including a plurality of pairing results, each pairing result indicating that a matching relationship exists between the face identifier and the body identifier;

[0011] Based on the relationship matching set, the first pedestrian trajectory and the second pedestrian trajectory are fused and corrected to obtain the trajectory information of the pedestrians within the target collection range;

[0012] In a second aspect, the present application provides a pedestrian trajectory generation device, comprising:

[0013] An acquisition module is configured to acquire at least one face image frame and at least one body image frame within a target acquisition range;

[0014] a determination module configured to determine at least one face region and a face identifier of each face region in each face image frame and generate a first pedestrian trajectory corresponding to the face identifier;

[0015] The determination module is further configured to determine at least one human body region in each human body image frame and a human body identifier for each human body region and generate a second pedestrian trajectory corresponding to the human body identifier;

[0016] An establishing module configured to establish a relationship matching set between the face identifier and the body identifier, the relationship matching set including a plurality of pairing results, each pairing result indicating a matching relationship between the face identifier and the body identifier;

[0017] A generation module is configured to fuse and correct the first pedestrian trajectory and the second pedestrian trajectory based on the relationship matching set to obtain the trajectory information of the pedestrian within the target collection range;

[0018] In a third aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement the method of the first aspect when executing the program.

[0019] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, the computer program being used to implement the method of the first aspect.

[0020] The technical solutions provided by the embodiments of the present application may have the following beneficial effects:

[0021] The pedestrian trajectory generation method, apparatus, device, and storage medium provided in the embodiments of the present application can obtain at least one face image frame and at least one body image frame within a target acquisition range; determine at least one face region and a face identifier for each face region in each face image frame and generate a first pedestrian trajectory corresponding to the face identifier; determine at least one body region and a body identifier for each body region in each body image frame and generate a second pedestrian trajectory corresponding to the body identifier; establish a relationship matching set between the face identifier and the body identifier; and fuse and correct the first pedestrian trajectory and the second pedestrian trajectory based on the relationship matching set to obtain trajectory information of pedestrians within the target acquisition range. The facial information and body information of pedestrians can be combined offline to obtain more accurate pedestrian trajectory information, thereby facilitating the tracking of pedestrian trajectories. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0023] Figure 1 A schematic diagram of pedestrian trajectory information provided in the related art;

[0024] Figure 2 A schematic diagram of the structure of a pedestrian trajectory generation system provided in an embodiment of the present application;

[0025] Figure 3 A flowchart of a method for generating pedestrian trajectories provided in an embodiment of the present application;

[0026] Figure 4 A schematic diagram of a world coordinate system provided in an embodiment of the present application;

[0027] Figure 5 A flowchart of another method for generating pedestrian trajectories provided in an embodiment of the present application;

[0028] Figure 6 A schematic diagram of pedestrian trajectory information provided in an embodiment of the present application;

[0029] Figure 7 A schematic diagram of the structure of a pedestrian trajectory generation device provided in an embodiment of the present application;

[0030] Figure 8 A schematic structural diagram of another pedestrian trajectory generating device provided in an embodiment of the present application;

[0031] Figure 9 A schematic structural diagram of another pedestrian trajectory generating device provided in an embodiment of the present application;

[0032] Figure 10A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the invention are shown in the accompanying drawings.

[0034] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0035] Figure 2 This is a schematic diagram of the structure of the pedestrian trajectory generation system provided by the embodiment of the present application. Figure 2 As shown, the system includes: at least one face camera 110, at least one body camera 120 and an image processing device 130, wherein the face camera 110 and the body camera 120 are respectively connected to the image processing device 130, and the connection can be a wired or wireless connection. The resolution of the face camera 110 can be 1080P, and the installation height is between 2 meters and 2.5 meters, so as to collect clear facial information. The face camera 110 can be a circular camera with an installation angle close to a horizontal angle; the resolution of the body camera 120 can be 720P, and the installation height is above 3 meters, so as to collect complete body information. The body camera can be a gun-type camera. The image processing device 130 can be a terminal device or server with image processing and data storage functions, such as a mobile computer or a desktop computer.

[0036] It can be understood that in the embodiment of the present application, the at least one face camera 110 and the at least one body camera 120 can be set in any location area where target monitoring is required. After the face camera 110 and the body camera 120 are powered on, they can collect picture information within their target collection range to generate a picture image, and send the picture image to the image processing device 130 in real time.

[0037] The image processing device 130 can receive the image. Since the camera (face camera and body camera) usually collects image information at a very high shooting frequency, the image processing device 130 can receive multiple image frames per unit time. The image processing device 130 can store the image in a video format. For a video file corresponding to a face camera, the image processing device 130 can obtain multiple first image frames in the video file, determine whether each first image frame contains a face area, and if so, determine that the first image frame is a face image frame, and determine at least one face area in each face image frame and a face marker for each face area. recognition, and generating a first pedestrian trajectory corresponding to the face identification; for a video file corresponding to a human body camera, the image processing device 130 can obtain multiple second image frames in the video file, determine whether each second image frame contains a human body area, and if so, determine that the second image frame is a human body image frame, and determine at least one human body area and a human body identification of each human body area in each human body image frame, and generate a second pedestrian trajectory corresponding to the human body identification; and establish a relationship matching set between the face identification and the human body identification; based on the relationship matching set, the first pedestrian trajectory and the second pedestrian trajectory are fused and corrected to obtain the trajectory information of the pedestrians within the target acquisition range.

[0038] The present invention provides a method for generating a pedestrian trajectory. Figure 2 In the image processing device of the system shown, Figure 3 As shown, the process includes:

[0039] Step 201: Acquire at least one face image frame and at least one body image frame within a target acquisition range.

[0040] In this step, for video files within the target acquisition range corresponding to the face camera, the video file can be parsed to obtain multiple first image frames, and face detection can be performed on each first image frame to determine whether the first image frame contains a face area. If so, the first image frame is determined to be a face image frame; if not, the first image frame is discarded; for video files within the target acquisition range corresponding to the human body camera, the video file can be parsed to obtain multiple second image frames, and human body detection can be performed on each second image frame to determine whether the second image frame contains a human body area. If so, the second image frame is determined to be a human body image frame; if not, the second image frame is discarded.

[0041] Step 202: Determine at least one face region in each face image frame and a face identifier for each face region and generate a first pedestrian trajectory corresponding to the face identifier.

[0042] In this step, the process of determining at least one face area in each face image frame and the face identifier of each face area may include: determining the face area position coordinates corresponding to each face area in each face image frame; performing feature extraction on each face area to obtain a face feature image corresponding to each face area; comparing the face feature image with a sample face feature image in a sample face feature library to obtain a face identifier corresponding to the face feature image; and establishing a correspondence between the face area position coordinates and the face identifier.

[0043] The process of comparing a facial feature image with sample facial feature images in a sample facial feature library to obtain a facial identifier corresponding to the facial feature image may include: comparing the facial feature image with each sample facial feature image in the sample facial feature library, determining a facial similarity value between the facial feature image and each sample facial feature image, obtaining at least one facial similarity value, determining whether the maximum facial similarity value among the at least one facial similarity value is greater than a preset facial similarity threshold, and if so, obtaining a facial identifier corresponding to the maximum facial similarity value, and determining the facial identifier corresponding to the maximum facial similarity value as a facial identifier of a face region; if not, setting a new facial identifier for the facial feature image, determining the facial identifier as a facial identifier of a face region, and storing the facial feature image and the corresponding facial identifier in the sample facial feature library, and updating the sample facial feature library. The facial similarity threshold can be determined based on actual needs, and the embodiments of the present application do not limit this.

[0044] It should be noted that, in the embodiment of the present application, there are two ways to obtain the sample face feature library:

[0045] In an optional real-time method, the sample facial feature library can be pre-established. The process of pre-establishing the sample facial feature library can be: first, capture a facial image of at least one target pedestrian, perform feature extraction on the facial image to obtain a sample facial feature image, assign a corresponding facial identifier to the sample facial feature image, establish a correspondence between each sample facial feature image and the corresponding facial identifier, and generate the sample facial feature library. The facial identifier can be the name of the target pedestrian.

[0046] In another optional real-time method, the sample facial feature library is not pre-established. After the pedestrian trajectory generation system is set up and run, the facial feature images in the preset number of facial image frames initially obtained can be determined as sample facial feature images, and a facial identifier can be set for each sample facial feature image. A correspondence between each sample facial feature image and the corresponding facial identifier is established to generate a sample facial feature library.

[0047] It should be noted that in an embodiment of the present application, the sample facial feature image corresponding to each facial identifier in the sample facial feature library can be updated after a first preset period. The process can be that after the first preset period, after obtaining a new facial feature image corresponding to the facial identifier in the sample facial feature library, the new facial feature image is determined as the updated sample facial image corresponding to the facial identifier, wherein the first preset period can be determined based on actual needs, and the embodiment of the present application does not limit this. For example, the sample facial feature library can be updated every three months. This can solve the problem of inaccurate comparison results caused by using the original sample facial feature image for feature comparison after a person's facial features change, thereby improving the recognition accuracy of the facial area.

[0048] It should be noted that, in an embodiment of the present application, each first image frame includes position information and time information, wherein the position information may be the position identifier of the face camera corresponding to the first image frame, and the time information may be the acquisition time of the first image frame. The face area determined based on the first image frame may include the face position information and time information corresponding to the face area.

[0049] Furthermore, in this step, a first pedestrian trajectory corresponding to the facial identifier can be generated. This process includes: determining the facial position information corresponding to each facial identifier, and concatenating the facial position information corresponding to the same facial identifier in the chronological order corresponding to each facial identifier to obtain the first pedestrian trajectory. It is understood that the number of the first pedestrian trajectory can be at least one.

[0050] Step 203: Determine at least one human region in each human image frame and a human identifier for each human region, and generate a second pedestrian trajectory corresponding to the human identifier.

[0051] In this step, the process of determining at least one human body region in each human body image frame and the human body identifier of each human body region may include: determining the human body region position coordinates corresponding to each human body region in each human body image frame; performing feature extraction on each human body region to obtain a human body feature image corresponding to each human body region; comparing the human body feature image with at least one sample human body feature image in a sample human body feature library to obtain a human body identifier corresponding to the human body feature image; performing coordinate system conversion on the human body region position coordinates to obtain converted human body region position coordinates; and establishing a correspondence between the converted human body region position coordinates and the human body identifier.

[0052] It can be understood that in the embodiment of the present application, the process of comparing the human feature image with at least one sample feature image in the sample human feature library to determine the human body identification of the human body area can refer to the above step 202, and the process of comparing the facial feature image with at least one sample feature image in the sample facial feature library to determine the facial identification of the facial area. The embodiment of the present application does not elaborate on this process. Among them, the method of obtaining the sample human feature library can refer to the method of obtaining the sample facial feature library in the above step 202. The embodiment of the present application does not elaborate on the process of obtaining the sample human feature library; similarly, the sample human images in the sample human feature library can also be updated after the first preset period to improve the recognition accuracy of the human body area.

[0053] It should be noted that in the embodiment of the present application, the position coordinates of the human body area in the second image frame are usually the position coordinates in the pixel coordinate system. It is usually necessary to perform coordinate conversion on the position coordinates of the human body area to the world coordinate system to obtain the updated position coordinates of the human body area to improve the accuracy of the position coordinates of the human body area finally determined.

[0054] The process of performing a coordinate system conversion on the human body region position coordinates to obtain the converted human body region position coordinates can include: establishing a world coordinate system with the ground as the origin, selecting multiple target positions on the ground, determining the position coordinates of each target position in the world coordinate system, and the position coordinates of each target position in the pixel coordinate system, determining a transformation matrix for converting each target position from the pixel coordinate system to the world coordinate system, and determining the transformation matrix as a homography matrix between the world coordinate system and the pixel coordinate system. It is understandable that since the world coordinate system takes the ground as the coordinate origin, the human foot region position coordinates can usually be converted to obtain the converted human body region position coordinates.

[0055] For example, Figure 4 As shown in the figure, a world coordinate system with O2 as the origin is established. The horizontal coordinate of the world coordinate system is X, and the vertical coordinate of the world coordinate system is Y. Four target positions C, D, E and F are arbitrarily selected. The position coordinates of the four target positions in the world coordinate system are C(X1, Y1), D(X2, Y2), E(X3, Y3) and F(X4, Y4), and the position coordinates of the four target positions in the pixel coordinate system are C(u1, v1), D(u2, v2), E(u3, v3) and F(u4, v4), respectively. Among them, the coordinate origin of the pixel coordinate system is O1, the horizontal coordinate of the pixel coordinate system is u, and the vertical coordinate of the pixel coordinate system is v. The transformation matrix H between the pixel coordinate system and the world coordinate system can be obtained, and the transformation matrix relationship between the pixel coordinate system and the world coordinate system can be determined as:

[0056]

[0057] Then, the coordinates of the human body region position can be transformed based on the homography matrix H to obtain the transformed human body region position coordinates P.

[0058] It should be noted that, in an embodiment of the present application, each second image frame includes position information and time information, wherein the position information may be the position identifier of the human body camera corresponding to the second image frame, and the time information may be the acquisition time of the second image frame. The human body area determined based on the second image frame may include the human body position information and time information corresponding to the human body area.

[0059] Furthermore, in this step, a second pedestrian trajectory corresponding to the human body identifier can be generated. This process includes: determining the human body position information corresponding to each human body identifier, and splicing the human body position information corresponding to the same human body identifier in the chronological order corresponding to each human body identifier to obtain the second pedestrian trajectory. It is understood that the number of the second pedestrian trajectory can be at least one.

[0060] It can be understood that in the embodiment of the present application, the execution order of the above-mentioned step 202 and step 203 does not affect the implementation of the pedestrian trajectory generation method. Step 202 can be executed before step 203, and step 202 can also be executed after step 203.

[0061] Step 204: Establish a relationship matching set between the face identifier and the body identifier.

[0062] In an embodiment of the present application, the trajectory information of the pedestrian can be generated offline. In order to improve the accuracy of the determined pedestrian trajectory information, it is necessary to establish a relationship matching set between the face identifier and the body identifier according to a preset period after obtaining a certain amount of face areas and face identifiers, as well as body areas and body identifiers. Based on the relationship matching set, the trajectory information of the pedestrians within the target collection range of each period is generated. More accurate trajectory information of the pedestrians within each cycle can be obtained. For example, starting at 23:50 every day, a matching relationship matching set between the face identifier and the body identifier is established based on at least one face area and its face identifier, and at least one body area and its identifier obtained on that day to generate the pedestrian trajectory information for that day.

[0063] In this step, if Figure 5 As shown, the process of establishing a relationship matching set between face identifiers and body identifiers may include:

[0064] Step 2041: Generate a cost matrix based on the face region position coordinates and face identifier of each face region, and the body region position coordinates and body identifier of each body region.

[0065] In this step, the process of generating the cost matrix may include: determining face and body groups and position identifiers of the face and body groups in the cost matrix to be generated based on the combined relationship between the face identifier of each face region and the body identifier of each body region; obtaining the face region position coordinates and the body region position coordinates corresponding to each face and body group; determining the overlap rate corresponding to each face and body group based on the face region position coordinates and the body region position coordinates corresponding to each face and body group; determining the cost value corresponding to each face and body group based on the difference between the standard value and the overlap rate; and generating the cost matrix based on the position identifiers and the cost value corresponding to the face and body group. The overlap rate is the ratio of the overlapping area of ​​the face region and the body region to the area of ​​the merged area, with the standard value being 1, and each element in the cost matrix is ​​used to indicate the possibility of an association between the face identifier and the body identifier.

[0066] For example, assuming that P face regions F and P face identifiers (f1, f2, f1…f3, f2) are obtained, and Q body regions B and Q body region identifiers (b1, b2, b4…b1, b2) are obtained, based on the combined relationship between the face identifier of each face region and the body identifier of each body region, it is determined that the face and body groups may include (f1, b1), (f1, b2), (f1, b4)…(f1, b1), (f1, b2); (f2, b1), (f2, b2), (f2, b4)…(f2, b1), (f2, b2);…(f2, b1), (f2, b2), (f2, b4)…(f2, b1), (f2, b2), then P*Q groups of face and body groups can be determined.

[0067] Then, the face region position coordinates and body region position coordinates corresponding to each face and body group are obtained. Assume that the face region position coordinates are (f x1 ,f y1 ;f x2 ,f y2 ), where (f x1 ,f y1 ) is the position coordinate of the upper left corner of the face area, (f x2 ,f y2 ) is the position coordinate of the lower right corner of the face area, assuming that the position coordinate of the human body area is (b x1 ,b y1 ; b x2 ,b y2 ), where (b x1 ,b y1 ) is the position coordinate of the upper left corner of the human body area, (b x2 ,b y2) is the position coordinate of the lower right corner of the human body area.

[0068] Furthermore, the overlap ratio (overlfp) corresponding to each face and body group is determined as Among them, f is the area of ​​the face region corresponding to the face and body group, B is the area of ​​the body region corresponding to the face and body group, and the difference between 1 and the overlap rate is calculated to determine the cost value corresponding to each face and body group. The cost matrix generated based on the position identifier and the cost value corresponding to the face and body grouping is shown in Table 1, where the cost matrix has P rows and Q columns. As shown in Table 1, the first row and the third row of the cost matrix are the cost values ​​corresponding to the face identifier f1, indicating that the face identifier of two face regions among the P human face regions is f1.

[0069] Table 1

[0070]

[0071]

[0072] Step 2042: Process the cost matrix to obtain a relationship matching set.

[0073] In this step, the cost matrix is ​​processed to obtain a relationship matching set. The process can be: using the Hungarian matching algorithm model to process the cost matrix to obtain the relationship matching set, wherein the input of the Hungarian matching algorithm model is the cost value corresponding to each face identifier and the cost value corresponding to each body identifier in the cost matrix, and the output of the Hungarian matching algorithm model is a relationship matching set between the face identifier and the body identifier. The relationship matching set includes multiple pairing results, and each pairing result indicates that there is a matching relationship between the face identifier and the body identifier.

[0074] Step 205: Based on the relationship matching set, the first pedestrian trajectory and the second pedestrian trajectory are fused and corrected to obtain the trajectory information of the pedestrian within the target collection range.

[0075] In this step, the second pedestrian trajectory can be corrected based on the first pedestrian trajectory corresponding to the relationship matching set to obtain the trajectory information of the pedestrian within the target collection range; the first pedestrian trajectory can also be corrected based on the second pedestrian trajectory corresponding to the relationship matching set to obtain the trajectory information of the pedestrian within the target collection range. In this embodiment of the present application, the first pedestrian trajectory and the second pedestrian trajectory are fused and corrected based on the relationship matching set to obtain the trajectory information of the pedestrian within the target collection range. There are two optional implementation methods:

[0076] In an optional implementation, the process of fusing and correcting the first pedestrian trajectory and the second pedestrian trajectory based on the relationship matching set to obtain the trajectory information of the pedestrian within the target acquisition range may be:

[0077] Determine the face identifier to be fused in the relationship matching set, the face identifier to be fused refers to the face identifier of the pedestrian corresponding to the trajectory information to be generated; obtain a first pedestrian trajectory corresponding to the face identifier to be fused, the first pedestrian trajectory includes multiple face position coordinates associated with the face identifier to be fused; based on the face identifier to be fused, determine the pairing result related to the face identifier to be fused, the pairing result includes the human body identifier corresponding to the face identifier to be fused; obtain a second pedestrian trajectory corresponding to the human body identifier corresponding to the face identifier to be fused, the second pedestrian trajectory includes multiple human body position coordinates associated with the human body identifier corresponding to the face identifier to be fused; correct the multiple human body position coordinates associated with the human body identifier corresponding to the face identifier to be fused to obtain the trajectory information of the pedestrian.

[0078] Among them, multiple human body position coordinates associated with the human body identifier corresponding to the face identifier to be fused are corrected to obtain the trajectory information of the pedestrian, including: determining the union of multiple face position coordinates and multiple human body position coordinates to obtain the corrected human body position information associated with the human body identifier corresponding to the face identifier to be fused; splicing the corrected human body position information according to the time information associated with the human body identifier corresponding to the face identifier to be fused to obtain the trajectory information of the pedestrian.

[0079] In another optional embodiment, the process of fusing and correcting the first pedestrian trajectory and the second pedestrian trajectory based on the relationship matching set to obtain the trajectory information of the pedestrian within the target collection range may be:

[0080] Determine the human body identifier to be fused in the relationship matching set, the human body identifier to be fused refers to the human body identifier of the pedestrian corresponding to the trajectory information to be generated; obtain a second pedestrian trajectory corresponding to the human body identifier to be fused, the second pedestrian trajectory includes multiple human body position coordinates associated with the human body identifier to be fused; based on the human body identifier to be fused, determine a pairing result related to the human body identifier to be fused, the pairing result includes the face identifier corresponding to the human body identifier to be fused; obtain a first pedestrian trajectory corresponding to the face identifier corresponding to the human body identifier to be fused, the first pedestrian trajectory includes multiple face position coordinates associated with the face identifier corresponding to the human body identifier to be fused; correct the multiple face position coordinates associated with the face identifier corresponding to the human body identifier to be fused to obtain the trajectory information of the pedestrian.

[0081] Among them, multiple face position coordinates associated with the face identifier corresponding to the human body identifier to be fused are corrected to obtain the pedestrian's trajectory information, including: determining the union of multiple face position coordinates and multiple body position coordinates to obtain the corrected face position information associated with the face identifier corresponding to the human body identifier to be fused; according to the time information associated with the face identifier corresponding to the human body identifier to be fused, the corrected face position information is spliced ​​to obtain the pedestrian's trajectory information.

[0082] For example, Figure 6 As shown, it is assumed that the determined relationship matching set between the face identifier and the body identifier includes (F_1, B_1) and (F_1, B_2), F_1 ​​is the face identifier, and B_1 and B_2 are the body identifiers.

[0083] Optionally, the face identifier F_1 to be fused in the relationship matching set can be determined, and the first pedestrian trajectory corresponding to the face identifier F_1 to be fused can be obtained, such as Figure 1 The pedestrian trajectory F_1 in the image is obtained, and the two pairing results related to the face identifier F_1 to be fused are determined to be (F_1, B_1) and (F_1, B_2). The human body identifiers in the two pairing results are obtained as B_1 and B_2. The second pedestrian trajectory associated with the human body identifier B_1 corresponding to the face identifier F_1 is obtained, such as Figure 1 The pedestrian trajectory B_1 in , the second pedestrian trajectory associated with the human body identifier B_2 corresponding to the face identifier F_1, such as Figure 1 The pedestrian trajectory B_2 in the figure is as follows. The first pedestrian trajectory F_1 includes multiple face position coordinate regions 1 and 2 associated with the face identifier F_1 to be fused, the second pedestrian trajectory B_1 includes multiple body position coordinate regions 1 and 2 associated with the body identifier B_1 corresponding to the face identifier F_1 to be fused, and the second pedestrian trajectory B_2 includes multiple body position coordinate regions 2, 3, and 4 associated with the body identifier B_2 corresponding to the face identifier F_1 to be fused.

[0084] Next, multiple face position coordinate regions 1 and 2, as well as multiple human body position coordinate regions 1 and 2; the union of regions 2, 3, and 4 are determined to obtain the corrected human body position information regions 1, 2, 3, and 4 associated with the human body identifier B_1 or B_2 corresponding to the face identifier F_1 to be fused; according to the time information in the face region corresponding to the face identifier F_1 in the pairing relationship (F_1, B_1) and (F_1, B_2), the human body position information is spliced ​​to obtain the pedestrian trajectory information such as Figure 6 Pedestrian trajectory B_1 in .

[0085] Optionally, the human body identifiers B_1 and B_2 to be fused in the relationship matching set can be determined, and the second pedestrian trajectories corresponding to the human body identifiers B_1 and B_2 to be fused can be obtained, such as Figure 1 The pedestrian trajectory B_1 and pedestrian trajectory B_2 in the image are obtained, and the two pairing results related to the human body identifiers B_1 and B_2 to be fused are determined to be (F_1, B_1) and (F_1, B_2). The face identifier F_1 in the two pairing results is obtained, and the first pedestrian trajectory associated with the face identifier F_1 corresponding to the human body identifiers B_1 and B_2 is obtained, such as Figure 1 The pedestrian trajectory F_1 in FIG. The first pedestrian trajectory F_1 includes multiple human position coordinate regions 1 and 2 associated with the human marker F_1 to be fused, the second pedestrian trajectory B_1 includes multiple facial position coordinate regions 1 and 2 associated with the facial marker B_1 corresponding to the human marker F_1 to be fused, and the second pedestrian trajectory B_2 includes multiple facial position coordinate regions 2, 3, and 4 associated with the facial marker B_2 corresponding to the human marker F_1 to be fused.

[0086] Next, multiple human body position coordinate regions 1 and 2, as well as multiple face position coordinate regions 1 and 2; the union of regions 2, 3, and 4 are determined to obtain the corrected face position information regions 1, 2, 3, and 4 associated with the face identifier B_1 or B_2 corresponding to the human body identifier F_1 to be fused; according to the time information in the human body regions corresponding to the human body identifiers B_1 and B_2 in the pairing relationships (F_1, B_1) and (F_1, B_2), the face position information is spliced ​​to obtain the pedestrian trajectory information such as Figure 6 Pedestrian trajectory F_1 in .

[0087] In summary, the pedestrian trajectory generation method provided in the embodiments of the present application can obtain at least one face image frame and at least one body image frame within the target acquisition range; determine at least one face region and a face identifier for each face region in each face image frame and generate a first pedestrian trajectory corresponding to the face identifier; determine at least one body region and a body identifier for each body region in each body image frame and generate a second pedestrian trajectory corresponding to the body identifier; establish a relationship matching set between the face identifier and the body identifier; and fuse and correct the first pedestrian trajectory and the second pedestrian trajectory based on the relationship matching set to obtain trajectory information of pedestrians within the target acquisition range. The facial information and body information of pedestrians can be combined offline to obtain more accurate pedestrian trajectory information, making it easier to track pedestrian trajectories.

[0088] The embodiment of the present application provides a pedestrian trajectory generation device, such as Figure 7 As shown, the device 30 includes:

[0089] An acquisition module 301 is configured to acquire at least one face image frame and at least one body image frame within a target acquisition range;

[0090] A determination module 302 is configured to determine at least one face region and a face identifier of each face region in each face image frame and generate a first pedestrian trajectory corresponding to the face identifier;

[0091] The determination module 302 is further configured to determine at least one human body region in each human body image frame and a human body identifier for each human body region and generate a second pedestrian trajectory corresponding to the human body identifier;

[0092] An establishing module 303 is configured to establish a relationship matching set between the face identifier and the body identifier, where the relationship matching set includes a plurality of pairing results, each pairing result indicating a matching relationship between the face identifier and the body identifier;

[0093] The generation module 304 is configured to fuse and correct the first pedestrian trajectory and the second pedestrian trajectory based on the relationship matching set to obtain the trajectory information of the pedestrian within the target collection range.

[0094] Optionally, the establishment module 303 is configured to:

[0095] A cost matrix is ​​generated based on the face region position coordinates and face identifier of each face region, and the body region position coordinates and body identifier of each body region, where each element in the cost matrix is ​​used to indicate the possibility of an association between the face identifier and the body identifier;

[0096] The cost matrix is ​​processed to obtain a set of relationship matching.

[0097] Optionally, the establishment module 303 is configured to:

[0098] Based on the combined relationship between the face identifier of each face region and the body identifier of each body region, determine the face and body groupings and the position identifiers of the face and body groupings in the cost matrix to be generated;

[0099] Obtaining the facial region position coordinates and the human body region position coordinates corresponding to each facial and human body group;

[0100] Determining an overlap rate corresponding to each face and body group based on the face region position coordinates and the body region position coordinates corresponding to each face and body group;

[0101] Determine the cost value corresponding to each face and body group based on the difference between the standard value and the overlap rate;

[0102] A cost matrix is ​​generated based on the location identifiers and the cost values ​​corresponding to the face and body groups.

[0103] Optionally, the generating module 304 is configured to:

[0104] Determine the face identifier to be fused in the relationship matching set, where the face identifier to be fused refers to the face identifier of the pedestrian corresponding to the trajectory information to be generated;

[0105] Acquire a first pedestrian trajectory corresponding to the face identifier to be fused, where the first pedestrian trajectory includes multiple face position coordinates associated with the face identifier to be fused;

[0106] Determine, based on the face identifier to be fused, a pairing result related to the face identifier to be fused, the pairing result including a body identifier corresponding to the face identifier to be fused;

[0107] Acquire a second pedestrian trajectory corresponding to the human body identifier corresponding to the face identifier to be fused, the second pedestrian trajectory including a plurality of human body position coordinates associated with the human body identifier corresponding to the face identifier to be fused;

[0108] The multiple human body position coordinates associated with the human body identifier corresponding to the face identifier to be fused are corrected to obtain the pedestrian trajectory information.

[0109] Optionally, the generating module 304 is configured to:

[0110] Determine the human body identifier to be fused in the relationship matching set, where the human body identifier to be fused refers to the human body identifier of the pedestrian corresponding to the trajectory information to be generated;

[0111] Acquire a second pedestrian trajectory corresponding to the human body identifier to be fused, where the second pedestrian trajectory includes a plurality of human body position coordinates associated with the human body identifier to be fused;

[0112] Based on the human body identifier to be fused, determining a pairing result related to the human body identifier to be fused, the pairing result including a face identifier corresponding to the human body identifier to be fused;

[0113] Acquire a first pedestrian trajectory corresponding to the face identifier corresponding to the human body identifier to be fused, the first pedestrian trajectory including multiple face position coordinates associated with the face identifier corresponding to the human body identifier to be fused;

[0114] The position coordinates of multiple faces associated with the face identifiers corresponding to the human body identifier to be fused are corrected to obtain the trajectory information of the pedestrian.

[0115] Optionally, the generating module 304 is configured to:

[0116] Determine a union of multiple face position coordinates and multiple body position coordinates to obtain corrected body position information associated with the body identifier corresponding to the face identifier to be fused;

[0117] According to the time information associated with the human body identifier corresponding to the face identifier to be fused, the corrected human body position information is spliced ​​to obtain the pedestrian's trajectory information.

[0118] Optionally, the generating module 304 is configured to:

[0119] Determine a union of multiple face position coordinates and multiple body position coordinates to obtain corrected face position information associated with the face identifier corresponding to the body identifier to be fused;

[0120] According to the time information associated with the face identifier corresponding to the human body identifier to be fused, the corrected face position information is spliced ​​to obtain the pedestrian's trajectory information.

[0121] Optional, such as Figure 8 As shown, the determination module 302 includes:

[0122] The face tracking unit 3021 is configured to determine the facial region position coordinates corresponding to each facial region in each facial image frame;

[0123] The face comparison unit 3022 is configured to extract features from each face region to obtain a face feature image corresponding to each face region;

[0124] Compare the facial feature image with sample facial feature images in a sample facial feature library to obtain a facial identifier corresponding to the facial feature image;

[0125] The face identifier generating unit 3023 is configured to establish a correspondence between the face region position coordinates and the face identifier.

[0126] Optional, such as Figure 9 As shown, the determination module 302 includes:

[0127] The human body tracking unit 3024 is configured to determine the human body region position coordinates corresponding to each human body region in each human body image frame;

[0128] The human body comparison unit 3025 is configured to extract features from each human body region to obtain a human body feature image corresponding to each human body region;

[0129] Comparing the human body feature image with at least one sample human body feature image in a sample human body feature library to obtain a human body identifier corresponding to the human body feature image;

[0130] A coordinate system conversion unit 3026 is configured to perform coordinate system conversion on the human body region position coordinates to obtain converted human body region position coordinates;

[0131] The human body identifier generating unit 3027 is configured to establish a corresponding relationship between the converted human body region position coordinates and the human body identifier.

[0132] In summary, the pedestrian trajectory generation device provided in the embodiment of the present application can obtain at least one face image frame and at least one body image frame within the target acquisition range; determine at least one face region and a face identifier of each face region in each face image frame and generate a first pedestrian trajectory corresponding to the face identifier; determine at least one body region and a body identifier of each body region in each body image frame and generate a second pedestrian trajectory corresponding to the body identifier; establish a relationship matching set between the face identifier and the body identifier; and fuse and correct the first pedestrian trajectory and the second pedestrian trajectory based on the relationship matching set to obtain the trajectory information of the pedestrian within the target acquisition range. The pedestrian's face information and body information can be combined in an offline manner to obtain more accurate pedestrian trajectory information, so as to facilitate the tracking of the pedestrian's trajectory.

[0133] Figure 10 1 is a computer device according to an exemplary embodiment, which includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 402 or programs loaded from a storage unit into a random access memory (RAM) 403. Various programs and data required for system operation are also stored in RAM 403. CPU 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to bus 404.

[0134] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, and the like; an output section including a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 408 including a hard disk and the like; and a communication section 409 including a network interface card such as a LAN card or a modem. The communication section 409 performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface 405 as needed. Removable media 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like, is installed in the drive 410 as needed, so that computer programs read therefrom can be installed into the storage section 408 as needed.

[0135] In particular, according to the embodiments of the present application, the above Figure 3 and Figure 5The described process can be implemented as a computer software program. For example, various embodiments of the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication portion, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU) 401, the above-mentioned functions defined in the system of the present application are performed.

[0136] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0137] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods, devices and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0138] The units involved in the embodiments described in the present application may be implemented by software or by hardware, and the described units may also be provided in a processor. The names of these units do not, in some cases, constitute a limitation on the units themselves. The described units or modules may also be provided in a processor, for example, they may be described as: a processor including an acquisition module, a determination module, an establishment module, and a generation module. The names of these units or modules do not, in some cases, constitute a limitation on the units or modules themselves, for example, the acquisition module may also be described as "an acquisition module for acquiring at least one face image frame and at least one body image frame within the target acquisition range."

[0139] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments, or may exist independently without being incorporated into the electronic device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device implements the pedestrian trajectory generation method described in the above embodiments.

[0140] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention herein is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features having similar functions disclosed in this application.

Claims

1. A pedestrian trajectory generation method, characterized in that: include: Acquire at least one face image frame and at least one body image frame within a target acquisition range; Determine at least one face region in each face image frame and a face identifier of each face region and generate a first pedestrian trajectory corresponding to the face identifier; Determine at least one human body region in each of the human body image frames and a human body identifier for each of the human body regions and generate a second pedestrian trajectory corresponding to the human body identifier; Establishing a relationship matching set between the face identifier and the body identifier, the relationship matching set including a plurality of pairing results, each pairing result indicating that a matching relationship exists between the face identifier and the body identifier; Based on the relationship matching set, the first pedestrian trajectory and the second pedestrian trajectory are fused and corrected to obtain the trajectory information of the pedestrian within the target collection range; The generating of the first pedestrian trajectory corresponding to the face identifier includes: Determine the facial position information corresponding to each of the facial identifiers, and splice the facial position information corresponding to the same facial identifier in the chronological order corresponding to the facial identifiers to obtain the first pedestrian trajectory; Generating a second pedestrian trajectory corresponding to the human body identifier includes: Determine the human body position information corresponding to each human body identifier, and splice the human body position information corresponding to the same human body identifier in the chronological order corresponding to the human body identifiers to obtain the second pedestrian trajectory; The establishing of a relationship matching set between the face identifier and the body identifier includes: generating a cost matrix based on the face region position coordinates and face identifier of each face region, and the body region position coordinates and body identifier of each body region, wherein each element in the cost matrix is ​​used to indicate a possibility of an association between the face identifier and the body identifier; The cost matrix is ​​processed to obtain the relationship matching set.

2. The method according to claim 1, characterized in that The generating of the cost matrix based on the face region position coordinates and the face identifier of each face region, and the body region position coordinates and the body identifier of each body region, comprises: Determining face and body groups, and position identifiers of the face and body groups in a cost matrix to be generated, based on a combination relationship between a face identifier of each face region and a body identifier of each body region; Obtaining the face region position coordinates and the body region position coordinates corresponding to each face and body group; Determining an overlap rate corresponding to each face and body group based on the face region position coordinates and the body region position coordinates corresponding to each face and body group; Determining a cost value corresponding to each of the face and body groups based on a difference between a standard value and the overlap rate; A cost matrix is ​​generated based on the location identifiers and the cost values ​​corresponding to the face and body groups.

3. The method according to claim 1, characterized in that The fusing and correcting the first pedestrian trajectory and the second pedestrian trajectory based on the relationship matching set to obtain the trajectory information of the pedestrian within the target collection range includes: Determining a face identifier to be fused in the relationship matching set, where the face identifier to be fused refers to a face identifier of a pedestrian corresponding to the trajectory information to be generated; Acquire the first pedestrian trajectory corresponding to the face identifier to be fused, where the first pedestrian trajectory includes multiple face position coordinates associated with the face identifier to be fused; Determining, based on the face identifier to be fused, a pairing result related to the face identifier to be fused, the pairing result including a body identifier corresponding to the face identifier to be fused; Acquire a second pedestrian trajectory corresponding to the human body identifier corresponding to the face identifier to be fused, wherein the second pedestrian trajectory includes a plurality of human body position coordinates associated with the human body identifier corresponding to the face identifier to be fused; Correct a plurality of human body position coordinates associated with the human body identifier corresponding to the face identifier to be fused to obtain the trajectory information of the pedestrian.

4. The method according to claim 1, wherein The fusing and correcting the first pedestrian trajectory and the second pedestrian trajectory based on the relationship matching set to obtain the trajectory information of the pedestrian within the target collection range includes: Determining a human body identifier to be fused in the relationship matching set, where the human body identifier to be fused refers to a human body identifier of a pedestrian corresponding to the trajectory information to be generated; Acquire the second pedestrian trajectory corresponding to the human body identifier to be fused, where the second pedestrian trajectory includes a plurality of human body position coordinates associated with the human body identifier to be fused; Based on the human body identifier to be fused, determining a pairing result related to the human body identifier to be fused, the pairing result including a face identifier corresponding to the human body identifier to be fused; Acquire a first pedestrian trajectory corresponding to the face identifier corresponding to the human body identifier to be fused, wherein the first pedestrian trajectory includes a plurality of face position coordinates associated with the face identifier corresponding to the human body identifier to be fused; Correct the multiple face position coordinates associated with the face identifier corresponding to the human body identifier to be fused to obtain the trajectory information of the pedestrian.

5. The method according to claim 3, characterized in that The step of correcting a plurality of human body position coordinates associated with the human body identifier corresponding to the face identifier to be fused to obtain the trajectory information of the pedestrian includes: Determine a union of the plurality of face position coordinates and the plurality of body position coordinates to obtain corrected body position information associated with the body identifier corresponding to the face identifier to be fused; The corrected human body position information is spliced ​​according to time information associated with the human body identifier corresponding to the face identifier to be fused to obtain the trajectory information of the pedestrian.

6. The method according to claim 4, characterized in that The step of correcting the multiple facial position coordinates associated with the facial identifier corresponding to the human body identifier to be fused to obtain the trajectory information of the pedestrian includes: Determine a union of the plurality of face position coordinates and the plurality of body position coordinates to obtain corrected face position information associated with the face identifier corresponding to the body identifier to be fused; The corrected face position information is spliced ​​according to time information associated with the face identifier corresponding to the human body identifier to be fused to obtain the trajectory information of the pedestrian.

7. The method according to claim 1, characterized in that The determining of at least one face region in each face image frame and a face identifier of each face region includes: Determining facial region position coordinates corresponding to each facial region in each facial image frame; Performing feature extraction on each of the face regions to obtain a face feature image corresponding to each of the face regions; Comparing the facial feature image with sample facial feature images in a sample facial feature library to obtain a facial identifier corresponding to the facial feature image; A correspondence between the facial region position coordinates and the facial identifier is established.

8. The method according to claim 1, characterized in that The determining of at least one human body region in each human body image frame and a human body identifier of each human body region includes: Determining the human body region position coordinates corresponding to each human body region in each of the human body image frames; Performing feature extraction on each of the human body regions to obtain a human body feature image corresponding to each of the human body regions; Comparing the human body feature image with at least one sample human body feature image in a sample human body feature library to obtain a human body identifier corresponding to the human body feature image; Performing coordinate system conversion on the human body region position coordinates to obtain converted human body region position coordinates; A corresponding relationship between the converted human body region position coordinates and the human body identifier is established.

9. A pedestrian trajectory generation device, characterized in that: include: An acquisition module is configured to acquire at least one face image frame and at least one body image frame within a target acquisition range; a determination module configured to determine at least one face region in each face image frame and a face identifier of each face region and generate a first pedestrian trajectory corresponding to the face identifier; The determination module is further configured to determine at least one human body region in each of the human body image frames and a human body identifier of each of the human body regions and generate a second pedestrian trajectory corresponding to the human body identifier; an establishing module configured to establish a relationship matching set between the face identifier and the body identifier, the relationship matching set including a plurality of pairing results, each pairing result indicating a matching relationship between the face identifier and the body identifier; A generating module is configured to fuse and correct the first pedestrian trajectory and the second pedestrian trajectory based on the relationship matching set to obtain trajectory information of pedestrians within the target collection range; The generating of the first pedestrian trajectory corresponding to the face identifier includes: Determine the facial position information corresponding to each of the facial identifiers, and splice the facial position information corresponding to the same facial identifier in the chronological order corresponding to the facial identifiers to obtain the first pedestrian trajectory; Generating a second pedestrian trajectory corresponding to the human body identifier includes: Determine the human body position information corresponding to each human body identifier, and splice the human body position information corresponding to the same human body identifier in the chronological order corresponding to the human body identifiers to obtain the second pedestrian trajectory; The establishing of a relationship matching set between the face identifier and the body identifier includes: generating a cost matrix based on the face region position coordinates and face identifier of each face region, and the body region position coordinates and body identifier of each body region, wherein each element in the cost matrix is ​​used to indicate a possibility of an association between the face identifier and the body identifier; The cost matrix is ​​processed to obtain the relationship matching set.

10. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement the method according to any one of claims 1 to 8 when executing the program.

11. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program is used to implement the method according to any one of claims 1 to 8.

Images