Multi-picture target matching method and device, electronic equipment and storage medium

Abstract

The application provides a multi-picture target matching method and device, electronic equipment and storage medium, and relates to the technical field of data analysis. The method can convert the coordinate information of each target in a first picture to a coordinate system of a second picture by calculating the coordinate transformation matrix between different pictures, and obtain the mapping coordinate information of each target in the first picture in the second picture. Based on the cost information of the mapping coordinate information and the coordinate information of each target in the second picture, the matching degree between the target corresponding to each mapping coordinate information and the target corresponding to each coordinate information in the second picture can be determined. Then, based on the matching degree between the target corresponding to each mapping coordinate information and the target corresponding to each coordinate information in the second picture, the matching result between each target in the first picture and each target in the second picture can be determined. The matching efficiency of the method is high.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data analysis, in particular to a multi-picture target matching method and device, electronic equipment and storage medium. BACKGROUND

[0002] In a target tracking scene, matching of the same target in pictures taken by multiple groups of cameras in parallel positions but with different viewing angles is involved to achieve one-to-one correspondence and tracking of the same target in multiple scenes across pictures.

[0003] Therefore, how to match targets in multiple pictures is a problem to be solved. SUMMARY

[0004] The present application aims to provide a multi-picture target matching method and device, electronic equipment and storage medium to achieve accurate matching of targets in multiple pictures in view of the deficiencies in the prior art.

[0005] To achieve the above object, the technical solutions adopted by the embodiments of the present application are as follows:

[0006] In a first aspect, the embodiments of the present application provide a multi-picture target matching method, comprising:

[0007] According to the plurality of key points marked in advance in the multiple shooting pictures with different shooting angles, coordinate transformation matrices between each two of the multiple shooting pictures are calculated;

[0008] According to the coordinate sequence of the first shooting picture and the coordinate transformation matrix between the first shooting picture and the second shooting picture, a mapping coordinate sequence of the first shooting picture in the coordinate system of the second shooting picture is determined, the coordinate sequence of the first shooting picture comprising coordinate information of each target in the first shooting picture, and the mapping coordinate sequence comprising mapping coordinate information of each target in the first shooting picture in the coordinate system of the second shooting picture;

[0009] According to the mapping coordinate sequence and the coordinate sequence of the second shooting picture, cost information between each mapping coordinate information and coordinate information of each target in the second shooting picture is determined, the coordinate sequence of the second shooting picture comprising coordinate information of each target in the second shooting picture;

[0010] According to the cost information between each mapping coordinate information and coordinate information of each target in the second shooting picture, a matching relationship between each target in the first shooting picture and each target in the second shooting picture is determined.

[0011] Optionally, the calculation of the coordinate transformation matrix between each pair of the photographed pictures according to the plurality of key points marked in each photographed picture comprises:

[0012] According to the coordinate information of each key point in the first photographed picture and the coordinate information of each key point in the second photographed picture, an affine transformation algorithm is used to determine the coordinate transformation matrix between the first photographed picture and the second photographed picture.

[0013] Optionally, the determination of the mapping coordinate sequence of the first photographed picture in the coordinate system of the second photographed picture according to the coordinate sequence of the first photographed picture and the coordinate transformation matrix between the first photographed picture and the second photographed picture comprises:

[0014] According to the coordinate information of each target in the first photographed picture and the coordinate transformation matrix between the first photographed picture and the second photographed picture, the mapping coordinate information of each target in the first photographed picture in the coordinate system of the second photographed picture is determined respectively;

[0015] According to the mapping coordinate information of each target in the coordinate system of the second photographed picture, the mapping coordinate sequence of the first photographed picture in the coordinate system of the second photographed picture is determined, wherein the order of each target in the mapping coordinate sequence corresponds to the order of each target in the coordinate sequence of the first photographed picture.

[0016] Optionally, the determination of the cost information between each mapping coordinate information and the coordinate information of each target in the second photographed picture according to the mapping coordinate sequence and the coordinate sequence of the second photographed picture comprises:

[0017] The overlap rate between each mapping coordinate information and the coordinate information of each target in the second photographed picture is calculated in sequence and pair by pair;

[0018] According to the overlap rate between each mapping coordinate information and the coordinate information of each target in the second photographed picture, the cost information between each mapping coordinate information and the coordinate information of each target in the second photographed picture is determined.

[0019] Optionally, the calculation of the overlap rate between each mapping coordinate information and the coordinate information of each target in the second photographed picture in sequence and pair by pair comprises:

[0020] The intersection between the first mapping coordinate information and the first coordinate information in the second photographed picture is determined to obtain an intersection calculation result;

[0021] The union between the first mapping coordinate information and the first coordinate information in the second photographed picture is determined to obtain a union calculation result;

[0022] divide the intersection calculation result by the union calculation result to obtain an overlap rate between the first mapping coordinate information and the first coordinate information in the second captured picture, so as to sequentially obtain an overlap rate between each mapping coordinate information and the coordinate information of each target in the second captured picture.

[0023] Optionally, the determining, according to the cost information between each mapping coordinate information and the coordinate information of each target in the second captured picture, of the matching relationship between each target in the first captured picture and each target in the second captured picture, comprises:

[0024] determining, according to the cost information between each mapping coordinate information and the coordinate information of each target in the second captured picture, a plurality of candidate matching results and a cost sum corresponding to each matching result, the matching result comprising a matching relationship between each mapping coordinate information and one coordinate information in the second captured picture, and the cost sum being a sum of the cost information between each mapping coordinate information and the matched coordinate information in the second captured picture under the matching result.

[0025] determining a target matching result according to the cost sum corresponding to each candidate matching result.

[0026] determining the matching relationship between each target in the first captured picture and each target in the second captured picture according to the target matching result.

[0027] Optionally, the determining, according to the target matching result, of the matching relationship between each target in the first captured picture and each target in the second captured picture, comprises:

[0028] determining, from the second captured picture, a second target matched by each mapping coordinate information according to the target coordinate information in the second captured picture corresponding to each mapping coordinate information under the target matching result.

[0029] determining, from the first captured picture, a first target matched by each mapping coordinate information according to the mapping relationship between each mapping coordinate information and the coordinate information of each target in the first captured picture.

[0030] determining that the first target in the first captured picture and the second target in the second captured picture match each other.

[0031] In a second aspect, the embodiments of the present application further provide a multi-picture target matching device, comprising a calculation module, a determination module and a matching module.

[0032] The calculation module is configured to calculate coordinate transformation matrices between each pair of captured pictures according to a plurality of key points pre-marked in the captured pictures with different shooting angles.

[0033] The determining module is configured to determine a mapping coordinate sequence of the first photograph in a coordinate system of the second photograph according to the coordinate sequence of the first photograph and a coordinate transformation matrix of the first photograph and the second photograph, the coordinate sequence of the first photograph comprising coordinate information of each target in the first photograph, and the mapping coordinate sequence comprising mapping coordinate information of each target in the first photograph in the coordinate system of the second photograph.

[0034] The determining module is configured to determine cost information between each mapping coordinate information and coordinate information of each target in the second photograph according to the mapping coordinate sequence and the coordinate sequence of the second photograph.

[0035] The matching module is configured to determine a matching relationship between each target in the first photograph and each target in the second photograph according to the cost information between each mapping coordinate information and coordinate information of each target in the second photograph.

[0036] Optionally, the calculating module is specifically configured to determine the coordinate transformation matrix between the first photograph and the second photograph by using an affine transformation algorithm according to the coordinate information of each key point in the first photograph and the coordinate information of each key point in the second photograph.

[0037] Optionally, the determining module is specifically configured to determine mapping coordinate information of each target in the first photograph in the coordinate system of the second photograph according to the coordinate information of each target in the first photograph and the coordinate transformation matrix of the first photograph and the second photograph.

[0038] The mapping coordinate sequence of the first photograph in the coordinate system of the second photograph is determined according to the mapping coordinate information of each target in the first photograph in the coordinate system of the second photograph, and the order of each target in the mapping coordinate sequence corresponds to the order of each target in the coordinate sequence of the first photograph.

[0039] Optionally, the determining module is specifically configured to calculate an overlap rate between each mapping coordinate information and coordinate information of each target in the second photograph in turn.

[0040] The cost information between each mapping coordinate information and coordinate information of each target in the second photograph is determined according to the overlap rate between each mapping coordinate information and coordinate information of each target in the second photograph.

[0041] Optionally, the determining module is specifically configured to determine an intersection between the first mapping coordinate information and the first coordinate information in the second captured picture, to obtain an intersection calculation result.

[0042] determine a union between the first mapping coordinate information and the first coordinate information in the second captured picture, to obtain a union calculation result.

[0043] divide the intersection calculation result by the union calculation result, to obtain an overlap rate between the first mapping coordinate information and the first coordinate information in the second captured picture, so as to sequentially obtain the overlap rate between each mapping coordinate information and the coordinate information of each target in the second captured picture.

[0044] Optionally, the matching module is specifically configured to determine a plurality of candidate matching results and a cost sum corresponding to each matching result according to the cost information between each mapping coordinate information and the coordinate information of each target in the second captured picture, the matching result including a matching relationship between each mapping coordinate information and one coordinate information in the second captured picture, and the cost sum being a sum of the cost information between each mapping coordinate information and the matched coordinate information in the second captured picture under the matching result.

[0045] determine a target matching result according to the cost sum corresponding to each candidate matching result.

[0046] determine a matching relationship between each target in the first captured picture and each target in the second captured picture according to the target matching result.

[0047] Optionally, the matching module is specifically configured to determine a second target matched by each mapping coordinate information from the second captured picture according to the target coordinate information of the second captured picture corresponding to each mapping coordinate information under the target matching result.

[0048] determine a first target matched by each mapping coordinate information from the first captured picture according to the mapping relationship between each mapping coordinate information and the coordinate information of each target in the first captured picture.

[0049] determine that the first target in the first captured picture and the second target in the second captured picture match each other.

[0050] In a third aspect, an electronic device is provided, including a processor, a storage medium, and a bus. The storage medium stores machine readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium through the bus. The processor executes the machine readable instructions to perform the steps of the multi-picture target matching method provided in the first aspect.

[0051] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program. When the computer program is run by a processor, the steps of the multi-picture target matching method according to the first aspect are performed.

[0052] The present application has the following beneficial effects:

[0053] The present application provides a multi-picture target matching method, device, electronic equipment and storage medium. The method can convert the coordinate information of each target in a first picture to a coordinate system of a second picture by calculating the coordinate transformation matrix between different pictures, and obtain the mapping coordinate information of each target in the first picture in the second picture. The matching degree between the target corresponding to each mapping coordinate information and the target corresponding to each coordinate information in the second picture can be determined based on the cost information of the mapping coordinate information and the coordinate information of each target in the second picture. Since each mapping coordinate information is obtained by converting the coordinate information of each target in the first picture, the target corresponding to each mapping coordinate information is a target in the first picture. Therefore, the matching result between each target in the first picture and each target in the second picture can be determined based on the matching degree between the target corresponding to each mapping coordinate information and the target corresponding to each coordinate information in the second picture. The target matching method provided by the present application can automatically realize the matching between targets in different pictures, effectively solve the matching problem between targets in multiple pictures, and has high matching efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be considered as limiting the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0055] Figure 1 Flowchart of the multi-picture target matching method provided by the embodiments of the present application Figure One ;

[0056] Figure 2 Flowchart of the multi-picture target matching method provided by the embodiments of the present application Figure Two ;

[0057] Figure 3 Coordinate mapping relationship diagram provided by the embodiments of the present application

[0058] Figure 4 Flowchart of the multi-picture target matching method provided by the embodiments of the present application Figure Three ;

[0059] Figure 5 Flowchart of a multi-picture target matching method provided by an embodiment of the present application Figure Four ;

[0060] Figure 6 Flowchart of a multi-picture target matching method provided by an embodiment of the present application Figure Five ;

[0061] Figure 7 Flowchart of a multi-picture target matching method provided by an embodiment of the present application Figure Six ;

[0062] Figure 8 Schematic diagram of a multi-picture target matching device provided by an embodiment of the present application

[0063] Figure 9 Schematic diagram of a structure of an electronic device provided by an embodiment of the present application DETAILED DESCRIPTION

[0064] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the following will be combined with the accompanying drawings for the embodiments of the present application to make a clear and complete description of the technical solutions in the embodiments of the present application. It should be understood that the accompanying drawings in the present application are only for the purpose of illustration and description, and are not used to limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn according to the actual proportions. The flowchart shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can not be implemented in sequence, and the steps without logical context relationship can be reversed in sequence or implemented simultaneously. In addition, one or more other operations can be added to the flowchart or removed from the flowchart by those skilled in the art under the guidance of the content of the present application.

[0065] In addition, the described embodiments are only some of the embodiments of the present application, not all the embodiments. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0066] It should be noted that the term “comprising” will be used in the embodiments of the present application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0067] The method can be applied to a traffic video scene, for example, a "holographic intersection", by fusing target information of pictures taken by multiple groups of cameras which are parallel but have different view angles. In the case that the same target appears in different pictures but the coordinate information in different pictures is inconsistent, the method can be used to fuse the information of the same target in multiple video pictures in real time, so as to generate a panoramic picture of the target under the holographic intersection in combination with a high-definition map, for accurate tracking of the target.

[0068] Figure 1 Flowchart of a multi-picture target matching method provided by an embodiment of the application Figure One The execution subject of the method can be a server, a processor or other computer device. Figure 1 As shown in the figure, the method can include the following steps.

[0069] S101. Calculate the coordinate transformation matrix between each pair of pictures according to a plurality of key points marked in advance in each picture taken at different angles.

[0070] Optionally, one picture can be taken by one shooting device, and each shooting device can be installed in parallel on the same plane, and the azimuth angle of the picture corresponding to each shooting device is different, so that each picture has a different shooting coordinate system.

[0071] For each picture, a point that exists in common and is not obviously occluded in each picture can be determined as a key point, and the position of the key point in each picture is marked, wherein the key point can also be referred to as a corner point, which can be defined as the intersection of two edges. For example, two pictures are included, and the head of the same person appears in both pictures, so the center point of the head of the person can be marked as a key point in both pictures, so that the same key points marked in the two pictures form a pair of key points.

[0072] Wherein, the key point can be marked multiple times, and at least three key points can be selected to be marked. The more key points that are marked, the higher the accuracy of the calculated coordinate transformation matrix between the pictures.

[0073] Here, a coordinate transformation matrix is calculated between any two pictures, and the coordinate transformation matrix is used to convert the shooting objects in the two pictures to one coordinate system. For example, the same person is included in picture 1 and picture 2, and through the coordinate transformation matrix, the coordinates of the person in picture 1 can be calculated to map to picture 2.

[0074] S102, determining a mapping coordinate sequence of the first shot picture in a coordinate system of the second shot picture according to the coordinate sequence of the first shot picture and the coordinate transformation matrix between the first shot picture and the second shot picture, the coordinate sequence of the first shot picture comprising coordinate information of each target in the first shot picture, and the mapping coordinate sequence comprising mapping coordinate information of each target in the first shot picture in the coordinate system of the second shot picture.

[0075] The first shot picture and the second shot picture herein can refer to any two different shot pictures, and in the embodiment, the target matching between the first shot picture and the second shot picture is taken as an example to describe the target matching principle and steps of the method. In actual application, the matching manner of targets in any two shot pictures can be performed according to the method.

[0076] Generally, a shot picture comprises a plurality of targets, and since the targets concerned herein are persons and vehicles, in the method, the persons or vehicles in the shot picture can be taken as targets, and other buildings, street lamps, etc. can be ignored.

[0077] Accordingly, the coordinate sequence of the first shot picture comprising coordinate information of each target in the first shot picture can be obtained.

[0078] Alternatively, according to the coordinate transformation matrix between the first shot picture and the second shot picture calculated above, the coordinate sequence of the first shot picture can be mapped to the coordinate system of the second shot picture to obtain a mapping coordinate sequence.

[0079] Since the persons or vehicles in the shot picture correspond to a region rather than a point, in the embodiment, the coordinate information of the target can be represented by the coordinates of a target frame surrounding the target. In one implementation, the coordinate information of the target can be represented as (x, y, w, h), wherein w represents the width of the target frame, h represents the height of the target frame, and (x, y) represents the top-left corner point of the target frame, that is, by a coordinate point and three parameters of height and width, the position of a unique target frame in the picture can be determined. Of course, (x, y) is not limited to representing the top-left corner point of the target frame, and can also represent the bottom-middle point, the bottom-left corner point, etc. of the target frame.

[0080] Of course, the coordinate information of the target can also be uniquely represented by the coordinate information of the four vertices of the target frame.

[0081] S103, determining cost information between each mapping coordinate information and coordinate information of each target in the second shot picture according to the mapping coordinate sequence and the coordinate sequence of the second shot picture, the coordinate sequence of the second shot picture comprising coordinate information of each target in the second shot picture.

[0082] Similarly, the coordinate sequence of the second shot is also composed of coordinate information of each target in the second shot. It is worth noting that each target contained in the first shot and each target contained in the second shot can be the same, different, or partially the same.

[0083] For each mapping coordinate information in the mapping coordinate sequence, the cost information between the mapping coordinate information and each coordinate information in the coordinate sequence of the second shot can be calculated respectively. The cost information can be used to judge the matching degree between two coordinate information to a certain extent. The smaller the cost is, the higher the matching degree is.

[0084] S104, according to the cost information between each mapping coordinate information and the coordinate information of each target in the second shot, determine the matching relationship between each target in the first shot and each target in the second shot.

[0085] Alternatively, based on the calculated cost information between each mapping coordinate information and each coordinate information in the second shot, the matching degree between each mapping coordinate information and each coordinate information in the second shot can be determined. Since the coordinate information can uniquely represent a target, the matching degree between the target corresponding to each mapping coordinate information and each target in the second shot corresponding to each coordinate information can be determined.

[0086] Since each mapping coordinate information is converted from the coordinate information of each target in the first shot, each mapping coordinate information has a corresponding relationship with the coordinate information of each target in the first shot, then the matching degree between each target in the first shot and each target in the second shot can be further determined, so as to obtain the matching relationship between each target in the first shot and each target in the second shot.

[0087] In some embodiments, based on the matching result, if two targets in two shots match, it can be considered that the two targets are the same target. Then, the latitude and longitude information of the target can be obtained according to the coordinate information of the target in any one of the two shots, so as to generate the picture of the target in the holographic intersection.

[0088] In summary, the multi-picture target matching method provided in the embodiment can convert the coordinate information of each target in the first picture to the coordinate system of the second picture by calculating the coordinate transformation matrix between different pictures, and obtain the mapping coordinate information of each target in the first picture in the second picture. The matching degree between the target corresponding to each mapping coordinate information and the target corresponding to each coordinate information in the second picture can be determined based on the cost information of the mapping coordinate information and the coordinate information of each target in the second picture. Since each mapping coordinate information is converted from the coordinate information of each target in the first picture, the target corresponding to each mapping coordinate information is a target in the first picture. Therefore, the matching result between each target in the first picture and each target in the second picture can be determined based on the matching degree between the target corresponding to each mapping coordinate information and the target corresponding to each coordinate information in the second picture. The target matching method can automatically realize the matching between targets in different pictures, effectively solve the matching problem between targets in multiple pictures, and has high matching efficiency.

[0089] Optionally, the calculation of the coordinate transformation matrix between each pair of pictures based on the plurality of key points marked in each picture can include: determining the coordinate transformation matrix between the first picture and the second picture based on the coordinate information of each key point in the first picture and the coordinate information of each key point in the second picture by using an affine transformation algorithm.

[0090] For each key point, the coordinate of each key point in the first picture and the coordinate of each key point in the second picture can be taken as a data pair, so as to obtain the data pair corresponding to each key point. The data pair corresponding to each key point is input into the inverse affine transformation algorithm, and the coordinate transformation matrix between the first picture and the second picture can be generated.

[0091] The coordinate transformation matrix can be composed of a rotation vector R and a translation vector T, and can be represented as H=[R,T]. The coordinate transformation matrix between the first picture and the second picture can be represented as H 1,2 =[R,T].

[0092] Figure 2 Flowchart of the multi-picture target matching method provided in the embodiment Figure Two Optionally, in step S102, the mapping coordinate sequence of the first picture in the coordinate system of the second picture can be determined based on the coordinate sequence of the first picture and the coordinate transformation matrix between the first picture and the second picture.

[0093] S201. Based on the coordinate information of each target in the first shooting frame and the coordinate transformation matrix between the first shooting frame and the second shooting frame, determine the mapping coordinate information of each target in the first shooting frame to the coordinate system of the second shooting frame.

[0094] It is worth noting that when calculating the mapped coordinate information, the coordinate information of each target in the first captured image can be taken only from the (x,y) elements of the coordinate information. The corresponding calculated mapped coordinate information is also a two-dimensional coordinate point. Based on this two-dimensional coordinate point and the w and h of the target box corresponding to the coordinate information (x,y), the mapped coordinate information (mapped target box) of each target in the first captured image in the coordinate system of the second captured image can be restored.

[0095] Assume a target P in frame 1 A The coordinate information is (x A y A The coordinate transformation matrix between shooting frame 1 and shooting frame 2 is H. 1,2 = [R, T], then, the target P A coordinate information (x) A y A The mapped coordinates of the image, when mapped to the coordinate system of the captured image 2, are: (x B’ y B’ )=(x A y A )·H 1,2 That is, using (x) A y A Right multiplied by H 1,2 This will allow you to obtain the target P in the captured image 1. A Target P mapped to frame 2 B’ .

[0096] In one feasible approach, assuming target 1 in the first captured image is mapped to the second captured image, the specific mapping steps can be as follows:

[0097] Select the lower left corner 1 and lower right corner 1 of the target frame corresponding to target 1 in the first shooting frame. Through the coordinate mapping matrix between the first shooting frame and the second shooting frame, the lower left corner 1 and lower right corner 1 can be mapped to the lower left corner 2 and lower right corner 2 in the second shooting frame, respectively. Based on the coordinates of the lower left corner 2 and lower right corner 2, the width of the target frame corresponding to target 1 in the first shooting frame mapped to the second shooting frame can be calculated. Since the original aspect ratio of target 1 and the aspect ratio after mapping remain unchanged, the height of target 1 in the second shooting frame can be obtained.

[0098] Then, based on the coordinate information (x) of target 1 in the first captured image... A y A ) mapped to the mapping point (x) in the second captured image B’ y B’ ), assuming coordinate information (x A y A (x) represents the coordinates of the midpoint of the bottom edge of the target box. B’ y B’ The target is mapped to the bottom midpoint of the target frame in the second shooting frame, and the mapped height and width are calculated above. Then the position of the target frame in the second shooting frame mapped from the first shooting frame can be obtained, thus obtaining the mapping coordinate information.

[0099] Figure 3 A schematic diagram of a coordinate mapping relationship provided for an embodiment of this application, such as... Figure 3 As shown, the target P in the captured image 1 A The coordinate information of the coordinate point (x A y A After mapping to the shooting screen 2, then according to P A The coordinate information (w, h) can be used to obtain the target P. A Target T mapped to shooting frame 2 B It can be seen that the target P A The positions are different in shooting frame 1 and shooting frame 2.

[0100] Therefore, the coordinate information of each target in the shooting frame 1 can be calculated using the above method to obtain its mapped coordinate information in the shooting frame 2.

[0101] S202. Based on the mapping coordinate information of each target mapped to the coordinate system of the second shooting frame, determine the mapping coordinate sequence of the first shooting frame mapped to the coordinate system of the second shooting frame, wherein the order of each target in the mapping coordinate sequence corresponds one-to-one with the order of each target in the coordinate sequence of the first shooting frame.

[0102] Based on the obtained mapping coordinate information, a mapping coordinate sequence can be generated from the first captured image to the coordinate system where the second captured image is located.

[0103] Assume the coordinate sequence of the first captured image is as follows: The coordinate sequence mapped from the coordinate sequence of the first shot to the coordinate system of the second shot is as follows: So, mapping coordinate information This is based on coordinate information. What was converted and Corresponding, by analogy, With Corresponding, etc.

[0104] Figure 4 Flowchart of the multi-picture target matching method provided by the embodiment of the present application Figure Three Optionally, in step S103, the cost information between each mapping coordinate information and the coordinate information of each target in the second captured picture is determined according to the mapping coordinate sequence and the coordinate sequence of the second captured picture, which can include:

[0105] S401, the overlap rate between each mapping coordinate information and the coordinate information of each target in the second captured picture is calculated in turn.

[0106] Suppose the coordinate sequence of the second captured picture is And the mapping coordinate sequence obtained above is Then, for each mapping coordinate information in the mapping coordinate sequence L B′ , an overlap rate can be calculated respectively with each coordinate information in the coordinate sequence L B .

[0107] For example: for , an overlap rate can be calculated respectively with each , and for , an overlap rate can be calculated respectively with each , since L B contains N coordinate information and L B′ contains M mapping coordinate information, M*N overlap rates can be calculated.

[0108] S402, the cost information between each mapping coordinate information and the coordinate information of each target in the second captured picture is determined according to the overlap rate between each mapping coordinate information and the coordinate information of each target in the second captured picture.

[0109] In some embodiments, the cost information between any one mapping coordinate information and the coordinate information of any one target in the second captured picture can be calculated according to the overlap rate between the two coordinate information.

[0110] Thus, the cost information between each mapping coordinate information and the coordinate information of each target in the second captured picture can be obtained.

[0111] Figure 5 Flowchart of the multi-picture target matching method provided by the embodiment of the present application Figure Four ; optionally, in step S401, the overlap rate between each mapping coordinate information and the coordinate information of each target in the second captured picture is calculated in turn, which can include:

[0112] S501, determine the intersection between the first mapping coordinate information and the first coordinate information in the second shooting picture, and obtain an intersection calculation result.

[0113] In this embodiment, when calculating the overlap rate between the two coordinate information, an IOU (Intersection over Union) calculation formula is used to obtain the result.

[0114] Since each coordinate information represents a target frame, the intersection between the first mapping coordinate information and the first coordinate information, i.e., the intersection between the two target frames, can be calculated.

[0115] S502, determine the union between the first mapping coordinate information and the first coordinate information in the second shooting picture, and obtain a union calculation result.

[0116] Similarly, the union between the first mapping coordinate information and the first coordinate information, i.e., the union between the two target frames, can also be calculated.

[0117] S503, divide the intersection calculation result by the union calculation result to obtain the overlap rate between the first mapping coordinate information and the first coordinate information in the second shooting picture, thereby sequentially obtaining the overlap rate between each mapping coordinate information and the coordinate information of each target in the second shooting picture.

[0118] Alternatively, the intersection result obtained by the above calculation is divided by the union result, and then the overlap rate between the first mapping coordinate information and the first coordinate information can be calculated. Similarly, the overlap rate between any mapping coordinate information and any coordinate information in the second shooting picture can also be sequentially calculated.

[0119] Suppose the overlap rate between the first mapping coordinate information and the first coordinate information is IOU 1,1 Then, the cost information between the first mapping coordinate information and the first coordinate information is 1-IOU 1,1 .

[0120] It is worth noting that the larger the IOU result between the two regions, i.e., the higher the overlap degree of the two regions, and the value of IOU is between 0 and 1. When IOU is 0, i.e., the two regions are almost completely non-overlapping, when IOU is 1, i.e., the two regions are completely overlapping, and when IOU is a decimal between 0 and 1, the two regions are partially overlapping.

[0121] Based on this, since the cost information between the two coordinate information is 1-IOU, the larger the IOU, i.e., the smaller the cost information, the more similar the two coordinate information.

[0122] Based on the above calculation, a cost matrix between the mapping coordinate information and the coordinate information of each target in the second shot picture can be obtained, and the size of the cost matrix is M by N, where M refers to the number of mapping coordinate information in the mapping coordinate sequence, and N refers to the number of coordinate information in the coordinate sequence of the second shot picture.

[0123] Figure 6 Flowchart of the multi-picture target matching method provided by the embodiment of the application Figure Five Optionally, in step S104, the matching relationship between each target in the first shot picture and each target in the second shot picture is determined according to the cost information between the mapping coordinate information and the coordinate information of each target in the second shot picture, which can include:

[0124] S601, determine a plurality of candidate matching results and a cost sum corresponding to each matching result according to the cost information between the mapping coordinate information and the coordinate information of each target in the second shot picture, the matching result including: the matching relationship between the mapping coordinate information and one coordinate information in the second shot picture, and the cost sum being the sum of the cost information between the mapping coordinate information and the matched coordinate information in the second shot picture under the matching result.

[0125] Optionally, based on the cost matrix obtained above, the Hungarian algorithm can be used for optimal matching according to the cost information in the cost matrix, so as to determine the optimal matching relationship between the mapping coordinate information and the coordinate information in the second shot picture, that is, to determine the uniquely matched coordinate information for each mapping coordinate information from the coordinate information in the second shot picture.

[0126] In the Hungarian algorithm, the matching target is to make the sum of the cost information between the mutually matched coordinate information reach the minimum after the mapping coordinate information is matched to a certain coordinate information in the second shot picture.

[0127] Optionally, since the mapping coordinate information and the coordinate information in the second shot picture both include a plurality of, it is necessary to find the best coordinate information matched by each mapping coordinate information from a global perspective, and it is also necessary to ensure that the sum of the cost information is the minimum.

[0128] Since any one mapping coordinate information can be matched to any one coordinate information in the second shot picture, there will be many possible matching results between the mapping coordinate sequence and the coordinate sequence of the second shot picture, which is similar to the exhaustive way, all possible matching results are obtained, each matching result is taken as a candidate matching result, and each matching result corresponds to a calculated cost sum. The cost sum refers to the sum of the cost information between the mapping coordinate information and the matched coordinate information under the matching result.

[0129] For example, the matching result 1 includes: the mapping coordinate information 1 matches the coordinate information 1 in the second shooting picture, the mapping coordinate information 2 matches the coordinate information 2 in the second shooting picture, and the mapping coordinate information 3 matches the coordinate information 3 in the second shooting picture. The cost information of the mapping coordinate information 1 and the coordinate information 1 is a, the cost information of the mapping coordinate information 2 and the coordinate information 2 is b, and the cost information of the mapping coordinate information 3 and the coordinate information 3 is c. Therefore, the cost sum corresponding to the matching result 1 is a+b+c.

[0130] S602, determining the target matching result according to the cost sum corresponding to each candidate matching result.

[0131] Optionally, the matching result with the minimum cost sum can be selected as the target matching result from the candidate matching results according to the cost sum corresponding to each candidate matching result.

[0132] S603, determining the matching relationship between each target in the first shooting picture and each target in the second shooting picture according to the target matching result.

[0133] Optionally, based on the determined target matching result, the matching relationship between each target in the first shooting picture and each target in the second shooting picture can be determined according to the matching relationship between the mapping coordinate information and the coordinate information in the second shooting picture in the target matching result.

[0134] Figure 7 Flowchart of the multi-picture target matching method provided by the embodiment of the application Figure Six Optionally, in step S603, the matching relationship between each target in the first shooting picture and each target in the second shooting picture can be determined according to the target matching result, which can include:

[0135] S701, determining each second target matched by each mapping coordinate information from the second shooting picture according to the target coordinate information of each mapping coordinate information in the second shooting picture in the target matching result.

[0136] Assuming that the target matching result includes: mapping coordinate information 1 corresponds to coordinate information 2 in the second shooting picture, mapping coordinate information 2 corresponds to coordinate information 4 in the second shooting picture, and mapping coordinate information 3 corresponds to coordinate information 6 in the second shooting picture, and the targets corresponding to each coordinate information in the second shooting picture are unique, assuming that the coordinate information 2 in the second shooting picture corresponds to the target a in the second shooting picture, and similarly, the coordinate information 4 in the second shooting picture corresponds to the target b in the second shooting picture, and the coordinate information 6 in the second shooting picture corresponds to the target c in the second shooting picture, then the second target in the second shooting picture matched by the mapping coordinate information 1 can be determined as the target a, the second target in the second shooting picture matched by the mapping coordinate information 2 can be determined as the target b, and the second target in the second shooting picture matched by the mapping coordinate information 3 can be determined as the target c.

[0137] S702, the mapping relationship between each mapping coordinate information and the coordinate information of each target in the first shooting picture, and the first target matched by each mapping coordinate information is determined from the first shooting picture.

[0138] According to the foregoing, the coordinate sequence of the first shooting picture and the mapping coordinate sequence of the first shooting picture The sorting of each target in the coordinate sequence of the first shooting picture and the mapping coordinate sequence of the first shooting picture is one-to-one, that is, each mapping coordinate information corresponds to each coordinate information in the first shooting picture one-to-one, and then based on the mapping relationship, the unique coordinate information corresponding to each mapping coordinate information in the first shooting picture can be determined, assuming that the mapping coordinate information 1 corresponds to the coordinate information 1 in the first shooting picture, the mapping coordinate information 2 corresponds to the coordinate information 2 in the first shooting picture, and the mapping coordinate information 3 corresponds to the coordinate information 3 in the first shooting picture, and the coordinate information 1 in the first shooting picture corresponds to the target d in the first shooting picture, the coordinate information 2 in the first shooting picture corresponds to the target e in the first shooting picture, and the coordinate information 3 in the first shooting picture corresponds to the target f in the first shooting picture, then the first target in the first shooting picture matched by the mapping coordinate information 1 can be determined as the target d, the first target in the first shooting picture matched by the mapping coordinate information 2 can be determined as the target e, and the first target in the first shooting picture matched by the mapping coordinate information 3 can be determined as the target f.

[0139] S703, determining that the first target in the first shooting picture and the second target in the second shooting picture match each other.

[0140] Then it can be determined that the target d in the first shooting picture and the target a in the second shooting picture match each other, the target e in the first shooting picture and the target b in the second shooting picture match each other, and the target f in the first shooting picture and the target c in the second shooting picture match each other.

[0141] The matching principle described above can be understood as follows: a and b are matched, and b and c are matched, so a and c can be considered to be matched. The reason why the coordinate sequence of the first shot picture is not directly matched with the coordinate sequence of the second shot picture is that the first shot picture and the second shot picture correspond to different coordinate systems and are not comparable, while the mapping coordinate sequence corresponding to the first shot picture and the coordinate sequence of the second shot picture belong to the same coordinate system and can be directly matched.

[0142] In summary, the embodiment of the present application provides a multi-picture target matching method. By calculating the coordinate transformation matrix between different shot pictures, the coordinate information of each target in the first shot picture can be converted to the coordinate system of the second shot picture to obtain the mapping coordinate information of each target in the first shot picture in the second shot picture. Based on the cost information of the mapping coordinate information and the coordinate information of each target in the second shot picture, the matching degree between the target corresponding to each mapping coordinate information and the target corresponding to each coordinate information in the second shot picture can be determined. Since each mapping coordinate information is obtained by converting the coordinate information of each target in the first shot picture, the target corresponding to each mapping coordinate information is a target in the first shot picture. Therefore, based on the matching degree between the target corresponding to each mapping coordinate information and the target corresponding to each coordinate information in the second shot picture, the matching result between each target in the first shot picture and each target in the second shot picture can be determined. The target matching method provided by the present application can automatically realize the matching of targets between different pictures, effectively solve the matching problem between multi-picture targets, and has high matching efficiency.

[0143] The following describes the device, equipment, storage medium, etc. for performing the multi-picture target matching method provided by the present application. The specific implementation process and technical effects are described above, and will not be described again.

[0144] Figure 8 A schematic diagram of a multi-picture target matching device provided by the embodiment of the present application is shown in the figure. The functions implemented by the multi-picture target matching device correspond to the steps of the method described above. The device can be understood as the server described above, or the processor of the server. It can also be understood as a component independent of the server or the processor, which realizes the functions of the present application under the control of the server, such as Figure 8 As shown in the figure, the device can include a calculation module 810, a determination module 820, and a matching module 830.

[0145] The calculation module 810 is configured to calculate the coordinate transformation matrix between each pair of shot pictures according to a plurality of key points marked in advance in each shot picture with different shooting angles.

[0146] determine, according to the coordinate sequence of the first photographed picture and the coordinate transformation matrix between the first photographed picture and the second photographed picture, a mapping coordinate sequence of the first photographed picture in a coordinate system of the second photographed picture, the coordinate sequence of the first photographed picture comprising coordinate information of each target in the first photographed picture, and the mapping coordinate sequence comprising mapping coordinate information of each target in the first photographed picture in the coordinate system of the second photographed picture;

[0147] determine, according to the mapping coordinate sequence and the coordinate sequence of the second photographed picture, cost information between each mapping coordinate information and coordinate information of each target in the second photographed picture, the coordinate sequence of the second photographed picture comprising coordinate information of each target in the second photographed picture;

[0148] determine, according to the cost information between each mapping coordinate information and coordinate information of each target in the second photographed picture, a matching relationship between each target in the first photographed picture and each target in the second photographed picture.

[0149] Optionally, the calculation module 810 is specifically configured to determine, according to the coordinate information of each key point in the first photographed picture and the coordinate information of each key point in the second photographed picture, a coordinate transformation matrix between the first photographed picture and the second photographed picture by using an affine transformation algorithm.

[0150] Optionally, the determination module 820 is specifically configured to determine, according to the coordinate information of each target in the first photographed picture and the coordinate transformation matrix between the first photographed picture and the second photographed picture, mapping coordinate information of each target in the first photographed picture in the coordinate system of the second photographed picture;

[0151] determine, according to the mapping coordinate information of each target in the first photographed picture in the coordinate system of the second photographed picture, a mapping coordinate sequence of the first photographed picture in the coordinate system of the second photographed picture, wherein the order of each target in the mapping coordinate sequence corresponds to the order of each target in the coordinate sequence of the first photographed picture.

[0152] Optionally, the determination module 820 is specifically configured to calculate, in sequence and two by two, an overlap rate between each mapping coordinate information and coordinate information of each target in the second photographed picture.

[0153] determine, according to the overlap rate between each mapping coordinate information and coordinate information of each target in the second photographed picture, the cost information between each mapping coordinate information and coordinate information of each target in the second photographed picture.

[0154] Optionally, the determination module 820 is specifically configured to determine an intersection between the first mapping coordinate information and the first coordinate information in the second photographed picture, to obtain an intersection calculation result.

[0155] determine a union of the first mapping coordinate information and the first coordinate information in the second captured picture, to obtain a union calculation result;

[0156] divide the intersection calculation result by the union calculation result, to obtain an overlap rate between the first mapping coordinate information and the first coordinate information in the second captured picture, so as to sequentially obtain the overlap rate between each mapping coordinate information and the coordinate information of each target in the second captured picture.

[0157] Optionally, the matching module 830 is specifically configured to determine a plurality of candidate matching results and a cost sum corresponding to each matching result according to the cost information between each mapping coordinate information and the coordinate information of each target in the second captured picture, the matching result including a matching relationship between each mapping coordinate information and one coordinate information in the second captured picture, and the cost sum being a sum of the cost information between each mapping coordinate information and the matched coordinate information in the second captured picture under the matching result.

[0158] determine a target matching result according to the cost sum corresponding to each candidate matching result.

[0159] determine a matching relationship between each target in the first captured picture and each target in the second captured picture according to the target matching result.

[0160] Optionally, the matching module 830 is specifically configured to determine a second target matched by each mapping coordinate information from the second captured picture according to the target coordinate information in the second captured picture corresponding to each mapping coordinate information under the target matching result.

[0161] determine a first target matched by each mapping coordinate information from the first captured picture according to a mapping relationship between each mapping coordinate information and the coordinate information of each target in the first captured picture.

[0162] determine that the first target in the first captured picture and the second target in the second captured picture match each other.

[0163] The apparatus is used for executing the method provided by the foregoing embodiments, and has similar implementation principles and technical effects, which will not be described herein again.

[0164] The above modules can be one or more integrated circuits configured to implement the above methods, for example, one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs), etc. For another example, when a certain module above is implemented in the form of a processing element scheduling code, the processing element can be a general purpose processor, such as a central processing unit (CPU) or other processor that can invoke code. For another example, the modules can be integrated together to implement in the form of a system on a chip (SOC).

[0165] The above modules can be connected or communicated with each other via wired connection or wireless connection. The wired connection can include metal cable, optical cable, hybrid cable, etc., or any combination thereof. The wireless connection can include connection in the form of LAN, WAN, Bluetooth, ZigBee, or NFC, etc., or any combination thereof. Two or more modules can be combined into a single module, and any one module can be divided into two or more units. It can be clearly understood by those skilled in the art that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, which will not be repeated herein.

[0166] Figure 9 A structural schematic diagram of an electronic device is provided for the embodiments of the present application, which can be a computing device with data processing function.

[0167] The device includes a processor 801 and a storage medium 802.

[0168] The storage medium 802 is configured to store a program, and the processor 801 invokes the program stored in the storage medium 802 to execute the above method embodiments. The specific implementation and technical effects are similar, and will not be repeated here.

[0169] The storage medium 802 stores program code, and when the program code is executed by the processor 801, the processor 801 executes various steps in the methods according to various exemplary embodiments of the present application described in the above “Exemplary Method” section of the specification.

[0170] The processor 801 can be a general processor, such as a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present application can be directly embodied as completed by a hardware processor, or completed by a combination of hardware and software modules in the processor.

[0171] The storage medium 802 is a non-volatile computer readable storage medium, and can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The storage medium can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card type storage medium, random access memory (RAM), static random access memory (SRAM), programmable read only memory (PROM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), magnetic storage medium, magnetic disk, optical disk, etc. The storage medium is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but is not limited to this. The storage medium 802 in the embodiments of the present application can also be a circuit or any other device capable of realizing a storage function, used to store program instructions and / or data.

[0172] Optionally, the present application also provides a program product, such as a computer readable storage medium, comprising a program for executing the above-mentioned method embodiments when executed by a processor.

[0173] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0174] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0175] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional unit.

[0176] The integrated unit realized in the form of software functional unit can be stored in a computer readable storage medium. The software functional unit stored in a storage medium includes a plurality of instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) execute part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk, and various program code storage media.

Claims

1. A multi-picture target matching method, characterized by, The method comprises the following steps: According to the plurality of key points marked in advance in each shooting picture with different shooting angles, the coordinate transformation matrix between each pair of shooting pictures is calculated; According to the coordinate sequence of the first shooting picture and the coordinate transformation matrix between the first shooting picture and the second shooting picture, the mapping coordinate sequence of the first shooting picture in the coordinate system of the second shooting picture is determined, the coordinate sequence of the first shooting picture comprises the coordinate information of each target in the first shooting picture, and the mapping coordinate sequence comprises the mapping coordinate information of each target in the first shooting picture in the coordinate system of the second shooting picture; According to the mapping coordinate sequence and the coordinate sequence of the second shooting picture, the cost information between each mapping coordinate information and the coordinate information of each target in the second shooting picture is determined, and the coordinate sequence of the second shooting picture comprises the coordinate information of each target in the second shooting picture; According to the cost information between each mapping coordinate information and the coordinate information of each target in the second shooting picture, the matching relationship between each target in the first shooting picture and each target in the second shooting picture is determined. According to the mapping coordinate sequence and the coordinate sequence of the second shooting picture, the cost information between each mapping coordinate information and the coordinate information of each target in the second shooting picture is determined, and the coordinate sequence of the second shooting picture comprises the coordinate information of each target in the second shooting picture; The overlapping rate between each mapping coordinate information and the coordinate information of each target in the second shooting picture is calculated in turn. According to the overlapping rate between each mapping coordinate information and the coordinate information of each target in the second shooting picture, the cost information between each mapping coordinate information and the coordinate information of each target in the second shooting picture is determined.

2. The method of claim 1, wherein, According to the plurality of key points marked in advance in each shooting picture with different shooting angles, the coordinate transformation matrix between each pair of shooting pictures is calculated, comprising: According to the coordinate information of each key point in the first shooting picture and the coordinate information of each key point in the second shooting picture, the affine transformation algorithm is used to determine the coordinate transformation matrix between the first shooting picture and the second shooting picture.

3. The method of claim 1, wherein, According to the coordinate sequence of the first shooting picture and the coordinate transformation matrix between the first shooting picture and the second shooting picture, the mapping coordinate sequence of the first shooting picture in the coordinate system of the second shooting picture is determined, the coordinate sequence of the first shooting picture comprises the coordinate information of each target in the first shooting picture, and the mapping coordinate sequence comprises the mapping coordinate information of each target in the first shooting picture in the coordinate system of the second shooting picture; According to the coordinate information of each target in the first shooting picture and the coordinate transformation matrix between the first shooting picture and the second shooting picture, the mapping coordinate information of each target in the first shooting picture in the coordinate system of the second shooting picture is determined respectively; According to the mapping coordinate information of each target in the first shooting picture in the coordinate system of the second shooting picture, the mapping coordinate sequence of the first shooting picture in the coordinate system of the second shooting picture is determined, and the order of each target in the mapping coordinate sequence corresponds to the order of each target in the coordinate sequence of the first shooting picture.

4. The method of claim 1, wherein, The overlapping rate between each mapping coordinate information and the coordinate information of each target in the second shooting picture is calculated in turn, comprising: determine an intersection between the first mapping coordinate information and the first coordinate information in the second shot, to obtain an intersection calculation result; determine a union between the first mapping coordinate information and the first coordinate information in the second shot, to obtain a union calculation result; divide the intersection calculation result by the union calculation result, to obtain an overlap rate between the first mapping coordinate information and the first coordinate information in the second shot, so as to sequentially obtain the overlap rate between each mapping coordinate information and the coordinate information of each target in the second shot.

5. The method of claim 1, wherein, The determining, according to the cost information between each mapping coordinate information and the coordinate information of each target in the second shot, of the matching relationship between each target in the first shot and each target in the second shot comprises: determining a plurality of candidate matching results and a cost sum corresponding to each matching result according to the cost information between each mapping coordinate information and the coordinate information of each target in the second shot, the matching result comprising a matching relationship between each mapping coordinate information and one coordinate information in the second shot, and the cost sum being a sum of the cost information between each mapping coordinate information and the matched coordinate information in the second shot under the matching result; determining a target matching result according to the cost sum corresponding to each candidate matching result; determining the matching relationship between each target in the first shot and each target in the second shot according to the target matching result.

6. The method of claim 5, wherein, The determining, according to the target matching result, of the matching relationship between each target in the first shot and each target in the second shot comprises: determining, from the second shot, a second target matched by each mapping coordinate information according to the target coordinate information in the second shot corresponding to each mapping coordinate information under the target matching result; determining, from the first shot, a first target matched by each mapping coordinate information according to the mapping relationship between each mapping coordinate information and the coordinate information of each target in the first shot; determining that the first target in the first shot and the second target in the second shot match each other.

7. A multi-picture target matching apparatus, characterized by comprising: comprise: a calculation module, a determination module, and a matching module; The calculation module is configured to calculate a coordinate transformation matrix between each pair of shots according to a plurality of key points marked in advance in each shot having different shooting angles. The determination module is configured to determine a mapping coordinate sequence of the first shot in a coordinate system of the second shot according to a coordinate sequence of the first shot and the coordinate transformation matrix between the first shot and the second shot, the coordinate sequence of the first shot comprising coordinate information of each target in the first shot, and the mapping coordinate sequence comprising mapping coordinate information of each target in the first shot in the coordinate system of the second shot. The determining module is configured to determine cost information between each of the mapping coordinate information and coordinate information of each target in the second captured image according to the mapping coordinate sequence and a coordinate sequence of the second captured image, the coordinate sequence of the second captured image comprising the coordinate information of each target in the second captured image. The matching module is configured to determine a matching relationship between each target in the first captured image and each target in the second captured image according to the cost information between each of the mapping coordinate information and the coordinate information of each target in the second captured image. The determining module is specifically configured to calculate, in sequence, an overlap rate between each of the mapping coordinate information and the coordinate information of each target in the second captured image. The cost information between each of the mapping coordinate information and the coordinate information of each target in the second captured image is determined according to the overlap rate between each of the mapping coordinate information and the coordinate information of each target in the second captured image.

8. An electronic device, comprising: The method comprises the following steps: A processor, a storage medium and a bus, the storage medium storing program instructions executable by the processor, the processor and the storage medium communicating through the bus when the electronic device is running, the processor executing the program instructions to perform the steps of the multi-image target matching method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by the processor to perform the steps of the multi-image target matching method according to any one of claims 1 to 6.

Images