Video-based vehicle matching method and device, electronic equipment and storage medium

Abstract

The disclosure provides a video-based vehicle matching method and device, electronic equipment and storage medium, artificial intelligence technology field, specifically related to computer vision and intelligent transportation technology field, and the scheme is: obtaining a first target image with a first vehicle to be matched and a second target image with a second vehicle to be matched from a video frame image; identifying the first target image and the second target image to obtain first key point information and first attribute information corresponding to the first vehicle to be matched, and second key point information and second attribute information corresponding to the second vehicle to be matched; in the case that the first attribute information matches the second attribute information, determining a matching result of the first vehicle to be matched and the second vehicle to be matched according to the first key point information and the second key point information. Therefore, the matching efficiency and credibility of vehicle matching can be improved, the inconsistency caused by angle transformation, shielding and the like is reduced, and the effectiveness of the vehicle matching result is improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to the technical field of computer vision and intelligent transportation, and more particularly to a vehicle matching method and device based on video, an electronic device, and a storage medium. BACKGROUND

[0002] At present, with the development of the national economy, more and more families are equipped with vehicles, and the traffic volume in cities is increasing, and the parking problem has become a major social problem. The disorderly parking of vehicles on the road will also increase the traffic pressure. In order to alleviate the problem of parking difficulty, many places have adopted the parking mode of orderly planning of road parking spaces, but if these parking spaces are not effectively managed, it will also cause the abuse of social resources, so effective management of road parking spaces is particularly important. Therefore, using a video-based road parking space automatic management system to match vehicles in the video can realize effective management of parking spaces. For example, the license plate of other vehicles can be taken without mistake during the vehicle parking process, the current vehicle in the parking space needs to be consistent with the parked vehicle during the periodic update after parking, and the vehicle also needs to be consistent during the unparking process, so as to avoid mistaken unparking, etc. SUMMARY

[0003] The present disclosure provides a vehicle matching method and device based on video, an electronic device, and a storage medium.

[0004] According to an aspect of the present disclosure, a vehicle matching method based on video is provided, which includes: obtaining a first target image with a first vehicle to be matched and a second target image with a second vehicle to be matched from a video frame image; identifying the first target image and the second target image to obtain first key point information and first attribute information corresponding to the first vehicle to be matched, and second key point information and second attribute information corresponding to the second vehicle to be matched; in the case that the first attribute information and the second attribute information match, determining a matching result of the first vehicle to be matched and the second vehicle to be matched according to the first key point information and the second key point information.

[0005] According to another aspect of the present disclosure, a video-based vehicle matching device is provided, comprising: a first acquisition module configured to acquire a first target image having a first vehicle to be matched and a second target image having a second vehicle to be matched from a video frame image; an identification module configured to identify the first target image and the second target image to obtain first key point information and first attribute information corresponding to the first vehicle to be matched, and second key point information and second attribute information corresponding to the second vehicle to be matched; and a first determination module configured to determine a matching result of the first vehicle to be matched and the second vehicle to be matched according to the first key point information and the second key point information in a case where the first attribute information matches the second attribute information.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to the first aspect of the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the method according to the first aspect of the present disclosure.

[0008] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method according to the first aspect of the present disclosure.

[0009] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings are used to better understand the present scheme, and do not limit the present disclosure. Among them:

[0011] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure;

[0012] Figure 2 is a schematic diagram according to the second embodiment of the present disclosure;

[0013] Figure 3 is a schematic diagram according to the third embodiment of the present disclosure;

[0014] Figure 4is a conversion schematic diagram of an image coordinate system and a world coordinate system according to an embodiment of the present disclosure;

[0015] Figure 5 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0016] Figure 6 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0017] Figure 7 is a schematic diagram according to a sixth embodiment of the present disclosure;

[0018] Figure 8 is a flow schematic diagram of a vehicle matching method based on video according to an embodiment of the present disclosure;

[0019] Figure 9 is a schematic diagram according to a seventh embodiment of the present disclosure;

[0020] Figure 10 shows a schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure. DETAILED DESCRIPTION

[0021] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to assist in understanding, which should be considered merely exemplary. Thus, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0022] Currently, in a video-based automatic management system for road parking spaces, the way for vehicle matching is generally a traditional person re-identification (ReID) algorithm, for example, by collecting and labeling the same target and different targets to form a sample set, using the sample set to train a deep learning model, and in use, using the deep learning model to compare the similarity of two vehicles to output the vehicle similarity.

[0023] However, the traditional ReID algorithm described above has three disadvantages in actual use:

[0024] (1) The vehicle similarity accuracy output by the ReID algorithm is low and does not have good explainability, making it difficult to adjust and increase;

[0025] (2) Different angles of the same vehicle will show great differences in appearance, and it is not reasonable to put them in the same class of samples for training, which may cause difficulty in fitting the training;

[0026] (3) When the same vehicle is partially occluded, the traditional ReID algorithm is also affected, resulting in low vehicle similarity accuracy.

[0027] To solve the above problems, the present disclosure provides a vehicle matching method and device based on video, an electronic device and a storage medium.

[0028] The vehicle matching method and device based on video, the electronic device and the storage medium of the embodiments of the present disclosure are described below with reference to the drawings.

[0029] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure. It should be noted that the embodiments of the present disclosure take the vehicle matching method based on video configured in the vehicle matching device based on video as an example, and the vehicle matching device based on video can be applied to any electronic device, so that the electronic device can perform the vehicle matching function based on video.

[0030] Among them, the electronic device can be any device with computing power, for example, it can be a personal computer (PC), a mobile terminal, etc., and the mobile terminal can be a mobile phone, a tablet computer, a personal digital assistant, a wearable device, etc. Hardware devices with various operating systems, touch screens and / or display screens.

[0031] As shown in Figure 1 The vehicle matching method based on video can include the following steps:

[0032] Step 101, obtaining a first target image with a first vehicle to be matched and a second target image with a second vehicle to be matched from a video frame image.

[0033] It should be noted that the vehicle matching method based on video of the embodiments of the present disclosure can be applied to different application scenarios, for example, it can be applied to vehicle matching in a parking lot, vehicle matching at a highway intersection, and vehicle matching management in other places.

[0034] In the embodiments of the present disclosure, the first target image or the second target image can be obtained from the video frame corresponding to the video collected by the camera in the corresponding scene, wherein the first target image has a first vehicle to be matched, and the second target image has a second vehicle to be matched.

[0035] Step 102, identifying the first target image and the second target image to obtain first key point information and first attribute information corresponding to the first vehicle to be matched, and second key point information and second attribute information corresponding to the second vehicle to be matched.

[0036] Further, feature recognition can be performed on the first target image and the second target image to obtain first key point information and first attribute information corresponding to the first to-be-matched vehicle in the first target image, and second key point information and second attribute information corresponding to the second to-be-matched vehicle. The attribute information can include but is not limited to vehicle color and vehicle model, and the key point information can include but is not limited to the middle of the vehicle wheel, the contact point of the vehicle wheel with the ground, the front lamp of the vehicle, and the rear lamp of the vehicle.

[0037] In step 103, in a case where the first attribute information matches the second attribute information, a matching result of the first to-be-matched vehicle and the second to-be-matched vehicle is determined according to the first key point information and the second key point information.

[0038] In the embodiment of the present disclosure, whether the first attribute information matches the second attribute information can be determined according to a matching result of the first attribute information and the second attribute information, and in a case where the first attribute information matches the second attribute information, the first key point information and the second key point information can be further matched to determine the matching result of the first to-be-matched vehicle and the second to-be-matched vehicle.

[0039] In summary, by further matching the vehicle key point information in a case where the first attribute information matches the second attribute information, the matching efficiency and the credibility of vehicle matching can be improved, the inconsistency caused by angle transformation, occlusion, and the like can be reduced, and the effectiveness of the vehicle matching result is improved.

[0040] In order to accurately determine the matching result of the first to-be-matched vehicle and the second to-be-matched vehicle, as shown in Figure 2 , the first attribute information and the second attribute information can be matched to determine the matching result of the first to-be-matched vehicle and the second to-be-matched vehicle, Figure 2 as shown in the second embodiment of the present disclosure. In the embodiment of the present disclosure, a key point pair of mutual matching of the first key point and the second key point can be determined, and according to the number of key point pairs and a set matching number threshold, the matching result of the first to-be-matched vehicle and the second to-be-matched vehicle can be determined, Figure 2 The embodiment shown can include the following steps:

[0041] In step 201, a first target image having a first to-be-matched vehicle and a second target image having a second to-be-matched vehicle are obtained from a video frame image.

[0042] In step 202, the first target image and the second target image are recognized to obtain first key point information and first attribute information corresponding to the first to-be-matched vehicle, and second key point information and second attribute information corresponding to the second to-be-matched vehicle.

[0043] In step 203, the first attribute information and the second attribute information are compared.

[0044] In the embodiments of the present disclosure, various attributes in the first attribute information are matched with corresponding attributes in the second attribute information respectively to determine the matching result of the first attribute information and the second attribute information.

[0045] In step 204, in the case that the first attribute information matches the second attribute information, a target key point matched with each first key point in the first key point information is determined from at least one second key point included in the second key point information to obtain a key point pair matched with each other.

[0046] In the embodiments of the present disclosure, according to the first coordinate information of the first key point in the first target image and the second coordinate information of the second key point in the second target image, a target key point matched with each first key point in the first key point information is determined to obtain a key point pair matched with each other.

[0047] As an example, the first coordinate information of any first key point in the first target image is obtained, and the second coordinate information of each second key point in the second key point information in the second target image is obtained; according to the size information of the first target image or the second target image, a position difference threshold is determined; according to the difference between the second coordinate information of each second key point and the first coordinate information, a target key point with a difference less than the position difference threshold is determined from each second key point; and the first key point and the target key point are taken as a key point pair.

[0048] That is, in order to accurately obtain a key point pair matched with each other, the first coordinate information of any first key point in the first target image is obtained, and the second coordinate information of each second key point in the second key point information in the second target image is obtained, then according to the first coordinate information and each second coordinate information, the difference between the first coordinate information and each second coordinate information is determined, for example, the chessboard distance is used to determine the difference between the first coordinate information and each second coordinate information, and the difference between the first coordinate information and each second coordinate information can be expressed as the following formula: D is = |X1-X2|+|Y1-Y2|, wherein X1, Y1 is the first coordinate information, and X2, Y2 is the second coordinate information.

[0049] Further, the difference between the second coordinate information of each second key point and the first coordinate information is compared with a position difference threshold to determine a target key point with a difference less than the position difference threshold from each second key point, and the first key point and the target key point are taken as a key point pair. The position difference threshold can be determined according to the size information of the first target image or the second target image, for example, the position difference threshold can be two thousandths of the length I w of the first target image or the second target image plus two thousandths of the width I h , that is, th = 2*I w / 1000+2*I h / 1000.

[0050] In step 205, when the number of key point pairs is greater than the set matching number threshold, a set matching precision is obtained.

[0051] Further, the number of key point pairs is compared with the set matching number threshold, and when the number of key point pairs is greater than the set matching number threshold, the set matching precision can be obtained. For example, the set matching number threshold is N, and when the number of key point pairs is greater than N, the set matching precision can be obtained.

[0052] In step 206, when the matching precision is less than the set precision threshold, it is determined that the first vehicle to be matched and the second vehicle to be matched are matched.

[0053] Further, the matching precision is compared with the set precision threshold, and when the matching precision is less than the set precision threshold, it is determined that the first vehicle to be matched and the second vehicle to be matched are matched. For example, in different application scenarios, the requirements for matching precision are different. For example, in some application scenarios, the requirement for matching precision is high, and the value of the set matching precision can be high. For example, in some other application scenarios, the requirement for matching precision is low, and the value of the set matching precision can be low. Therefore, when the requirement for matching precision is low and the value of the set matching precision is low, the matching precision is less than the set precision threshold, and it is determined that the first vehicle to be matched and the second vehicle to be matched are matched.

[0054] It should be noted that the execution processes of steps 201 to 202 can be realized by any one of the embodiments of the present disclosure, and the present disclosure does not limit this, and will not be repeated here.

[0055] In summary, by comparing the first attribute information and the second attribute information, in the case that the first attribute information and the second attribute information are matched, from the at least one second key point included in the second key point information, the target key point matched by each first key point included in the first key point information is determined to obtain the key point pairs matched with each other; when the number of key point pairs is greater than the set matching number threshold, the set matching precision is obtained; when the matching precision is less than the set precision threshold, it is determined that the first vehicle to be matched and the second vehicle to be matched are matched. Therefore, in the case that the number of key point pairs is greater than the set matching number threshold, according to the set matching precision, the matching result of the first vehicle to be matched and the second vehicle to be matched can be determined.

[0056] In order to determine the matching result of the first vehicle to be matched and the second vehicle to be matched when the number of key point pairs is less than or equal to the set matching number threshold, or when the matching precision is greater than or equal to the set precision threshold, as shown in Figure 3 Figure 3 ​is a schematic diagram according to a third embodiment of the present disclosure, in embodiments of the present disclosure, when the number of key point pairs is less than or equal to a set matching number threshold, or when the matching accuracy is greater than or equal to a set accuracy threshold, the matching result of the first vehicle to be matched and the second vehicle to be matched can be determined according to the size information and the pose information of the first vehicle to be matched and the size information and the pose information of the second vehicle to be matched, Figure 3 The embodiments shown can include the following steps:

[0057] Step 301, obtaining a first target image with a first vehicle to be matched and a second target image with a second vehicle to be matched from a video frame image.

[0058] Step 302, identifying the first target image and the second target image to obtain first key point information and first attribute information corresponding to the first vehicle to be matched, and second key point information and second attribute information corresponding to the second vehicle to be matched.

[0059] Step 303, comparing the first attribute information and the second attribute information.

[0060] Step 304, in the case where the first attribute information matches the second attribute information, determining the target key point matched by each first key point in the first key point information from at least one second key point contained in the second key point information, to obtain a mutually matched key point pair.

[0061] Step 305, when the number of key point pairs is greater than a set matching number threshold, obtaining a set matching accuracy.

[0062] Step 306, when the matching accuracy is less than a set accuracy threshold, determining that the first vehicle to be matched matches the second vehicle to be matched.

[0063] Step 307, when the number of key point pairs is less than or equal to a set matching number threshold, or when the matching accuracy is greater than or equal to a set accuracy threshold, according to the mapping relationship between the image coordinate system and the world coordinate system, transforming each point of the region of the first vehicle to be matched in the first target image to the world coordinate system, and transforming each point of the region of the second vehicle to be matched in the second target image to the world coordinate system, to obtain the size information and the pose information of the first vehicle to be matched and the size information and the pose information of the second vehicle to be matched.

[0064] In embodiments of the present disclosure, when the number of key point pairs is less than or equal to a set matching number threshold, or when the matching accuracy is greater than or equal to a set accuracy threshold, the mapping relationship between the image coordinate system and the time coordinate system can be obtained, and then according to the mapping relationship, each point of the region of the first vehicle to be matched in the first target image can be transformed to the world coordinate system, such as Figure 4In the embodiment, the ground is taken as the X-axis and the Y-axis, and the vertical direction to the ground is taken as the Z-axis, and then the X-axis, the Y-axis and the Z-axis are taken as a world coordinate system, and each point (2D) in the region of the first vehicle to be matched in the first target image is transformed to the world coordinate system (3D), and according to the coordinates of each point in the region of the first vehicle to be matched in the world coordinate system, the size information and the pose information of the first vehicle to be matched can be determined, wherein the size information can include but is not limited to the length, the width and the height of the first vehicle to be matched, and the pose information can include the body direction and the angle of the first vehicle to be matched in the world coordinate system.

[0065] It should be noted that in the embodiment, the first vehicle to be matched can be taken as a cuboid, that is, it is considered to have six faces, and according to the different body directions, three faces can be displayed in the first target image, and by transforming each point in the region of the first vehicle to be matched in the first target image to the world coordinate system, the displayed face in the first target image can be extracted according to the coordinates of each point in the region of the first vehicle to be matched in the world coordinate system, such as Figure 4 In the embodiment, the front face, the top face and the side face A are displayed, and if the direction is reversed, the rear face, the top face and the side face B can be obtained in the first target image.

[0066] Similarly, the size information and the pose information of the second vehicle to be matched can be obtained.

[0067] In step 308, the matching result of the first vehicle to be matched and the second vehicle to be matched is determined according to the size information and the pose information of the first vehicle to be matched and the size information and the pose information of the second vehicle to be matched.

[0068] Further, the first side face image of the first vehicle to be matched is extracted from the first target image, the second side face image of the second vehicle to be matched is extracted from the second target image, the first side face image and the second side face image are aligned according to the pose information of the first vehicle to be matched and the pose information of the second vehicle to be matched, the similarity of the first vehicle to be matched and the second vehicle to be matched is determined according to the aligned first side face image and the second side face image, and then the matching result of the first vehicle to be matched and the second vehicle to be matched can be determined according to the similarity of the first vehicle to be matched and the second vehicle to be matched.

[0069] It should be noted that the execution process of steps 301 to 307 can be realized by any one of the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be repeated here.

[0070] In summary, by performing three-dimensional conversion on each point of the region of the first to-be-matched vehicle in the first target image and on each point of the region of the second to-be-matched vehicle in the second target image, the size information and the pose information of the first to-be-matched vehicle and the size information and the pose information of the second to-be-matched vehicle can be accurately determined. Furthermore, according to the size information and the pose information of the first to-be-matched vehicle and the size information and the pose information of the second to-be-matched vehicle, the matching result of the first to-be-matched vehicle and the second to-be-matched vehicle can be accurately determined.

[0071] To more clearly illustrate how to determine the matching result of the first to-be-matched vehicle and the second to-be-matched vehicle according to the size information and the pose information of the first to-be-matched vehicle and the size information and the pose information of the second to-be-matched vehicle, as shown in Figure 5 Figure 5 is a schematic diagram according to the fourth embodiment of the present disclosure. In the embodiment of the present disclosure, the first side image of the first to-be-matched vehicle and the second side image of the second to-be-matched vehicle can be aligned, and the aligned first side image and the second side image can be input into a similarity model to obtain the similarity of the first side image and the second side image. According to the similarity and a set similarity threshold, the matching result of the first to-be-matched vehicle and the second to-be-matched vehicle can be determined, Figure 5 The embodiment shown in

[0072] Step 501, obtaining a first target image having a first to-be-matched vehicle and a second target image having a second to-be-matched vehicle from a video frame image.

[0073] Step 502, identifying the first target image and the second target image to obtain first key point information and first attribute information corresponding to the first to-be-matched vehicle and second key point information and second attribute information corresponding to the second to-be-matched vehicle.

[0074] Step 503, comparing the first attribute information and the second attribute information.

[0075] Step 504, in the case where the first attribute information and the second attribute information match, determining a target key point matched by each first key point included in the first key point information from at least one second key point included in the second key point information to obtain a key point pair that matches each other.

[0076] Step 505, when the number of key point pairs is greater than a set matching number threshold, obtaining a set matching precision.

[0077] Step 506, when the matching precision is less than a set precision threshold, determining that the first to-be-matched vehicle and the second to-be-matched vehicle match.

[0078] ​In step 507, when the number of key point pairs is less than or equal to the set matching number threshold, or when the matching accuracy is greater than or equal to the set accuracy threshold, the points of the region of the first to-be-matched vehicle in the first target image are transformed to the world coordinate system according to the mapping relationship between the image coordinate system and the world coordinate system, and the points of the region of the second to-be-matched vehicle in the second target image are transformed to the world coordinate system, to obtain the size information and the pose information of the first to-be-matched vehicle and the size information and the pose information of the second to-be-matched vehicle.

[0079] In step 508, the first side image of the first to-be-matched vehicle is extracted from the first target image, and the second side image of the second to-be-matched vehicle is extracted from the second target image.

[0080] In the embodiments of the present disclosure, the first side image of the first to-be-matched vehicle can be extracted from the first target image to obtain the first side image of the first to-be-matched vehicle, and the second side image of the second to-be-matched vehicle can be extracted in the same way to obtain the second side image of the second to-be-matched vehicle.

[0081] In step 509, the first side image and the second side image are aligned according to the similarities and differences between the body directions of the first to-be-matched vehicle and the second to-be-matched vehicle, to obtain the aligned first side image and the aligned second side image.

[0082] In the embodiments of the present disclosure, the pose information includes the body direction, when the body direction of the first to-be-matched vehicle is consistent with the body direction of the second to-be-matched vehicle, the first side image and the second side image can be directly aligned in angle and scale to obtain the aligned first side image and the aligned second side image, when the body direction of the first to-be-matched vehicle is inconsistent with the body direction of the second to-be-matched vehicle, the first side image or the second side image can be mirror-inverted to make the body directions of the first side image and the second side image the same, and then the first side image and the second side image are aligned to obtain the aligned first side image and the aligned second side image.

[0083] In step 510, the aligned first side image and the aligned second side image are input into a similarity model to obtain the similarity of the aligned first side image and the aligned second side image.

[0084] Further, the aligned first side image and the aligned second side image are input into the similarity model, and the similarity model can output the similarity of the aligned first side image and the aligned second side image. It should be noted that the similarity model can be a trained neural network model, and the trained similarity model can learn the corresponding relationship between images and image similarity.

[0085] At step 511, the matching result of the first vehicle to be matched and the second vehicle to be matched is determined according to the similarity and the set similarity threshold.

[0086] In the embodiments of the present disclosure, the similarity of the aligned first side view image and the second side view image can be compared with the set similarity threshold. When the similarity is greater than the set similarity threshold, it can be determined that the first vehicle to be matched and the second vehicle to be matched are matched. When the similarity is less than or equal to the set similarity threshold, it can be determined that the first vehicle to be matched and the second vehicle to be matched are not matched.

[0087] It should be noted that before comparing the similarity of the aligned first side view image and the second side view image with the set similarity threshold, the set similarity threshold can be determined first.

[0088] As an example, the first similarity threshold is generated according to the size information of the first vehicle to be matched and the size information of the second vehicle to be matched. The second similarity threshold is generated when the matching accuracy is greater than or equal to the set accuracy threshold. The set similarity threshold is determined according to the first similarity threshold and the second similarity threshold.

[0089] That is, in order to accurately determine the matching result of the first vehicle to be matched and the second vehicle to be matched, the size information of the first vehicle to be matched and the size information of the second vehicle to be matched can be compared. Then, according to the comparison result of the size information of the first vehicle to be matched and the size information of the second vehicle to be matched, the corresponding first similarity threshold is generated. For example, when the size information of the first vehicle to be matched is consistent with the size information of the second vehicle to be matched, the corresponding first similarity threshold can be 0.5. For another example, when the size information of the first vehicle to be matched is inconsistent with the size information of the second vehicle to be matched, the corresponding first similarity threshold can be 0.7. When the matching accuracy is greater than or equal to the set accuracy threshold, the second similarity threshold can be generated. For example, in some scenarios, the set matching accuracy value is high, the matching accuracy is greater than or equal to the set accuracy threshold, and the second similarity threshold can be 0.3. Then, when the number of key point pairs of the first vehicle to be matched and the second vehicle to be matched is greater than the set matching number threshold, the matching accuracy is greater than or equal to the set accuracy threshold, that is, when the first vehicle to be matched and the second vehicle to be matched are highly similar, in order to make the first vehicle to be matched and the second vehicle to be matched consistent in matching, the smaller one of the first similarity threshold and the second similarity threshold can be used as the set similarity threshold (for example, 0.3).

[0090] It should be noted that the execution process of steps 501 to 507 can be realized by any one of the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be repeated.

[0091] In summary, by aligning the first side image of the first to-be-matched vehicle and the second side image of the second to-be-matched vehicle, and inputting the aligned first side image and second side image into the similarity model, the similarity of the first side image and the second side image can be obtained. According to the similarity and the set similarity threshold, the matching result of the first to-be-matched vehicle and the second to-be-matched vehicle can be accurately determined.

[0092] To clearly illustrate how the first attribute information and the second attribute information are compared, as shown in Figure 6 Figure 6 is a schematic diagram according to the fifth embodiment of the present disclosure. In the embodiments of the present disclosure, the color of the first to-be-matched vehicle and the color of the second to-be-matched vehicle can be compared, and the vehicle model of the first to-be-matched vehicle and the vehicle model of the second to-be-matched vehicle can be compared. According to the comparison result of the color of the first to-be-matched vehicle and the color of the second to-be-matched vehicle, and the comparison result of the vehicle model of the first to-be-matched vehicle and the vehicle model of the second to-be-matched vehicle, the matching result of the first attribute information and the second attribute information is determined, Figure 6 The embodiments shown can include the following steps:

[0093] Step 601, obtaining a first target image having a first to-be-matched vehicle and a second target image having a second to-be-matched vehicle from a video frame image.

[0094] Step 602, identifying the first target image and the second target image to obtain first key point information and first attribute information corresponding to the first to-be-matched vehicle, and second key point information and second attribute information corresponding to the second to-be-matched vehicle. The first attribute information includes a first color and a first vehicle model of the first to-be-matched vehicle, and the second attribute information includes a second color and a second vehicle model of the second to-be-matched vehicle.

[0095] Step 603, comparing the first color with the second color.

[0096] In the embodiments of the present disclosure, the RGB (red, green, blue) value corresponding to the first color is compared with the RGB corresponding to the second color to determine whether the first color matches the second color.

[0097] Step 604, in the case where the first color matches the second color, comparing the first vehicle model with the second vehicle model.

[0098] Further, in the case where the first color matches the second color, the first vehicle model of the first to-be-matched vehicle and the second vehicle model of the second to-be-matched vehicle are compared to determine whether the first vehicle model matches the second vehicle model.

[0099] ​In addition, in a case where the first color and the second color do not match, it is determined that the first attribute information and the second attribute information do not match.

[0100] In a case where the first vehicle model and the second vehicle model match, it is determined that the first attribute information and the second attribute information match.

[0101] In the embodiments of the present disclosure, in a case where the first vehicle model and the second vehicle model match, it is determined that the first attribute information and the second attribute information match; and in a case where the first vehicle model and the second vehicle model do not match, it is determined that the first attribute information and the second attribute information do not match.

[0102] In a case where the first attribute information and the second attribute information match, a matching result of the first to-be-matched vehicle and the second to-be-matched vehicle is determined according to the first key point information and the second key point information.

[0103] It should be noted that the execution processes of steps 601 to 602 and step 606 can be implemented by any one of the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this, and will not be repeated here.

[0104] In summary, by comparing the color of the first to-be-matched vehicle with the color of the second to-be-matched vehicle, and comparing the vehicle model of the first to-be-matched vehicle with the vehicle model of the second to-be-matched vehicle, according to the comparison result of the color of the first to-be-matched vehicle and the color of the second to-be-matched vehicle, and the comparison result of the vehicle model of the first to-be-matched vehicle and the vehicle model of the second to-be-matched vehicle, the matching result of the first attribute information and the second attribute information can be accurately determined.

[0105] In order to further improve the accuracy of the similarity model, as shown in Figure 7 , the similarity model can be adjusted according to the first target image and the second target image as training data, Figure 7 as shown in the sixth embodiment of the present disclosure, the first target image and the second target image can be used as training data to adjust the model parameters of the similarity model, Figure 7 The embodiments shown can include the following steps:

[0106] Step 701: obtaining a first target image with a first to-be-matched vehicle and a second target image with a second to-be-matched vehicle from a video frame image.

[0107] Step 702: identifying the first target image and the second target image to obtain first key point information and first attribute information corresponding to the first to-be-matched vehicle, and second key point information and second attribute information corresponding to the second to-be-matched vehicle.

[0108] In step 703, in a case where the first attribute information matches the second attribute information, a matching result of the first to-be-matched vehicle and the second to-be-matched vehicle is determined according to the first key point information and the second key point information.

[0109] In step 704, in a case where the first to-be-matched vehicle matches the second to-be-matched vehicle, the first target image and the second target image are taken as positive sample data in the training data.

[0110] In the embodiments of the present disclosure, in a case where the first to-be-matched vehicle matches the second to-be-matched vehicle, that is, the similarity between the first to-be-matched vehicle and the second to-be-matched vehicle is greater than a set similarity threshold, the first target image corresponding to the first to-be-matched vehicle and the second target image corresponding to the second to-be-matched vehicle are taken as positive sample data in the training data.

[0111] In step 705, in a case where the first to-be-matched vehicle does not match the second to-be-matched vehicle, the first target image and the second target image are taken as negative sample data in the training data.

[0112] Similarly, in a case where the first to-be-matched vehicle does not match the second to-be-matched vehicle, that is, the similarity between the first to-be-matched vehicle and the second to-be-matched vehicle is less than or equal to a set similarity threshold, the first target image corresponding to the first to-be-matched vehicle and the second target image corresponding to the second to-be-matched vehicle are taken as negative sample data in the training data.

[0113] In step 706, the model parameters in the similarity model are adjusted by using the positive sample data and / or the negative sample data in the training data.

[0114] Further, the model parameters in the similarity model are adjusted by using the positive sample data and / or the negative sample data in the training data, so as to improve the accuracy of the similarity model.

[0115] It should be noted that the execution processes of steps 701 to 703 can be implemented by any one of the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described again.

[0116] In summary, by taking the first target image and the second target image as positive sample data in the training data in a case where the first to-be-matched vehicle matches the second to-be-matched vehicle, and taking the first target image and the second target image as negative sample data in the training data in a case where the first to-be-matched vehicle does not match the second to-be-matched vehicle, and adjusting the model parameters in the similarity model by using the positive sample data and / or the negative sample data in the training data, the first target image and the second target image are taken as training data to adjust the model parameters in the similarity model according to the matching result of the first to-be-matched vehicle and the second to-be-matched vehicle, so as to improve the accuracy of the similarity model.

[0117] To illustrate the above embodiments more clearly, examples are given below for illustration.

[0118] As Figure 8 shown, Figure 8 is a schematic flowchart of a video-based vehicle matching method according to an embodiment of the present disclosure. The specific process is as follows:

[0119] (1) Obtain input: Obtain a first target image and a second target image from the video frames corresponding to the video captured by the camera;

[0120] (2) Compare whether the vehicle attributes in the two images are the same: Compare the colors of the first vehicle to be matched in the first target image and the second vehicle to be matched in the second target image. When the color of the first vehicle to be matched matches the color of the second vehicle to be matched, match the vehicle models of the first vehicle to be matched and the second vehicle to be matched. When the colors and models of the first vehicle to be matched are both the same as those of the second vehicle to be matched, proceed to step 3; otherwise, the first vehicle to be matched and the second vehicle to be matched are inconsistent;

[0121] (3) Compare the key point information of the first vehicle to be matched and the second vehicle to be matched, determine vehicle consistency based on the positional relationship of the corresponding key points in the image, and use the chessboard distance to find the position difference D of the key points where the first vehicle to be matched and the second vehicle to be matched are matched in the two images (the first target image and the second target image) is =|X1 - X2| + |Y1 - Y2|, where X1 and Y1 are the first coordinate information of the first key point in the first target image, and X2 and Y2 are the second coordinate information of each second key point in the second key point information in the second target image. Calculate the threshold th as two thousandths of the length I w of the first target image or the second target image (the first target image and the second target image have the same size) plus two thousandths of the width I h of the first target image or the second target image (the first target image and the second target image have the same size), that is, th = 2 * I w / 1000 + 2 * I h / l000. When D is <th, it is considered that the key points are matched consistently. When the number of consistently matched key points is greater than the set threshold N, it is considered that the vehicle heights are approximately the same. At this time, according to the setting of the sensitivity (matching accuracy), if the sensitivity (matching accuracy) is set to low, it is considered that the vehicles are the same and directly return; if the sensitivity (matching accuracy) is set to high, enter step 4 with the matching threshold th0 (0.3); if the number of consistently matched key points is less than the set threshold N, enter step 4;

[0122] (4) With the 3D modeling configuration of the image scene, the coordinate information of the first to-be-matched vehicle in the first target image, the key point information of the first to-be-matched vehicle, the coordinate information of the second to-be-matched vehicle in the second target image, and the key point information of the second to-be-matched vehicle as inputs, a 2D (image coordinate system) to 3D (world coordinate system) operation is performed on each point of the region of the first to-be-matched vehicle in the first target image to obtain a target 3D-box frame. From this operation, the length, width, height, angle, orientation, and other information of the first to-be-matched vehicle can be obtained. The first to-be-matched vehicle is regarded as a cuboid, that is, it is considered to have six faces. Generally, three faces can be displayed in the first target image according to different orientations. The displayed faces in the first target image can be extracted using the information obtained by the 2D to 3D conversion. For example, the vehicle head, roof, and side A are displayed in the first target image. If the orientation is reversed, the vehicle tail, roof, and side B can be obtained from the first target image. Similarly, the length, width, height, angle, orientation, and other information of the second to-be-matched vehicle can be obtained, and the displayed faces in the second target image can be extracted using the information obtained by the 2D to 3D conversion.

[0123] (5) Compare the orientations of the vehicles in the two images. If the orientations are different, go to step 6. If the orientations are the same, go to step 7.

[0124] (6) Compare the wheelbase, width, and height information of the two vehicles. Extract the side face images in the first target image and the second target image. If the compared information is consistent, go to step 8 with a threshold th1 (0.5). If the compared information is inconsistent, go to step 8 with a threshold th2 (0.7).

[0125] (7) Compare the wheelbase, width, and height information of the two vehicles (the first to-be-matched vehicle and the second to-be-matched vehicle). Extract the side face images in the two images. If the compared information is consistent, go to step 8 with a threshold th1 (0.5). If the compared information is inconsistent, go to step 8 with a threshold th2 (0.7).

[0126] (8) Align the vehicle profile images extracted in the previous step in terms of angle and scale, and then input them into a traditional ReID algorithm. Analyze the aligned images using a trained deep learning model to obtain their similarity. Compare the similarity with the threshold passed in the previous step. If the similarity is higher than the threshold, consider the first to-be-matched vehicle and the second to-be-matched vehicle in the two images to be the same vehicle. Otherwise, consider them to be different vehicles.

[0127] The method for vehicle matching based on video provided in the embodiments of the present disclosure comprises the following steps: obtaining a first target image with a first vehicle to be matched and a second target image with a second vehicle to be matched from a video frame image; identifying the first target image and the second target image to obtain first key point information and first attribute information corresponding to the first vehicle to be matched, and second key point information and second attribute information corresponding to the second vehicle to be matched; and determining a matching result of the first vehicle to be matched and the second vehicle to be matched according to the first key point information and the second key point information in the case that the first attribute information matches the second attribute information. The method can improve the matching efficiency and reliability of vehicle matching, reduce the inconsistency caused by angle transformation and occlusion, and improve the effectiveness of the vehicle matching result by further matching the vehicle key point information in the case that the first attribute information matches the second attribute information.

[0128] In order to realize the above-mentioned embodiments, the present disclosure provides a device for vehicle matching based on video.

[0129] Figure 9 is a schematic diagram according to the seventh embodiment of the present disclosure. As shown in Figure 9 The device for vehicle matching based on video 900 comprises a first obtaining module 910, an identifying module 920 and a first determining module 930.

[0130] The first obtaining module 910 is configured to obtain a first target image with a first vehicle to be matched and a second target image with a second vehicle to be matched from a video frame image; the identifying module 920 is configured to identify the first target image and the second target image to obtain first key point information and first attribute information corresponding to the first vehicle to be matched, and second key point information and second attribute information corresponding to the second vehicle to be matched; and the first determining module 930 is configured to determine a matching result of the first vehicle to be matched and the second vehicle to be matched according to the first key point information and the second key point information in the case that the first attribute information matches the second attribute information.

[0131] As a possible implementation manner of the embodiments of the present disclosure, the first determining module 930 is specifically configured to determine a target key point matched by each first key point in the first key point information from at least one second key point contained in the second key point information to obtain a key point pair matched with each other; obtain a set matching precision when the number of the key point pairs is greater than a set matching number threshold; and determine that the first vehicle to be matched and the second vehicle to be matched are matched when the matching precision is less than a set precision threshold.

[0132] As a possible implementation manner of the embodiments of the present disclosure, the device for vehicle matching based on video 900 further comprises a transformation module.

[0133] The transformation module is configured to, when the number of key point pairs is less than or equal to a set matching number threshold or when the matching accuracy is greater than or equal to a set accuracy threshold, transform each point of a region of the first to-be-matched vehicle in the first target image to a world coordinate system and transform each point of a region of the second to-be-matched vehicle in the second target image to the world coordinate system according to a mapping relationship between the image coordinate system and the world coordinate system, to obtain the size information and the pose information of the first to-be-matched vehicle and the size information and the pose information of the second to-be-matched vehicle. The first determination module 930 is further configured to determine the matching result of the first to-be-matched vehicle and the second to-be-matched vehicle according to the size information and the pose information of the first to-be-matched vehicle and the size information and the pose information of the second to-be-matched vehicle.

[0134] As a possible implementation manner of the embodiment of the present disclosure, the pose information includes a vehicle body direction, and the first determination module 930 is further configured to: extract a first side image of the first to-be-matched vehicle from the first target image and extract a second side image of the second to-be-matched vehicle from the second target image; align the first side image and the second side image according to similarities and differences between the vehicle body direction of the first to-be-matched vehicle and the vehicle body direction of the second to-be-matched vehicle, to obtain an aligned first side image and an aligned second side image; input the aligned first side image and the aligned second side image into a similarity model, to obtain a similarity of the aligned first side image and the aligned second side image; and determine the matching result of the first to-be-matched vehicle and the second to-be-matched vehicle according to the similarity and a set similarity threshold.

[0135] As a possible implementation manner of the embodiment of the present disclosure, the vehicle matching apparatus 900 based on a video further includes a first generation module, a second generation module, and a second determination module.

[0136] The first generation module is configured to generate a first similarity threshold according to the size information of the first to-be-matched vehicle and the size information of the second to-be-matched vehicle. The second generation module is configured to generate a second similarity threshold when the matching accuracy is greater than or equal to a set accuracy threshold. The second determination module is configured to determine the set similarity threshold according to the first similarity threshold and the second similarity threshold.

[0137] As a possible implementation manner of the embodiment of the present disclosure, the first attribute information includes a first color and a first vehicle model of the first to-be-matched vehicle, and the second attribute information includes a second color and a second vehicle model of the second to-be-matched vehicle. The vehicle matching apparatus based on a video further includes a first comparison module and a second comparison module.

[0138] The first comparison module is configured to compare the first color with the second color; the second comparison module is configured to compare the first vehicle model with the second vehicle model when the first color matches the second color; and the first determination module is configured to determine that the first attribute information matches the second attribute information when the first vehicle model matches the second vehicle model.

[0139] As a possible implementation manner of the embodiments of the present disclosure, the first determination module 930 is further configured to: obtain first coordinate information of any first key point in the first target image, and obtain second coordinate information of each second key point in the second target image in the second key point information; determine a position difference threshold according to size information of the first target image or the second target image; determine a target key point from the second key points according to a difference between the second coordinate information and the first coordinate information of each second key point, the difference being less than the position difference threshold; and take the first key point and the target key point as a key point pair.

[0140] As a possible implementation manner of the embodiments of the present disclosure, the vehicle matching apparatus 900 based on a video further includes a second acquisition module and an adjustment module.

[0141] The second acquisition module is configured to take the first target image and the second target image as positive sample data in the training data when the first vehicle to be matched matches the second vehicle to be matched; and the second acquisition module is further configured to take the first target image and the second target image as negative sample data in the training data when the first vehicle to be matched does not match the second vehicle to be matched; and the adjustment module is configured to adjust model parameters in the similarity model by using the positive sample data and / or the negative sample data in the training data.

[0142] The vehicle matching apparatus based on a video according to the embodiments of the present disclosure can obtain a first target image having a first vehicle to be matched and a second target image having a second vehicle to be matched from a video frame image; identify the first target image and the second target image to obtain first key point information and first attribute information corresponding to the first vehicle to be matched, and second key point information and second attribute information corresponding to the second vehicle to be matched; and determine a matching result of the first vehicle to be matched and the second vehicle to be matched according to the first key point information and the second key point information when the first attribute information matches the second attribute information. The apparatus can realize further matching of vehicle key point information when the first attribute information matches the second attribute information, improve matching efficiency and reliability of vehicle matching, reduce inconsistency caused by angle transformation, occlusion, and the like, and improve effectiveness of a vehicle matching result.

[0143] To achieve the above-mentioned embodiments, the present disclosure further proposes an electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the video-based vehicle matching method of the above-mentioned embodiments.

[0144] To achieve the above-mentioned embodiments, the present disclosure further proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to perform the video-based vehicle matching method of the above-mentioned embodiments.

[0145] To achieve the above-mentioned embodiments, the present disclosure further proposes a computer program product comprising a computer program, which, when executed by a processor, implements the video-based vehicle matching method of the above-mentioned embodiments.

[0146] It should be noted that in the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are all carried out on the premise of obtaining the consent of the user, and all comply with the relevant legal regulations and do not violate public order and good customs.

[0147] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0148] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present disclosure described and / or claimed in this document.

[0149] As Figure 10As shown, the device 1000 includes a computing unit 1001 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded into a random access memory (RAM) 1003 from a storage unit 1008. Various programs and data required for the operation of the device 1000 can also be stored in the RAM 1003. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0150] A plurality of components in the device 1000 are connected to the I / O interface 1005, including an input unit 1006 such as a keyboard, a mouse, etc., an output unit 1007 such as various types of displays, speakers, etc., a storage unit 1008 such as a magnetic disk, an optical disk, etc., and a communication unit 1009 such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0151] The computing unit 1001 can be various general and / or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1001 performs various methods and processes described above, such as the video-based vehicle matching method. For example, in some embodiments, the video-based vehicle matching method can be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the video-based vehicle matching method described above can be performed. Alternatively, in other embodiments, the computing unit 1001 can be configured to perform the video-based vehicle matching method by any other appropriate means, such as by means of firmware.

[0152] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a load programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0153] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, or entirely on a remote machine or server.

[0154] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0155] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0156] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0157] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server is generally established by computer programs running on the respective computers and having a client-server relationship to each other. A server can also be a server of a distributed system, or a server in connection with a blockchain.

[0158] It should be noted that artificial intelligence is a discipline that studies enabling computers to simulate some thinking processes and intelligent behaviors of people (such as learning, reasoning, thinking, planning, etc.), both hardware and software technologies. Artificial intelligence hardware technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing, etc.; artificial intelligence software technology mainly includes computer vision technology, speech recognition technology, natural language processing technology, and machine learning / deep learning, big data processing technology, knowledge graph technology, etc. several major directions.

[0159] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps described in the present disclosure can be performed in parallel, in series, or in a different order, as long as the desired results of the technology disclosed in the present disclosure are achieved, which is not limited herein.

[0160] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A video-based vehicle matching method, comprising: obtaining a first target image with a first vehicle to be matched and a second target image with a second vehicle to be matched from video frame images; performing recognition on the first target image and the second target image to obtain first key point information and first attribute information corresponding to the first vehicle to be matched, and second key point information and second attribute information corresponding to the second vehicle to be matched; in a case where the first attribute information matches the second attribute information, determining a target key point matched with each first key point included in the first key point information from at least one second key point included in the second key point information to obtain a key point pair matched with each other; in a case where a number of the key point pairs is greater than a set matching number threshold, obtaining a matching accuracy; in a case where the matching accuracy is less than a set accuracy threshold, determining that the first vehicle to be matched matches the second vehicle to be matched; in a case where the number of the key point pairs is less than or equal to the set matching number threshold, or in a case where the matching accuracy is greater than or equal to the set accuracy threshold, determining a matching result of the first vehicle to be matched and the second vehicle to be matched according to size information and pose information of the first vehicle to be matched and size information and pose information of the second vehicle to be matched.

2. The method of claim 1, wherein, the determining the matching result of the first vehicle to be matched and the second vehicle to be matched according to the size information and the pose information of the first vehicle to be matched and the size information and the pose information of the second vehicle to be matched comprises: transforming each point of a region of the first vehicle to be matched in the first target image to a world coordinate system and transforming each point of a region of the second vehicle to be matched in the second target image to the world coordinate system according to a mapping relationship between an image coordinate system and the world coordinate system to obtain the size information and the pose information of the first vehicle to be matched and the size information and the pose information of the second vehicle to be matched; determining the matching result of the first vehicle to be matched and the second vehicle to be matched according to the size information and the pose information of the first vehicle to be matched and the size information and the pose information of the second vehicle to be matched.

3. The method of claim 2, wherein, the pose information comprises a body direction, and the determining the matching result of the first vehicle to be matched and the second vehicle to be matched according to the size information and the pose information of the first vehicle to be matched and the size information and the pose information of the second vehicle to be matched comprises: extracting a first side image of the first vehicle to be matched from the first target image and extracting a second side image of the second vehicle to be matched from the second target image; aligning the first side image and the second side image according to a difference between the body direction of the first vehicle to be matched and the body direction of the second vehicle to be matched to obtain an aligned first side image and an aligned second side image; inputting the aligned first side image and the aligned second side image into a similarity model to obtain a similarity of the aligned first side image and the aligned second side image. According to the similarity and a set similarity threshold, a matching result of the first vehicle to be matched and the second vehicle to be matched is determined.

4. The method of claim 3, wherein, Before the matching result of the first vehicle to be matched and the second vehicle to be matched is determined according to the similarity and the set similarity threshold, the method further includes: According to the size information of the first vehicle to be matched and the size information of the second vehicle to be matched, a first similarity threshold is generated; When the matching accuracy is greater than or equal to a set accuracy threshold, a second similarity threshold is generated; According to the first similarity threshold and the second similarity threshold, the set similarity threshold is determined.

5. The method of claim 1, wherein, The first attribute information includes a first color and a first vehicle model of the first vehicle to be matched, and the second attribute information includes a second color and a second vehicle model of the second vehicle to be matched. Before the matching result of the first vehicle to be matched and the second vehicle to be matched is determined according to the first key point information and the second key point information in the case that the first attribute information and the second attribute information match, the method further includes: The first color and the second color are compared; In the case that the first color and the second color match, the first vehicle model and the second vehicle model are compared; In the case that the first vehicle model and the second vehicle model match, it is determined that the first attribute information and the second attribute information match.

6. The method of claim 1, wherein, In the case that the first attribute information and the second attribute information match, a target key point to which each first key point included in the first key point information matches is determined from at least one second key point included in the second key point information, to obtain a key point pair that matches each other. First coordinate information of any first key point in the first target image is obtained, and second coordinate information of each second key point in the second target image in the second key point information is obtained; According to size information of the first target image or the second target image, a position difference threshold is determined; According to a difference value between the second coordinate information of each second key point and the first coordinate information, a target key point having a difference value less than the position difference threshold is determined from each second key point; The first key point and the target key point are taken as a key point pair.

7. The method of any one of claims 1-6, wherein, The method further includes: When the first vehicle to be matched and the second vehicle to be matched match, the first target image and the second target image are taken as positive sample data in training data; When the first vehicle to be matched and the second vehicle to be matched do not match, the first target image and the second target image are taken as negative sample data in training data; Model parameters in a similarity model are adjusted by using the positive sample data and / or the negative sample data in the training data.

8. A vehicle matching device based on video, comprising: A first acquisition module is configured to acquire a first target image having a first vehicle to be matched and a second target image having a second vehicle to be matched from a video frame image. The recognition module is configured to recognize the first target image and the second target image to obtain first key point information and first attribute information corresponding to the first to-be-matched vehicle and second key point information and second attribute information corresponding to the second to-be-matched vehicle. The first determination module is configured to, in a case where the first attribute information matches the second attribute information, determine, from at least one second key point included in the second key point information, a target key point matched with each first key point included in the first key point information, to obtain a key point pair that matches each other; in a case where a quantity of the key point pairs is greater than a set matching quantity threshold, acquire a matching precision; in a case where the matching precision is less than a set precision threshold, determine that the first to-be-matched vehicle matches the second to-be-matched vehicle; in a case where the quantity of the key point pairs is less than or equal to the set matching quantity threshold, or in a case where the matching precision is greater than or equal to the set precision threshold, determine a matching result of the first to-be-matched vehicle and the second to-be-matched vehicle according to size information and pose information of the first to-be-matched vehicle and size information and pose information of the second to-be-matched vehicle.

9. The apparatus of claim 8, wherein, The apparatus further includes: The transformation module is configured to, in a case where the quantity of the key point pairs is less than or equal to the set matching quantity threshold, or in a case where the matching precision is greater than or equal to the set precision threshold, transform, according to a mapping relationship between an image coordinate system and a world coordinate system, each point of a region of the first to-be-matched vehicle in the first target image to the world coordinate system and each point of a region of the second to-be-matched vehicle in the second target image to the world coordinate system, to obtain the size information and the pose information of the first to-be-matched vehicle and the size information and the pose information of the second to-be-matched vehicle. The first determination module is further configured to determine the matching result of the first to-be-matched vehicle and the second to-be-matched vehicle according to the size information and the pose information of the first to-be-matched vehicle and the size information and the pose information of the second to-be-matched vehicle.

10. The apparatus of claim 9, wherein, The pose information includes a vehicle body direction, and the first determination module is further configured to: extract a first side image of the first to-be-matched vehicle from the first target image and a second side image of the second to-be-matched vehicle from the second target image; align the first side image and the second side image according to a difference between the vehicle body direction of the first to-be-matched vehicle and the vehicle body direction of the second to-be-matched vehicle, to obtain an aligned first side image and an aligned second side image; input the aligned first side image and the aligned second side image into a similarity model, to obtain a similarity of the aligned first side image and the aligned second side image; determine the matching result of the first to-be-matched vehicle and the second to-be-matched vehicle according to the similarity and a set similarity threshold.

11. The apparatus of claim 10, wherein, The apparatus further includes: The first generation module is configured to generate a first similarity threshold according to the size information of the first to-be-matched vehicle and the size information of the second to-be-matched vehicle. The second generation module is configured to generate a second similarity threshold when the matching accuracy is greater than or equal to a set accuracy threshold. The second determination module is configured to determine the set similarity threshold according to the first similarity threshold and the second similarity threshold.

12. The apparatus of claim 8, wherein, The first attribute information includes a first color and a first vehicle model of the first vehicle to be matched, and the second attribute information includes a second color and a second vehicle model of the second vehicle to be matched. The device further includes: The first comparison module is configured to compare the first color with the second color. The second comparison module is configured to compare the first vehicle model with the second vehicle model when the first color matches the second color. The first determination module is configured to determine that the first attribute information and the second attribute information match when the first vehicle model matches the second vehicle model.

13. The apparatus of claim 8, wherein, The first determination module is further configured to: obtain first coordinate information of any first key point in the first target image, and obtain second coordinate information of each second key point in the second target image in the second key point information; determine a position difference threshold according to size information of the first target image or the second target image; determine a target key point from each second key point according to a difference between the second coordinate information of each second key point and the first coordinate information, wherein the difference of the target key point is less than the position difference threshold; and take the first key point and the target key point as a key point pair.

14. The apparatus of any one of claims 8-13, wherein, The device further includes: The second acquisition module is configured to take the first target image and the second target image as positive sample data in training data when the first vehicle to be matched matches the second vehicle to be matched. The second acquisition module is further configured to take the first target image and the second target image as negative sample data in training data when the first vehicle to be matched does not match the second vehicle to be matched. The adjustment module is configured to adjust model parameters in a similarity model using positive sample data and / or negative sample data in the training data.

15. An electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

16. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of any one of claims 1-7.

17. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1-7.

17. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1-7.

Images