Vehicle merging and identification method, device and equipment

By extracting vehicle image information from the camera video stream and converting it into real world coordinates, combining the target chassis graphics and attribute information to match, the problem of low accuracy of vehicle merger recognition in offline retail scenarios is solved, and high accuracy and low accuracy are achieved Cost of vehicle merger identification.

CN118692009BActive Publication Date: 2024-11-19ZHE JIANG SHEN XIANG ZHI NENG KE JI YOU XIAN GONG SI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411155141.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-22
Publication Date
2024-11-19
Estimated Expiration
2044-08-22

AI Technical Summary

Technical Problem

Existing vehicle merger recognition technology In offline retail scenarios, due to the high maintenance cost of data base database and the untimely update of data, the accuracy of vehicle merger recognition is not high, especially in multi-camera scenarios due to the differences in angle and morphology.

Method used

By extracting vehicle image information, including chassis image coordinates and attribute information from the camera video stream, converting it into actual world coordinates, and matching with target chassis graphics and attribute information, vehicle merge and recognition are realized, and dependence on the data base database is reduced.

Benefits of technology

It improves the accuracy of vehicle merger recognition, reduces data maintenance costs, avoids identification errors caused by incomplete data base database, and is suitable for vehicle merger processing in multi-camera scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118692009B_ABST
    Figure CN118692009B_ABST
Patent Text Reader

Abstract

The embodiment of the present application provides a vehicle merging and identifying method, device and equipment, the method comprising: an electronic device obtains a video stream sent by at least one shooting device, and extracts vehicle image information from each video stream; the vehicle image information includes chassis image coordinates and vehicle attribute information; the actual world coordinates corresponding to the chassis image coordinates are determined, and the target chassis graphics corresponding to the vehicle image information are determined based on the actual world coordinates; for any two vehicle image information, if the two target chassis graphics and two vehicle attribute information corresponding to the two vehicle image information meet the target matching conditions, it is determined that the two vehicle image information correspond to the same target vehicle, and the two vehicle image information are added to the target group corresponding to the target vehicle; according to the vehicle image information in the target group, the target attribute information corresponding to the target vehicle is determined. This improves the accuracy of vehicle merging and identifying, and also reduces the data maintenance cost.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a vehicle merging and identifying method, device and equipment. Background Art

[0002] With the continuous development of computer and Internet technology, there is an increasing demand for statistical analysis of customer flow data in offline retail scenarios. In offline scenarios such as car showrooms, in order to accurately analyze the customer flow data corresponding to different vehicles, it is usually necessary to merge and identify the same vehicle that appears in the camera.

[0003] In the related art, when merging and identifying vehicles, electronic devices usually compare and identify the vehicle images taken by the camera with the vehicle data in the database. This vehicle merging and identification method has high data maintenance costs, and due to the untimely and incomplete updating of the database data, it is easy to have low accuracy in vehicle merging and identification. Summary of the invention

[0004] Multiple aspects of the present application provide a vehicle merging and identifying method, device and equipment, which can reduce data maintenance costs and improve the accuracy of vehicle merging and identifying.

[0005] In a first aspect, an embodiment of the present application provides a vehicle merging and identifying method, comprising:

[0006] Acquire a video stream sent by at least one shooting device, and extract vehicle image information from each of the video streams; the vehicle image information includes chassis image coordinates and vehicle attribute information;

[0007] Determine the actual world coordinates corresponding to the chassis image coordinates, and determine the target chassis graphics corresponding to the vehicle image information based on the actual world coordinates;

[0008] For any two vehicle image information, if the two target chassis graphics and two vehicle attribute information corresponding to the two vehicle image information meet the target matching condition, it is determined that the two vehicle image information correspond to the same target vehicle, and the two vehicle image information are added to the target group corresponding to the target vehicle;

[0009] According to the vehicle image information in the target group, target attribute information corresponding to the target vehicle is determined.

[0010] In a possible implementation manner, extracting vehicle image information from each of the video streams includes:

[0011] For each image frame of the video stream, determining a vehicle image region in the image frame according to a preset vehicle detection model;

[0012] The vehicle image area is identified and processed according to a preset vehicle feature extraction model to obtain vehicle image information corresponding to the vehicle image area.

[0013] In a possible implementation, the method further includes:

[0014] Extracting pedestrian trajectory information from each of the video streams;

[0015] Determining a target interaction event corresponding to the vehicle image information according to the pedestrian trajectory information and the vehicle image information;

[0016] Determining whether the vehicle image information meets a preset data condition according to the target interaction event;

[0017] If the vehicle image information does not meet the preset data condition, the vehicle image information is deleted.

[0018] In a possible implementation manner, the vehicle image information further includes vehicle integrity information and chassis image coordinate confidence; and determining whether the vehicle image information satisfies a preset data condition according to the target interaction event includes:

[0019] If the number of data missing coordinates in the chassis image coordinates is greater than a preset number threshold, it is determined that the vehicle image information does not meet the preset data condition;

[0020] If the average chassis frame pixel length corresponding to the chassis image coordinates is less than a preset length threshold, it is determined that the vehicle image information does not meet the preset data condition;

[0021] If the average value of the chassis image coordinate confidence is less than the target confidence threshold, or there is a chassis image coordinate confidence that is less than a preset minimum confidence threshold, it is determined that the vehicle image information does not meet the preset data condition; the target confidence threshold is determined according to the target interaction event number and a first mapping relationship; the first mapping relationship is a mapping relationship between the target interaction event number and a preset confidence threshold;

[0022] If the vehicle integrity information is less than the target integrity threshold, it is determined that the vehicle image information does not meet the preset data conditions; the target integrity threshold is determined based on the target number of interaction events and a second mapping relationship; the second mapping relationship is a mapping relationship between the target number of interaction events and a preset integrity threshold.

[0023] In a possible implementation, the method further includes:

[0024] Determine the target ratio threshold condition corresponding to the two vehicle image information according to the matching result of the two vehicle attribute information and the third mapping relationship; the third mapping relationship includes a mapping relationship between the matching result of the vehicle attribute information and the preset ratio threshold condition;

[0025] If the intersection area of ​​the two target chassis graphics corresponding to the two vehicle image information meets the target ratio threshold condition, it is determined that the two target chassis graphics corresponding to the two vehicle image information and the two vehicle attribute information meet the target matching condition.

[0026] In a possible implementation manner, determining target attribute information corresponding to the target vehicle according to the vehicle image information in the target group includes:

[0027] For the vehicle image information in the target group, determining a target quality score corresponding to the vehicle image information; the target quality score is determined according to the confidence of the chassis image coordinates and the confidence of the vehicle attribute information;

[0028] The vehicle image information is grouped according to different vehicle attribute information to obtain target subgroups; the vehicle attribute information in each target subgroup is the same; the vehicle attribute information includes at least one of a vehicle brand, a vehicle model, and a vehicle color;

[0029] The vehicle attribute information corresponding to the target subgroup with the largest average value of the target quality score is determined as the target attribute information corresponding to the target vehicle.

[0030] In a second aspect, an embodiment of the present application provides a vehicle merging and identifying device, including:

[0031] An acquisition module, used to acquire a video stream sent by at least one shooting device, and extract vehicle image information from each of the video streams; the vehicle image information includes chassis image coordinates and vehicle attribute information;

[0032] A first determination module, configured to determine the actual world coordinates corresponding to the chassis image coordinates, and determine the target chassis graphic corresponding to the vehicle image information based on the actual world coordinates;

[0033] A merging module, for any two vehicle image information, if the two target chassis graphics and two vehicle attribute information corresponding to the two vehicle image information meet the target matching condition, then determine that the two vehicle image information correspond to the same target vehicle, and add the two vehicle image information to the target group corresponding to the target vehicle;

[0034] The second determination module is used to determine the target attribute information corresponding to the target vehicle according to the vehicle image information in the target group.

[0035] In a possible implementation manner, the acquisition module is specifically configured to:

[0036] For each image frame of the video stream, determining a vehicle image region in the image frame according to a preset vehicle detection model;

[0037] The vehicle image area is identified and processed according to a preset vehicle feature extraction model to obtain vehicle image information corresponding to the vehicle image area.

[0038] In a possible implementation manner, the device is further used for:

[0039] Extracting pedestrian trajectory information from each of the video streams;

[0040] Determining a target interaction event corresponding to the vehicle image information according to the pedestrian trajectory information and the vehicle image information;

[0041] Determining whether the vehicle image information meets a preset data condition according to the target interaction event;

[0042] If the vehicle image information does not meet the preset data condition, the vehicle image information is deleted.

[0043] In a possible implementation manner, the vehicle image information further includes vehicle integrity information and chassis image coordinate confidence; and the device is further used for:

[0044] If the number of data missing coordinates in the chassis image coordinates is greater than a preset number threshold, it is determined that the vehicle image information does not meet the preset data condition;

[0045] If the average chassis frame pixel length corresponding to the chassis image coordinates is less than a preset length threshold, it is determined that the vehicle image information does not meet the preset data condition;

[0046] If the average value of the chassis image coordinate confidence is less than the target confidence threshold, or there is a chassis image coordinate confidence that is less than a preset minimum confidence threshold, it is determined that the vehicle image information does not meet the preset data condition; the target confidence threshold is determined according to the target interaction event number and a first mapping relationship; the first mapping relationship is a mapping relationship between the target interaction event number and a preset confidence threshold;

[0047] If the vehicle integrity information is less than the target integrity threshold, it is determined that the vehicle image information does not meet the preset data conditions; the target integrity threshold is determined based on the target number of interaction events and a second mapping relationship; the second mapping relationship is a mapping relationship between the target number of interaction events and a preset integrity threshold.

[0048] In a possible implementation manner, the device is further used for:

[0049] Determine the target ratio threshold condition corresponding to the two vehicle image information according to the matching result of the two vehicle attribute information and the third mapping relationship; the third mapping relationship includes a mapping relationship between the matching result of the vehicle attribute information and the preset ratio threshold condition;

[0050] If the intersection area of ​​the two target chassis graphics corresponding to the two vehicle image information meets the target ratio threshold condition, it is determined that the two target chassis graphics corresponding to the two vehicle image information and the two vehicle attribute information meet the target matching condition.

[0051] In a possible implementation manner, the second determining module is specifically configured to:

[0052] For the vehicle image information in the target group, determining a target quality score corresponding to the vehicle image information; the target quality score is determined according to the confidence of the chassis image coordinates and the confidence of the vehicle attribute information;

[0053] The vehicle image information is grouped according to different vehicle attribute information to obtain target subgroups; the vehicle attribute information in each target subgroup is the same; the vehicle attribute information includes at least one of a vehicle brand, a vehicle model, and a vehicle color;

[0054] The vehicle attribute information corresponding to the target subgroup with the largest average value of the target quality score is determined as the target attribute information corresponding to the target vehicle.

[0055] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory and a processor;

[0056] The memory stores computer-executable instructions;

[0057] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the vehicle merging and identification method described in any one of the first aspects.

[0058] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer execution instructions, which, when executed by a processor, are used to implement the vehicle merging and identification method described in any one of the first aspects.

[0059] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the vehicle merging and identification method shown in any one of the first aspects.

[0060] In an embodiment of the present application, an electronic device obtains a video stream sent by at least one shooting device, and extracts vehicle image information from each video stream; the vehicle image information includes chassis image coordinates and vehicle attribute information; the actual world coordinates corresponding to the chassis image coordinates are determined, and a target chassis graphic corresponding to the vehicle image information is determined based on the actual world coordinates; for any two vehicle image information, if the two target chassis graphics corresponding to the two vehicle image information and the two vehicle attribute information meet the target matching condition, it is determined that the two vehicle image information correspond to the same target vehicle, and the two vehicle image information are added to the target group corresponding to the target vehicle; based on the vehicle image information in the target group, the target attribute information corresponding to the target vehicle is determined. In the present application, the electronic device extracts vehicle image information from a video stream captured by at least one shooting device, and then determines the target chassis graphic based on the chassis image coordinates in the vehicle image information, determines whether the two vehicle image information correspond to the same target vehicle based on the matching of the target chassis graphic and the vehicle attribute information, and then determines the target attribute information of the target vehicle based on the vehicle image information in the same target group. In this way, the same vehicle can be merged and identified through comprehensive judgment of multiple features such as the target chassis graphic and the vehicle attribute information, thereby improving the accuracy of vehicle merged identification. At the same time, there is no need to compare and identify with the data base, which can reduce data maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0062] Figure 1 A schematic diagram of an application scenario provided for an exemplary embodiment of the present application;

[0063] Figure 2 A flow chart of a vehicle merging and identifying method provided for an exemplary embodiment of the present application;

[0064] Figure 3 A flow chart of another vehicle merging and identifying method provided for an exemplary embodiment of the present application;

[0065] Figure 4 A schematic diagram of a vehicle image information extraction logic provided for an exemplary embodiment of the present application;

[0066] Figure 5 A schematic diagram of vehicle image information provided for an exemplary embodiment of the present application;

[0067] Figure 6 A schematic diagram of determining the intersection area of ​​a target chassis figure provided by an exemplary embodiment of the present application;

[0068] Figure 7 A schematic diagram of a system architecture for vehicle merging and identification provided for an exemplary embodiment of the present application;

[0069] Figure 8 A schematic diagram of the structure of a vehicle merging and identifying device provided for an exemplary embodiment of the present application;

[0070] Fig. 9 A schematic structural diagram of an electronic device provided for an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of them. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0072] In offline car sales scenarios, merchants (or stores, etc.) usually need to conduct statistical analysis on passenger flow indicators such as browsing rate and vehicle entry rate of vehicles of different brands and models, and then adjust sales strategies based on passenger flow indicators. In order to achieve comprehensive statistical analysis of passenger flow indicators, merchants first need to merge the same vehicles captured by the camera.

[0073] In order to realize the vehicle merging and identification based on the camera, in the related technology, the electronic device usually compares and identifies the vehicle image captured by the camera with the vehicle data in the data base to realize the vehicle merging and identification. This vehicle merging and identification method has high data maintenance costs for the data base, and on the other hand, due to product changes in offline retail scenarios, the data in the data base cannot be updated in time or the data in the data base is incomplete, so the vehicle image captured by the camera cannot be matched with the vehicle data in the data base, resulting in low accuracy of vehicle merging and identification. In addition, since there are multiple cameras in the offline scene, the installation positions and shooting angles of different cameras are inconsistent, and the angles and shapes of the collected vehicle images vary greatly, which further leads to low accuracy of vehicle merging and identification.

[0074] In order to solve the above problems, the present application provides a vehicle merging and identifying method, device and equipment, in which the electronic device extracts vehicle image information from a video stream captured by at least one shooting device, and then performs coordinate conversion according to the chassis image coordinates in the vehicle image information to obtain the actual world coordinates, and then determines the target chassis graphics corresponding to the vehicle image information according to the actual world coordinates, and then for any two vehicle image information, the electronic device determines whether the two vehicle image information correspond to the same target vehicle based on the matching of the target chassis graphics and the vehicle attribute information. If the target matching conditions are met, the two vehicle image information are added to the same target group, and finally the electronic device can determine the target attribute information of the target vehicle based on the vehicle image information in the same target group, thereby realizing the merging and identifying of the vehicle. In this way, the electronic device in the present application can perform merging and identifying based on a combination of multiple features such as vehicle chassis image coordinates and vehicle attribute information, which can improve the accuracy of vehicle merging and identifying, and at the same time, there is no need to perform comparative identification based on the data base, which can reduce the data maintenance cost.

[0075] Figure 1 A schematic diagram of an application scenario provided by an exemplary embodiment of the present application. Figure 1 As shown, in the related art, when performing vehicle merging and identification, the vehicle image taken by the camera is usually compared and matched with the data base to achieve vehicle merging and identification. This vehicle merging and identification method has a high data cost and the accuracy of merging and identification is not high.

[0076] In the embodiment of the present application, the electronic device extracts vehicle image information from the video stream of at least one shooting device, and then determines the target chassis graphic according to the chassis image coordinates in the vehicle image information, and then determines whether the two vehicle image information correspond to the same target vehicle based on the matching of the target chassis graphic and the vehicle attribute information for any two vehicle image information, and then determines the target attribute information of the target vehicle based on the vehicle image information in the same target group. In this way, the electronic device can improve the accuracy of vehicle merging and recognition, and can also reduce data maintenance costs.

[0077] The technical solution shown in the present application is described in detail below through specific embodiments. It should be noted that the following embodiments can exist independently or in combination with each other, and the same or similar contents will not be described repeatedly in different embodiments.

[0078] Figure 2 A flow chart of a vehicle merging and identification method provided for an exemplary embodiment of the present application. Figure 2 , the vehicle merging and identification method may include:

[0079] S201. Obtain video streams sent by at least one shooting device, and extract vehicle image information from each video stream; the vehicle image information includes chassis image coordinates and vehicle attribute information.

[0080] The execution subject of the embodiment of the present application may be an electronic device, or a vehicle merging and identifying device provided in the electronic device. The vehicle merging and identifying device may be implemented by software, or by a combination of software and hardware. For ease of understanding, the following description will be given by taking the execution subject as an electronic device as an example. The electronic device may specifically refer to an edge server, a cloud, or a mobile phone, etc., and of course may also be other types. The embodiment of the present application does not limit the specific type of the electronic device.

[0081] In the embodiment of the present application, the shooting device may refer to a camera in an offline sales scenario, and specifically may be an Internet Protocol Camera (IPC), etc. The embodiment of the present application does not limit the specific type of the shooting device. It should be noted that in the embodiment of the present application, the electronic device may perform merged recognition based on a video stream captured by a camera, i.e., merged recognition of vehicles under a single camera, or may perform merged recognition based on video streams captured by two or more cameras, i.e., merged recognition of vehicles under multiple cameras.

[0082] The video stream may refer to a video stream captured and sent by a camera. In a possible implementation, the video stream is a video stream collected by the camera within a preset time period. Specifically, in order to avoid unclear shooting due to unstable ambient light, the camera may upload the video stream within a preset time period to ensure the clarity of the video stream. The preset time period may be from 10 a.m. to 6 p.m., etc., and may be flexibly set based on geographic location, season, actual sales time, etc., and the embodiments of the present application are not limited to this.

[0083] Vehicle image information may refer to various information corresponding to the vehicle image in the video stream, and the vehicle image information may also be referred to as vehicle sampling data. The vehicle image information may include chassis image coordinates and vehicle attribute information, wherein the chassis image coordinates may refer to the image coordinates of four points of the vehicle chassis frame. The vehicle attribute information may specifically include attribute information of the vehicle itself, such as vehicle brand, vehicle model, and vehicle color, and may also include shooting attribute information such as shooting device identification and shooting time. Of course, the vehicle image information may also include other types of information, which may be flexibly set based on actual needs, and the embodiments of the present application are not limited to this.

[0084] Specifically, in an offline retail scenario, an electronic device can obtain a video stream sent by at least one shooting device, and then detect and identify each video stream, extract vehicle image information from each video stream, and subsequently merge and identify each vehicle image information.

[0085] S202: Determine the actual world coordinates corresponding to the chassis image coordinates, and determine the target chassis graphics corresponding to the vehicle image information based on the actual world coordinates.

[0086] In the embodiment of the present application, the actual world coordinates may refer to the world coordinates corresponding to the four points of the vehicle chassis frame in the vehicle image information. The target chassis figure may refer to a polygon composed of the actual world coordinates of the four points of the vehicle chassis frame, which is used to represent the actual position of the vehicle corresponding to the vehicle image information.

[0087] In offline retail scenarios, cars used for exhibition in car showrooms are usually fixed in position, so the actual position of the vehicle can be used as a judgment feature for vehicle merging and identification. The chassis image coordinates in the vehicle image information extracted by the electronic device from each video stream refer to the pixel coordinates of the vehicle chassis frame in the image. The electronic device can convert the chassis image coordinates into real world coordinates through coordinate mapping conversion, and compose a target chassis graphic representing the actual position of the vehicle based on the real world coordinates. The conversion method from chassis image coordinates to real world coordinates can be converted and calculated through homography matrices, etc., and this is not limited in the embodiments of the present application.

[0088] S203. For any two vehicle image information, if the two target chassis graphics corresponding to the two vehicle image information and the two vehicle attribute information meet the target matching condition, it is determined that the two vehicle image information correspond to the same target vehicle, and the two vehicle image information are added to the target group corresponding to the target vehicle.

[0089] In the embodiment of the present application, the target matching condition may refer to the matching condition between the preset target chassis graphics and the vehicle attribute information. For example, the target matching condition may be that the vehicle brand, vehicle model, and vehicle color are the same in the two vehicle image information, and the intersection area (or overlapping area) of the target chassis graphics in the two target chassis graphics is higher than the target ratio threshold. Of course, the target matching condition may also include other matching conditions, which are not limited in the embodiment of the present application. The target vehicle may refer to the same vehicle corresponding to the two vehicle image information that meet the target matching condition, and the target vehicle may correspond to a unique vehicle identification (Identity document, ID). The target group may refer to the group corresponding to the target vehicle, and the target group corresponds to the vehicle image information that meets the target matching condition and corresponds to the same target vehicle.

[0090] In this step, after the electronic device obtains the target vehicle graphics corresponding to the vehicle image information, it can compare and match any two vehicle image information in the video stream collected by each shooting device, and specifically determine whether the two vehicle attribute information and the two target chassis graphics corresponding to the two vehicle image information meet the target matching conditions. If the target matching conditions are met, the electronic device can determine that the two vehicle image information correspond to the same target vehicle, and the two vehicle image information can be added to the same target group; if the target matching conditions are not met, the electronic device can determine that the two vehicle image information do not correspond to the same target vehicle. In addition, the electronic device can merge the two matched vehicle image information in series through recursive processing. For example, if the vehicle image information A matches the vehicle image information B (i.e., meets the target matching conditions), and the vehicle image information B matches the vehicle image information C, then the vehicle image information A, B, and C correspond to the same target vehicle and can be added to the same target group. In this way, the electronic device matches the two vehicle image information in pairs through the target chassis graphics and the vehicle attribute information, and can realize the vehicle merging processing in a single-mirror or multi-mirror scene based on multiple features, which can improve the accuracy and rationality of vehicle merging.

[0091] S204: Determine target attribute information corresponding to the target vehicle according to the vehicle image information in the target group.

[0092] In an embodiment of the present application, the target attribute information may refer to the attribute information corresponding to the target vehicle, specifically, the vehicle brand, vehicle color, and vehicle model of the target vehicle. After the electronic device completes the pairwise matching of the vehicle image information in all video streams, for any target group, the electronic device may determine the target attribute information corresponding to the target vehicle by grouping comparison, sorting comparison, or threshold comparison based on the confidence level of the vehicle attribute information in each vehicle image information. In this way, the electronic device can accurately determine the target attribute information of the target vehicle by identifying the vehicle image information in the target group, and at the same time, there is no need to compare and identify with the data base, which can reduce data maintenance costs.

[0093] In an embodiment of the present application, an electronic device obtains a video stream sent by at least one shooting device, and extracts vehicle image information from each video stream; the vehicle image information includes chassis image coordinates and vehicle attribute information; the actual world coordinates corresponding to the chassis image coordinates are determined, and a target chassis graphic corresponding to the vehicle image information is determined based on the actual world coordinates; for any two vehicle image information, if the two target chassis graphics corresponding to the two vehicle image information and the two vehicle attribute information meet the target matching condition, it is determined that the two vehicle image information correspond to the same target vehicle, and the two vehicle image information are added to the target group corresponding to the target vehicle; based on the vehicle image information in the target group, the target attribute information corresponding to the target vehicle is determined. In the present application, the electronic device extracts vehicle image information from a video stream captured by at least one shooting device, and then determines the target chassis graphic based on the chassis image coordinates in the vehicle image information, determines whether the two vehicle image information correspond to the same target vehicle based on the matching of the target chassis graphic and the vehicle attribute information, and then determines the target attribute information of the target vehicle based on the vehicle image information in the same target group. In this way, the same vehicle can be merged and identified through comprehensive judgment of multiple features such as the target chassis graphic and the vehicle attribute information, thereby improving the accuracy of vehicle merged identification. At the same time, there is no need to compare and identify with the data base, which can reduce data maintenance costs.

[0094] Based on the above embodiments, Figure 3 A flowchart of another vehicle merging and identification method provided for an exemplary embodiment of the present application. Figure 3 , the vehicle merging and identification method may include:

[0095] S301. Obtain a video stream sent by at least one shooting device, and determine a vehicle image area in each image frame of the video stream according to a preset vehicle detection model.

[0096] S302, identifying and processing the vehicle image area according to a preset vehicle feature extraction model to obtain vehicle image information corresponding to the vehicle image area; the vehicle image information includes chassis image coordinates and vehicle attribute information.

[0097] In the embodiments of the present application, the preset vehicle detection model may refer to a pre-trained algorithm model for detecting and identifying vehicles in an image frame. The preset vehicle feature extraction model may refer to a pre-trained algorithm model for extracting vehicle attribute features. The specific algorithms of the preset vehicle detection model and the preset vehicle feature extraction model may be selected based on actual needs, and the embodiments of the present application do not limit this. The vehicle image area may refer to the image area in the image frame that includes the vehicle image.

[0098] Specifically, after acquiring the video stream uploaded by the shooting device, the electronic device can perform image analysis on each video stream. For each image frame of the video stream, the electronic device can first determine the vehicle image area in the image frame based on the preset vehicle detection model, and specifically can calibrate the vehicle image area in the form of an image identification frame; then the electronic device can perform feature recognition and extraction on the vehicle image area based on the preset vehicle feature extraction model to obtain vehicle image information corresponding to the vehicle graphic area, which can specifically include chassis image coordinates and vehicle attribute information, etc. In addition, the confidence of each feature can also be obtained, such as the confidence of chassis image coordinates and the confidence of vehicle attribute information, etc. In this way, the electronic device can obtain the vehicle image information corresponding to each image frame by analyzing and identifying the video stream frame by frame, which can realize the accurate extraction of vehicle image information and improve the accuracy of subsequent vehicle merge recognition.

[0099] For example, Figure 4 A schematic diagram of the logic of extracting vehicle image information provided by an exemplary embodiment of the present application. Figure 4 As shown, the electronic device obtains a video stream reported by at least one shooting device, and then performs frame-by-frame recognition and analysis on the video stream. First, it can perform vehicle recognition detection on the image frame based on a preset vehicle detection model to determine the vehicle image area in the image frame; then the electronic device can detect and identify the vehicle image area based on a preset vehicle feature extraction model to determine the vehicle image information corresponding to the vehicle image area, thereby achieving accurate extraction of vehicle image information.

[0100] S303, extracting pedestrian trajectory information from each video stream; and determining a target interaction event corresponding to the vehicle image information according to the pedestrian trajectory information and the vehicle image information.

[0101] In the embodiment of the present application, pedestrian trajectory information may refer to the movement trajectory (track) of pedestrians (such as customers or consumers) in the video stream, and the pedestrian trajectory information may include a series of trajectory points, each of which may include the coordinate information and time information of the trajectory point. Target interaction events may refer to interaction events between pedestrians and vehicles, such as vehicle browsing events and vehicle entry and exit events.

[0102] In this step, after acquiring the video stream, the electronic device can identify the pedestrian trajectory information corresponding to each pedestrian based on the preset pedestrian tracking detection algorithm, and then determine whether the vehicle image information has a target interaction event based on the pedestrian trajectory information and the vehicle image information. Specifically, the electronic device can determine that there is a vehicle browsing event in the vehicle image information based on the trajectory point coordinate information in the pedestrian trajectory information and the chassis image coordinates in the vehicle image information, when the distance between the trajectory point coordinate information and the chassis image coordinates is less than the distance threshold and the residence time of the pedestrian trajectory information is greater than the time threshold; when there is a cross-line behavior between the pedestrian trajectory information and the vehicle chassis image coordinates, for example, when the pedestrian trajectory information and the vehicle chassis image coordinates intersect, the electronic device can determine that there is a vehicle entry and exit event in the vehicle image information. Of course, the target interaction event can also include other events, and other determination methods can also be used, which is not limited in the embodiments of the present application.

[0103] S304. Determine whether the vehicle image information meets a preset data condition according to the target interaction event; if the vehicle image information does not meet the preset data condition, delete the vehicle image information.

[0104] In an embodiment of the present application, the preset data condition may refer to a data screening condition for vehicle image information. Since the electronic device may extract a large amount of vehicle image information from the video stream sent by at least one camera, in order to reduce the amount of data calculation, the electronic device may screen and filter the vehicle image information, delete the vehicle image information that does not meet the preset data condition, and implement data cleaning of the vehicle image information, which can reduce the amount of calculation and improve the calculation efficiency of vehicle merging and identification. The preset data condition may refer to whether the confidence of the chassis image coordinates is greater than or equal to the target confidence threshold, etc., which can be flexibly set based on actual needs, and the embodiment of the present application does not limit this.

[0105] In a possible implementation, the vehicle image information further includes vehicle integrity information and chassis image coordinate confidence; determining whether the vehicle image information satisfies the preset data condition in step 305 may be implemented by the following steps (1) to (4):

[0106] (1) If the number of coordinates with missing data in the chassis image coordinates is greater than a preset number threshold, it is determined that the vehicle image information does not meet the preset data condition.

[0107] In the embodiment of the present application, the data missing coordinates may refer to the chassis image coordinates whose coordinate values ​​are the default values ​​(or default values). Since the shooting angle of the shooting device may vary, the shooting device may only capture a portion of the vehicle in some image frames in the video stream. For the incomplete vehicle image in the image frame, the electronic device usually uses the missing value to fill the chassis image coordinates corresponding to the chassis frame that has not been captured when determining the chassis image coordinates. At this time, the data missing coordinates can be expressed as (0.0, 0.0). The preset quantity threshold may refer to a preset data critical value when the data missing coordinates in the chassis image coordinates are missing. The preset quantity threshold may be 1, etc. If the number of data missing coordinates in the chassis image coordinates of the vehicle image information is greater than the preset quantity threshold, for example, the number of data missing coordinates is 2, which is greater than the preset quantity threshold 1, then the electronic device can determine that the vehicle image information is not completely captured and does not meet the preset data condition. In this way, the electronic device can filter out the vehicle image information that is not completely captured by the vehicle by comparing the number of data missing coordinates in the chassis image coordinates with the preset quantity threshold, and can reduce the amount of invalid data.

[0108] (2) If the average pixel length of the chassis frame corresponding to the chassis image coordinates is less than a preset length threshold, it is determined that the vehicle image information does not meet the preset data condition.

[0109] In the embodiment of the present application, the chassis frame pixel length may refer to the pixel length of the four sides corresponding to the chassis image coordinates. The preset length threshold may refer to a preset pixel length critical value, for example, the preset length threshold may be 200 pixels.

[0110] Specifically, in offline sales scenarios, when photographing a vehicle, the photographing device may be too far away, resulting in a small vehicle image. A small vehicle image can easily lead to deviations in the calculation of the actual position of the vehicle, affecting the accuracy of subsequent vehicle merging and identification. On this basis, when the average value of the chassis frame pixel length corresponding to the chassis image coordinates is less than the preset length threshold, the electronic device can determine that the vehicle image information does not meet the preset data conditions. Of course, the electronic device compares the pixel lengths of the four sides of the chassis frame corresponding to the chassis image coordinates with the preset length threshold respectively. If the number of chassis frame sides less than the preset length threshold is greater than 2, the electronic device can determine that the vehicle image information does not meet the preset data conditions. In this way, the electronic device can filter out the vehicle image information that is small due to the long shooting distance by comparing and judging the average value of the chassis frame pixel length corresponding to the chassis image coordinates with the preset length threshold, which can improve the accuracy of subsequent vehicle merging and identification.

[0111] (3) If the average value of the chassis image coordinate confidence is less than the target confidence threshold, or there is a chassis image coordinate confidence that is less than a preset minimum confidence threshold, it is determined that the vehicle image information does not meet the preset data condition; the target confidence threshold is determined based on the target number of interaction events and the first mapping relationship; the first mapping relationship is a mapping relationship between the target number of interaction events and the preset confidence threshold.

[0112] In the embodiment of the present application, the vehicle image information may also include vehicle integrity information and chassis image coordinate confidence. The vehicle integrity information is used to characterize the integrity of the vehicle, and may be detected and identified by a preset vehicle feature extraction model based on the vehicle image area and directly output; the chassis image coordinate confidence may refer to the confidence corresponding to the four chassis image coordinates of the vehicle chassis frame, and the chassis image coordinate confidence may also be synchronously output by the preset vehicle feature extraction model when determining the chassis image coordinates. Of course, the vehicle integrity information and chassis image coordinate confidence may also be determined by other methods, and the embodiment of the present application does not limit this.

[0113] The target confidence threshold may refer to the critical value of the confidence determined by the electronic device, and the target confidence threshold may be determined according to the number of target interaction events and the first mapping relationship, wherein the number of target interaction events may refer to the total number of target interaction events corresponding to the vehicle image information; the first mapping relationship may refer to the mapping relationship between the number of target interaction events and the preset confidence threshold. Since in actual scenarios, the data reference value of vehicles with target interaction events is higher, in the first mapping relationship, the larger the number of target interaction events, the lower the preset confidence threshold corresponding to the number of target interaction events. Exemplarily, when the target interaction event is 0, the preset confidence threshold may be 0.8; when the target interaction event is 1, the preset confidence threshold may be 0.7; when the target interaction event is 2, the preset confidence threshold may be 0.6. Of course, the specific numerical values ​​in the first mapping relationship may also be other numerical values, which may be flexibly set based on actual needs, and the embodiments of the present application are not limited to this. The preset confidence minimum threshold may refer to the preset minimum confidence value, for example, 0.2, 0.3, etc.

[0114] Specifically, the electronic device can determine whether the vehicle image information meets the preset data condition according to the chassis image coordinate confidence. When there is a chassis image confidence less than the preset minimum confidence threshold in the chassis image confidence of the vehicle image information, the electronic device can determine that there are unreliable chassis image coordinates in the vehicle image information, and the vehicle image information does not meet the preset data condition. Alternatively, the electronic device can determine the target confidence threshold corresponding to the vehicle image information according to the number of target interaction events, and if the average value of the chassis image coordinate confidence is less than the target confidence threshold, the electronic device can determine that the vehicle image information does not meet the preset data condition. In this way, the electronic device can filter out the vehicle image information with low chassis image coordinate confidence by comparing the chassis image coordinate confidence with the preset minimum confidence threshold, or comparing the average value of the chassis image coordinate confidence with the target confidence threshold, so as to avoid the negative impact of invalid data on vehicle merge recognition and improve calculation efficiency and recognition accuracy.

[0115] (4) If the vehicle integrity information is less than the target integrity threshold, it is determined that the vehicle image information does not meet the preset data condition; the target integrity threshold is determined based on the target number of interaction events and a second mapping relationship; the second mapping relationship is a mapping relationship between the target number of interaction events and the preset integrity threshold.

[0116] In an embodiment of the present application, the target completeness threshold may be a critical value of vehicle completeness determined by an electronic device. The target completeness threshold may be determined based on the number of target interaction events and the second mapping relationship. Since in actual scenarios, the reference value of vehicle image information with target interaction events is higher, in the second mapping relationship, the larger the number of target interaction events, the lower the preset completeness threshold corresponding to the number of target interaction events. Exemplarily, when the number of target interaction events is 0, the preset completeness threshold may be 0.8; when the number of target interaction events is 1, the preset completeness threshold may be 0.7; when the number of target interaction events is 2, the preset completeness threshold may be 0.6. Of course, the specific numerical values ​​in the second mapping relationship may also be other numerical values, which may be flexibly set based on actual needs, and the embodiment of the present application does not limit this.

[0117] In this step, the electronic device can determine the target completeness threshold corresponding to the vehicle image information according to the number of target interaction events corresponding to the vehicle image information and the second mapping relationship; then the electronic device can compare and match the vehicle completeness information in the vehicle image information with the target completeness threshold. If the vehicle completeness information is less than the target completeness threshold, the electronic device can determine that the vehicle captured in the vehicle image information is incomplete, and the vehicle image information does not meet the preset data conditions and can be deleted. In this way, the electronic device can filter out the incomplete vehicle image information of the vehicle by comparing and matching the vehicle completeness information with the target completeness threshold, avoid the calculation of invalid data, and save computing resources.

[0118] It should be noted that in addition to the above-mentioned preset data conditions, the electronic device can also filter the vehicle image information through other preset data conditions. For example, the number of vehicle image information captured by the same camera at the same position in the video stream is less than 2. In offline sales scenarios, there may be a situation where the exhibition vehicle moves. At this time, the camera may only capture the vehicle once during the timed sampling process. At this time, the exhibition time of the vehicle image information in the actual scene is short, and no subsequent merging and identification is required. The embodiments of the present application do not limit the specific types of preset data conditions.

[0119] For example, Figure 5 A schematic diagram of vehicle image information provided by an exemplary embodiment of the present application. Figure 5 As shown, the vehicle image information includes chassis image coordinates, vehicle attribute information, vehicle integrity, and the confidence level of each information. The vehicle attribute information includes vehicle brand, vehicle model, and vehicle color, as well as the camera identification and the shooting time. Figure 5 In the chassis image coordinates, there are two chassis image coordinate confidences that are less than the preset minimum confidence threshold (0.2). The vehicle image information corresponding to the vehicle is incomplete, and the vehicle image information does not meet the preset data conditions. The electronic device can delete the vehicle image information. It should be noted that the vehicle image information may also include other information, which is not limited in the present embodiment of the application. In addition, for Figure 5 For the chassis frame of the vehicle that cannot be photographed, the electronic device can also directly use the default value to represent it, setting the coordinates to (0.0, 0.0). Figure 5 There will be two data missing coordinates in the chassis image coordinates, and the vehicle image information does not meet the preset data conditions and needs to be deleted.

[0120] S305: Determine the actual world coordinates corresponding to the chassis image coordinates, and determine the target chassis graphics corresponding to the vehicle image information based on the actual world coordinates.

[0121] In the embodiment of the present application, the chassis image coordinates in the vehicle image information belong to the pixel coordinates. In order to determine the actual position of the vehicle, the electronic device can convert the chassis image coordinates, which are pixel coordinates, into actual world coordinates. Specifically, the electronic device can determine the homography matrix between the world coordinate system and the pixel coordinate system, and then perform coordinate mapping conversion based on the homography matrix to convert the chassis image coordinates into actual world coordinates. The electronic device can specifically calculate the homography matrix based on the shooting parameters of the shooting device, the world coordinates of the calibration points, and the pixel coordinates of the calibration points. Of course, other calculation methods can also be used, and the embodiment of the present application does not limit this. Afterwards, the electronic device can connect the actual world coordinates corresponding to the vehicle image information by connecting the coordinates, and connect the actual world coordinates of the four points of the vehicle chassis frame to form a quadrilateral, so as to obtain the target chassis graphic corresponding to the vehicle image information, and realize accurate representation of the actual position of the vehicle.

[0122] S306. For any two vehicle image information, if the two target chassis graphics corresponding to the two vehicle image information and the two vehicle attribute information meet the target matching condition, it is determined that the two vehicle image information correspond to the same target vehicle, and the two vehicle image information are added to the target group corresponding to the target vehicle.

[0123] In a possible implementation, whether the two target chassis graphics and the two vehicle attribute information corresponding to the two vehicle image information meet the target matching condition can be determined by the following steps (5) to (6):

[0124] (5) For any two vehicle image information, determine the target ratio threshold condition corresponding to the two vehicle image information according to the matching result of the two vehicle attribute information and the third mapping relationship; the third mapping relationship includes a mapping relationship between the matching result of the vehicle attribute information and the preset ratio threshold condition.

[0125] In the embodiment of the present application, the target ratio threshold condition may refer to the ratio threshold condition that the electronic device must meet when matching two vehicle image information. The target ratio threshold condition may be determined based on the matching result of the vehicle attribute information and the third mapping relationship. The third mapping relationship includes a mapping relationship between different matching results of the vehicle attribute information and different preset ratio threshold conditions. In the third mapping relationship, the higher the matching degree of the vehicle attribute information, the lower the preset ratio threshold condition. Exemplarily, if all vehicle attribute information in the two vehicle image information does not match, the preset ratio threshold condition may be that the ratio of the intersection area to the area of ​​the two target chassis graphics is greater than 0.8; if the vehicle model matches and the vehicle color does not match in the two vehicle image information, the preset ratio threshold condition may be that the ratio of the intersection area to the area of ​​the two target chassis graphics is greater than 0.7; if the vehicle model matches and the vehicle color matches in the two vehicle image information, the preset ratio threshold condition may be that the ratio of the intersection area to the area of ​​the two target chassis graphics is greater than 0.5. Of course, the third mapping relationship may also include other matching results and preset ratio threshold conditions, which are not limited in the embodiment of the present application.

[0126] In addition, the electronic device can also add a judgment condition that the sum of the ratios of the intersection area to the area of ​​the two target chassis graphics is greater than the preset ratio threshold in the preset ratio threshold condition, so as to prevent the situation where the camera distortion causes obvious coordinate differences and large differences in the area of ​​the vehicle target chassis graphics when the vehicle is at different distances from the camera, thereby improving the accuracy of vehicle merging; the electronic device can also add more matching results of vehicle attribute information in the third mapping relationship, such as color mismatch, which can further distinguish between mutually exclusive colors (such as black and white, etc.) and non-mutually exclusive colors, whether the shooting device identification corresponds to the same camera, whether the shooting time is the same time, etc., and corresponding preset ratio threshold conditions are set respectively, which can further improve the rationality and accuracy of vehicle merging. Of course, the above-mentioned preset ratio threshold conditions can be obtained based on the training of the vehicle merging recognition model, and the embodiments of the present application are not limited to this.

[0127] (6) If the intersection area of ​​the two target chassis graphics corresponding to the two vehicle image information meets the target ratio threshold condition, it is determined that the two target chassis graphics corresponding to the two vehicle image information and the two vehicle attribute information meet the target matching condition.

[0128] In an embodiment of the present application, after the electronic device determines the target ratio threshold condition based on the matching result of the vehicle attribute information and the third mapping relationship, it can further determine whether the intersection area of ​​the two target chassis graphics corresponding to the two vehicle image information meets the target ratio threshold condition. If so, the electronic device can determine that the two target chassis graphics corresponding to the two vehicle image information and the two vehicle attribute information meet the target matching condition, and can determine that the two vehicle image information correspond to the same target vehicle, and can add the two vehicle image information to the same target group to achieve accurate and reasonable merging of the vehicle image information.

[0129] For example, Figure 6 The following is a schematic diagram of determining the intersection area of ​​a target chassis pattern provided by an exemplary embodiment of the present application. Figure 6 As shown, the two target chassis graphics corresponding to the two vehicle image information are target chassis graphics 601 and target chassis graphics 602. The intersection of the target chassis graphics 601 and the target chassis graphics 602 is Figure 6 The black filled area in the image is the intersection area. The electronic device can determine the target ratio threshold condition based on the matching result of the two vehicle attribute information corresponding to the two vehicle image information and the third mapping relationship. If the intersection area meets the target ratio threshold condition, for example, the ratio of the intersection area to the target chassis graphic 601 is greater than 0.8 and the ratio of the intersection area to the target chassis graphic 602 is greater than 0.8, then the electronic device can determine that the two target chassis graphics corresponding to the two vehicle image information and the two vehicle attribute information meet the target matching condition, and the two vehicle image information correspond to the same target vehicle and can be merged into the same target group to achieve accurate vehicle merging processing.

[0130] S307. Determine a target quality score corresponding to the vehicle image information in the target group; the target quality score is determined according to the confidence of the chassis image coordinates and the confidence of the vehicle attribute information.

[0131] In the embodiment of the present application, the target quality score may refer to the quality score of the vehicle image information as a whole determined by the electronic device according to the confidence of the chassis image coordinates and the confidence of the vehicle attribute information, and the target quality score may reflect the credibility of the vehicle image information as a whole. The electronic device may determine the target quality score of each vehicle image information in the target group, and may subsequently determine the target attribute information of the target vehicle based on the target quality score.

[0132] S308. Group the vehicle image information according to different vehicle attribute information to obtain target sub-groups; the vehicle attribute information in each target sub-group is the same; the vehicle attribute information includes at least one of a vehicle brand, a vehicle model, and a vehicle color.

[0133] S309: Determine the vehicle attribute information corresponding to the target subgroup with the largest average target quality score as the target attribute information corresponding to the target vehicle.

[0134] In the embodiment of the present application, the target subgroup may refer to a subgroup obtained after secondary grouping of the vehicle attribute information based on the vehicle image information. The electronic device may group the vehicle image information in the target group according to the different vehicle attribute information to obtain multiple target subgroups, and then use the vehicle attribute information in the target subgroup with the largest average target quality score as the target attribute information of the target vehicle. Exemplarily, the electronic device may group the vehicle image information according to the vehicle model to obtain multiple target subgroups, and the vehicle model in each target subgroup is the same, and then the electronic device may use the vehicle signal corresponding to the target subgroup with the largest average target quality score as the vehicle model in the target attribute information; the electronic device may also group the vehicle image information according to the vehicle color to obtain multiple target subgroups, and the vehicle color in each target subgroup is the same, and then the electronic device may use the vehicle signal corresponding to the target subgroup with the largest average target quality score as the vehicle model in the target attribute information. Of course, the electronic device may also use the vehicle attribute information of the target subgroup with the largest sum of target quality scores as the target attribute information, and the embodiment of the present application is not limited to this.

[0135] In this way, the electronic device can improve the accuracy and rationality of target attribute information recognition by re-grouping the vehicle image information in the same target group according to the vehicle attribute information to obtain multiple target sub-groups, and then use the vehicle attribute information of the target sub-group with the highest average target quality score as the target attribute information of the target vehicle. At the same time, there is no need to compare and identify with the data base, which can reduce data maintenance costs.

[0136] It should be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0137] Based on the above embodiments, Figure 7 A schematic diagram of a vehicle merging and identification system architecture provided for an exemplary embodiment of the present application. Figure 7As shown, the system architecture of vehicle merge identification can include cameras, edge servers, network management clusters, message queue (MQ) middleware clusters, cloud microservice clusters, databases, merge identification servers, and offline data computing platforms. Among them, 0.0 means that the shooting device (i.e., camera) is connected to the IoT platform through triple login registration to build a communication channel. Among them, the triple can refer to the unique identifier of a device in the IoT, usually including three elements: device ID, product ID, and device secret. The device ID is the unique identifier of the device, the product ID is the unique identifier of the product to which the device belongs, and the device secret is the key for encrypted communication between the device and the server. Through the device triple, the IoT platform can accurately identify and manage each device and ensure the security and reliability of communication between devices.

[0138] 0.1 means that the cloud microservice application cluster can send configuration information through the Message Queuing Telemetry Transport (MQTT) or Hypertext Transfer Protocol (HTTP) channel. 0.2 means that the edge computing server can report data to the server through the MQTT or HTTP channel, such as MQTT reporting data (such as vehicle image information), HTTP uploading pictures, etc. 0.3 means that after the gateway receives the request, it can forward it to the cloud microservice cluster for processing.

[0139] 1.1 indicates that the edge server pulls the stream from the camera for calculation and processing, calls the algorithm model to extract the vehicle image information and pedestrian trajectory information, and determines the target interaction event. 1.2 indicates that the edge server reports various data through the MQTT channel. 1.3 indicates that the service is asynchronously buffered through the MQ middleware and forwarded to the cloud microservice cluster for processing. 1.4, 1.5, and 1.6 indicate that the cloud microservice cluster processes the data and asynchronously stores the sampled data for merging and recognition processing. 1.7 and 1.8 indicate that the merging and recognition server pulls the vehicle image information, performs data filtering and merging and recognition processing, and persists the final data into the database. 1.9 and 1.10 indicate that the offline data computing cluster performs some data analysis and synchronizes the results to the database. Of course, it should be noted that the system architecture of the vehicle merging and recognition may also include other devices or adopt other structures, which is not limited to this in the embodiments of the present application.

[0140] In the embodiment of the present application, the electronic device does not rely on hardware devices such as laser radar to realize position judgment. Based on the coordinate mapping conversion, the target chassis graphic corresponding to the vehicle image information can be determined, so as to realize accurate judgment of the actual position of the vehicle; and the electronic device performs a comprehensive judgment of multiple features based on the vehicle attribute information and the matching of the target chassis graphic, with high accuracy; in addition, there is no need to compare and identify with the data base, which reduces the data maintenance cost, and can also avoid misjudgment and misidentification caused by new data such as new cars, and improve the accuracy of vehicle merging and identification; in addition, the electronic device can perform vehicle merging and identification for the video streams of one or more shooting devices, avoiding the data limitations of a single mirror, and can realize vehicle data merging and identification under multiple mirrors, thereby improving the comprehensiveness of passenger flow data analysis and statistics.

[0141] Figure 8 For a structural diagram of a vehicle merging and identifying device provided by an exemplary embodiment of the present application, see Figure 8 The vehicle merging and identifying device 80 comprises:

[0142] The acquisition module 81 is used to acquire the video stream sent by at least one shooting device and extract vehicle image information from each video stream; the vehicle image information includes chassis image coordinates and vehicle attribute information;

[0143] A first determination module 82, used to determine the actual world coordinates corresponding to the chassis image coordinates, and determine the target chassis graphics corresponding to the vehicle image information based on the actual world coordinates;

[0144] A merging module 83 is used for determining that the two vehicle image information correspond to the same target vehicle if the two target chassis graphics and two vehicle attribute information corresponding to the two vehicle image information meet the target matching condition, and adding the two vehicle image information to the target group corresponding to the target vehicle;

[0145] The second determination module 84 is used to determine the target attribute information corresponding to the target vehicle according to the vehicle image information in the target group.

[0146] In a possible implementation, the acquisition module 81 is specifically configured to:

[0147] For each image frame of the video stream, a vehicle image region in the image frame is determined according to a preset vehicle detection model;

[0148] The vehicle image area is identified and processed according to a preset vehicle feature extraction model to obtain vehicle image information corresponding to the vehicle image area.

[0149] In a possible implementation manner, the device 80 is further used for:

[0150] Extract pedestrian trajectory information from each video stream;

[0151] Determine a target interaction event corresponding to the vehicle image information according to the pedestrian trajectory information and the vehicle image information;

[0152] According to the target interaction event, determine whether the vehicle image information meets the preset data conditions;

[0153] When the vehicle image information does not meet the preset data condition, the vehicle image information is deleted.

[0154] In a possible implementation manner, the vehicle image information further includes vehicle integrity information and chassis image coordinate confidence; the device 80 is further used for:

[0155] If the number of coordinates with missing data in the chassis image coordinates is greater than a preset number threshold, it is determined that the vehicle image information does not meet the preset data condition;

[0156] If the average pixel length of the chassis frame corresponding to the chassis image coordinates is less than the preset length threshold, it is determined that the vehicle image information does not meet the preset data condition;

[0157] If the average value of the chassis image coordinate confidence is less than the target confidence threshold, or there is a chassis image coordinate confidence that is less than a preset confidence minimum threshold, it is determined that the vehicle image information does not meet the preset data condition; the target confidence threshold is determined according to the target interaction event number and the first mapping relationship; the first mapping relationship is a mapping relationship between the target interaction event number and the preset confidence threshold;

[0158] If the vehicle integrity information is less than the target integrity threshold, it is determined that the vehicle image information does not meet the preset data conditions; the target integrity threshold is determined based on the target number of interaction events and the second mapping relationship; the second mapping relationship is the mapping relationship between the target number of interaction events and the preset integrity threshold.

[0159] In a possible implementation manner, the device 80 is further used for:

[0160] Determine the target ratio threshold condition corresponding to the two vehicle image information according to the matching result of the two vehicle attribute information and the third mapping relationship; the third mapping relationship includes a mapping relationship between the matching result of the vehicle attribute information and the preset ratio threshold condition;

[0161] If the intersection area of ​​the two target chassis graphics corresponding to the two vehicle image information meets the target ratio threshold condition, it is determined that the two target chassis graphics corresponding to the two vehicle image information and the two vehicle attribute information meet the target matching condition.

[0162] In a possible implementation manner, the second determining module 84 is specifically configured to:

[0163] For the vehicle image information in the target group, determine the target quality score corresponding to the vehicle image information; the target quality score is determined according to the confidence of the chassis image coordinates and the confidence of the vehicle attribute information;

[0164] The vehicle image information is grouped according to different vehicle attribute information to obtain target subgroups; the vehicle attribute information in each target subgroup is the same; the vehicle attribute information includes at least one of a vehicle brand, a vehicle model, and a vehicle color;

[0165] The vehicle attribute information corresponding to the target subgroup with the largest average value of the target quality score is determined as the target attribute information corresponding to the target vehicle.

[0166] The vehicle merging and identification device 80 provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar and will not be repeated here.

[0167] Fig. 9 For a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application, see Fig. 9 The electronic device 90 may include a processor 91 and a memory 92. Exemplarily, the processor 91 and the memory 92 are interconnected via a bus 93.

[0168] Memory 92 stores computer executable instructions;

[0169] The processor 91 executes the computer-executable instructions stored in the memory 92 , so that the processor 91 executes the vehicle merging and identification method as shown in the above method embodiment.

[0170] Accordingly, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the vehicle merging and identification method of the above method embodiment.

[0171] Accordingly, an embodiment of the present application may also provide a computer program product, including a computer program, which, when executed by a processor, can implement the vehicle merging and identification method shown in the above method embodiment.

[0172] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0173] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0174] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0175] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0176] In a typical configuration, a computing device includes one or more processors, input / output interfaces, network interfaces, and memory.

[0177] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0178] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0179] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0180] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A vehicle merging and identification method, characterized in that: include: Obtaining a video stream sent by at least one shooting device, and extracting vehicle image information from each of the video streams; The vehicle image information includes chassis image coordinates and vehicle attribute information; Determine the actual world coordinates corresponding to the chassis image coordinates, and determine the target chassis graphics corresponding to the vehicle image information based on the actual world coordinates; For any two vehicle image information, if the two target chassis graphics and two vehicle attribute information corresponding to the two vehicle image information meet the target matching condition, it is determined that the two vehicle image information correspond to the same target vehicle, and the two vehicle image information are added to the target group corresponding to the target vehicle; Determining target attribute information corresponding to the target vehicle according to the vehicle image information in the target group; The method further comprises: Determine the target ratio threshold condition corresponding to the two vehicle image information according to the matching result of the two vehicle attribute information and the third mapping relationship; the third mapping relationship includes a mapping relationship between the matching result of the vehicle attribute information and the preset ratio threshold condition; If the intersection area of ​​the two target chassis graphics corresponding to the two vehicle image information meets the target ratio threshold condition, it is determined that the two target chassis graphics corresponding to the two vehicle image information and the two vehicle attribute information meet the target matching condition.

2. The method according to claim 1, characterized in that The extracting vehicle image information from each of the video streams comprises: For each image frame of the video stream, determining a vehicle image region in the image frame according to a preset vehicle detection model; The vehicle image area is identified and processed according to a preset vehicle feature extraction model to obtain vehicle image information corresponding to the vehicle image area.

3. The method according to claim 1, characterized in that: The method further comprises: Extracting pedestrian trajectory information from each of the video streams; Determining a target interaction event corresponding to the vehicle image information according to the pedestrian trajectory information and the vehicle image information; Determining whether the vehicle image information meets a preset data condition according to the target interaction event; If the vehicle image information does not meet the preset data condition, the vehicle image information is deleted.

4. The method according to claim 3, characterized in that The vehicle image information also includes vehicle integrity information and chassis image coordinate confidence; and determining whether the vehicle image information meets a preset data condition according to the target interaction event includes: If the number of data missing coordinates in the chassis image coordinates is greater than a preset number threshold, it is determined that the vehicle image information does not meet the preset data condition; If the average chassis frame pixel length corresponding to the chassis image coordinates is less than a preset length threshold, it is determined that the vehicle image information does not meet the preset data condition; If the average value of the chassis image coordinate confidence is less than the target confidence threshold, or there is a chassis image coordinate confidence that is less than a preset minimum confidence threshold, it is determined that the vehicle image information does not meet the preset data condition; the target confidence threshold is determined according to the target interaction event number and a first mapping relationship; the first mapping relationship is a mapping relationship between the target interaction event number and a preset confidence threshold; If the vehicle integrity information is less than the target integrity threshold, it is determined that the vehicle image information does not meet the preset data conditions; the target integrity threshold is determined based on the target number of interaction events and a second mapping relationship; the second mapping relationship is a mapping relationship between the target number of interaction events and a preset integrity threshold.

5. The method according to any one of claims 1 to 4, characterized in that: The step of determining target attribute information corresponding to the target vehicle according to the vehicle image information in the target group includes: For the vehicle image information in the target group, determining a target quality score corresponding to the vehicle image information; the target quality score is determined according to the confidence of the chassis image coordinates and the confidence of the vehicle attribute information; The vehicle image information is grouped according to different vehicle attribute information to obtain target subgroups; the vehicle attribute information in each target subgroup is the same; the vehicle attribute information includes at least one of a vehicle brand, a vehicle model, and a vehicle color; The vehicle attribute information corresponding to the target subgroup with the largest average value of the target quality score is determined as the target attribute information corresponding to the target vehicle.

6. A vehicle merging and identification device, characterized in that: include: An acquisition module, used to acquire a video stream sent by at least one shooting device, and extract vehicle image information from each of the video streams; The vehicle image information includes chassis image coordinates and vehicle attribute information; A first determination module, configured to determine the actual world coordinates corresponding to the chassis image coordinates, and determine the target chassis graphic corresponding to the vehicle image information based on the actual world coordinates; A merging module, for any two vehicle image information, if the two target chassis graphics and two vehicle attribute information corresponding to the two vehicle image information meet the target matching condition, then determine that the two vehicle image information correspond to the same target vehicle, and add the two vehicle image information to the target group corresponding to the target vehicle; A second determination module is used to determine target attribute information corresponding to the target vehicle according to the vehicle image information in the target group; The device is also used for: Determine the target ratio threshold condition corresponding to the two vehicle image information according to the matching result of the two vehicle attribute information and the third mapping relationship; the third mapping relationship includes a mapping relationship between the matching result of the vehicle attribute information and the preset ratio threshold condition; If the intersection area of ​​the two target chassis graphics corresponding to the two vehicle image information meets the target ratio threshold condition, it is determined that the two target chassis graphics corresponding to the two vehicle image information and the two vehicle attribute information meet the target matching condition.

7. An electronic device, characterized in that: include: Memory and processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the vehicle merging and identification method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the vehicle merging and identification method according to any one of claims 1 to 5.

9. A computer program product, characterized in that It comprises a computer program, which, when executed by a computer, implements the vehicle merging and identification method as described in any one of claims 1 to 5.