Re-identification method and device, computer device and storage medium

Abstract

The application relates to a re-identification method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a point cloud mapping image of a target object and a monitoring image of the target object; the point cloud mapping image is obtained by mapping three-dimensional point cloud data of the target object collected by a laser radar; the monitoring image is obtained by a monitoring camera; feature extraction is respectively performed on the point cloud mapping image and the monitoring image to obtain a mapping feature map corresponding to the point cloud mapping image and a monitoring feature map corresponding to the monitoring image; and re-identification processing is performed on the target object based on the mapping feature map and the monitoring feature map. The method can improve the accuracy of re-identification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of re-identification, and in particular to a re-identification method and device, a computer device and a storage medium. BACKGROUND

[0002] With the development of image processing technology, target re-identification technology has emerged. Target re-identification, also known as target re-identification, is currently focused on two fields of pedestrian re-identification and vehicle re-identification, and is mainly used to solve target cross-camera and cross-field tracking. For example, vehicle re-identification is to find the same vehicle from vehicle images collected by different cameras.

[0003] However, vehicle images may vary greatly with the distance between the vehicle and the monitoring camera, resulting in the vehicle not being the same vehicle, i.e. low re-identification accuracy. SUMMARY

[0004] Therefore, it is necessary to provide a re-identification method, device, computer device and storage medium capable of improving re-identification accuracy to solve the above technical problems.

[0005] A re-identification method, the method comprising:

[0006] obtaining a point cloud mapping image of a target object and a monitoring image of the target object; the point cloud mapping image is mapped from three-dimensional point cloud data of the target object collected by a laser radar; the monitoring image is obtained by a monitoring camera;

[0007] respectively extracting features of the point cloud mapping image and the monitoring image to obtain a mapping feature map corresponding to the point cloud mapping image and a monitoring feature map corresponding to the monitoring image;

[0008] based on the mapping feature map and the monitoring feature map, performing re-identification processing on the target object.

[0009] In one embodiment, the above-mentioned obtaining a point cloud mapping image of a target object comprises:

[0010] obtaining three-dimensional point cloud data obtained by scanning a target scene by a laser radar; the target object is located in the target scene;

[0011] mapping the three-dimensional point cloud data to obtain a two-dimensional image;

[0012] determining a three-dimensional recognition region in which the target object is located in the three-dimensional point cloud data by using a target detection algorithm;

[0013] mapping the three-dimensional recognition region to the two-dimensional image to obtain a first recognition region in which the target object is located in the two-dimensional image, and taking an image in the first recognition region as the point cloud mapping image.

[0014] In one of the embodiments, the mapping of the three-dimensional point cloud data to obtain the two-dimensional image comprises:

[0015] According to the preset mapping relationship, mapping the point cloud points in the preset range in the three-dimensional point cloud data from the three-dimensional coordinate system to the two-dimensional coordinate system;

[0016] According to the position of each point cloud point in the two-dimensional coordinate system and the height data of each point cloud point, obtaining the bird's eye view;

[0017] According to the position of each point cloud point in the two-dimensional coordinate system and the intensity data of each point cloud point, obtaining the intensity map;

[0018] According to the density of the point cloud points in the bird's eye view, obtaining the density map;

[0019] Merging the bird's eye view, the intensity map and the density map to obtain the two-dimensional image.

[0020] In one of the embodiments, the bird's eye view is obtained according to the position of each point cloud point in the two-dimensional coordinate system and the height data of each point cloud point, comprising:

[0021] According to the height data of each point cloud point, determining the pixel value corresponding to each point cloud point;

[0022] According to the position of each point cloud point in the two-dimensional coordinate system and the pixel value corresponding to each point cloud point, constructing the bird's eye view.

[0023] In one of the embodiments, the intensity map is obtained according to the position of each point cloud point in the two-dimensional coordinate system and the intensity data of each point cloud point, comprising:

[0024] According to the intensity data of each point cloud point, determining the pixel value corresponding to each point cloud point;

[0025] According to the position of each point cloud point in the two-dimensional coordinate system and the pixel value corresponding to each point cloud point, constructing the intensity map.

[0026] In one of the embodiments, the density map is obtained according to the density of the point cloud points in the bird's eye view, comprising:

[0027] According to the arrangement of the pixel points in the bird's eye view, determining the arrangement of the pixel points in the density map;

[0028] According to the number of point clouds of the i-th pixel point in the bird's eye view, the minimum value of the number of point clouds in each pixel point and the maximum value of the number of point clouds in each pixel point, calculating the density of the i-th pixel point; i is a positive integer;

[0029] According to the calculated density of each pixel point, determining the pixel value of each pixel point;

[0030] According to the arrangement of the pixel points in the density map and the pixel values of the pixel points, the density map is constructed.

[0031] In one of the embodiments, the merging of the bird's eye view, the intensity map and the density map to obtain the point cloud mapping image comprises:

[0032] The bird's eye view, the intensity map and the density map are merged as images of R, G and B channels respectively to obtain a two-dimensional image.

[0033] In one of the embodiments, the obtaining of the monitoring image of the target object comprises:

[0034] The photographing image obtained by photographing the target scene by the monitoring camera is obtained.

[0035] According to the correspondence between the radar coordinates and the camera coordinates set in advance, the stereoscopic recognition region is mapped to the photographing image to obtain a second recognition region in the photographing image where the target object is located, and the image in the second recognition region is taken as the monitoring image.

[0036] In one of the embodiments, the feature extraction from the point cloud mapping image and the monitoring image respectively to obtain the mapping feature map corresponding to the point cloud mapping image and the camera feature map corresponding to the camera image comprises:

[0037] The point cloud mapping image and the monitoring image are respectively input into the pre-trained feature extraction network to obtain the mapping feature map and the monitoring feature map output by the feature extraction network.

[0038] In one of the embodiments, the re-identification processing of the target object based on the mapping feature map and the monitoring feature map comprises:

[0039] The mapping feature map and the monitoring feature map are respectively searched in the pre-set feature map database to obtain a plurality of first feature maps corresponding to the mapping feature map and a plurality of second feature maps corresponding to the monitoring feature map; the feature map database stores a plurality of candidate feature maps, and each candidate feature map has a corresponding object.

[0040] The target feature map is determined according to the plurality of first feature maps and the plurality of second feature maps; wherein the object corresponding to the target feature map is the same object as the target object.

[0041] In one of the embodiments, the searching of the mapping feature map and the monitoring feature map in the pre-set feature map database respectively to obtain the plurality of first feature maps corresponding to the mapping feature map and the plurality of second feature maps corresponding to the monitoring feature map comprises:

[0042] Calculate a first similarity between the mapping feature map and each of the plurality of candidate feature maps respectively, and take a candidate feature map whose first similarity meets a first preset condition as a first feature map; the first preset condition includes that the first similarity is greater than a preset threshold;

[0043] Calculate a second similarity between the monitoring feature map and each of the plurality of candidate feature maps respectively, and take a candidate feature map whose second similarity meets a second preset condition as a second feature map; the second preset condition includes that the second similarity is greater than a preset threshold.

[0044] In one of the embodiments, the determining of the target feature map according to the plurality of first feature maps and the plurality of second feature maps includes:

[0045] Filtering the same feature maps from the plurality of first feature maps and the plurality of second feature maps as third feature maps;

[0046] Calculating a target similarity corresponding to each third feature map according to the first similarity and the second similarity corresponding to the third feature map;

[0047] Ranking the target similarities corresponding to the plurality of third feature maps, and determining the target feature map according to the ranking result.

[0048] In one of the embodiments, the calculating of the target similarity of each third feature map according to the first similarity and the second similarity corresponding to the third feature map includes:

[0049] For each third feature map, performing weighted sum calculation on the first similarity and the second similarity to obtain the target similarity.

[0050] A re-identification method, the method includes:

[0051] Obtaining a point cloud mapping image of a target object; the point cloud mapping image is obtained by mapping three-dimensional point cloud data of the target object collected by a laser radar;

[0052] Performing feature extraction on the point cloud mapping image to obtain a mapping feature map corresponding to the point cloud mapping image;

[0053] Performing re-identification processing on the target object based on the mapping feature map.

[0054] A feature map database, the feature map database stores a plurality of candidate feature maps, and each of the candidate feature maps has a corresponding object;

[0055] The feature map database is used for performing re-identification processing on the target object based on the mapping feature map.

[0056] A re-identification device, the device includes:

[0057] An image acquisition module is configured to acquire a point cloud mapping image of a target object and acquire a monitoring image of the target object. The point cloud mapping image is mapped from three-dimensional point cloud data of the target object collected by a laser radar. The monitoring image is captured by a monitoring camera.

[0058] A feature extraction module is configured to perform feature extraction on the point cloud mapping image and the monitoring image respectively to obtain a mapping feature map corresponding to the point cloud mapping image and a monitoring feature map corresponding to the monitoring image.

[0059] A re-identification module is configured to perform re-identification processing on the target object based on the mapping feature map and the monitoring feature map.

[0060] In one of the embodiments, the image acquisition module includes:

[0061] A point cloud data acquisition submodule is configured to acquire three-dimensional point cloud data obtained by scanning a target scene by a laser radar. The target object is located in the target scene.

[0062] A mapping submodule is configured to perform mapping processing on the three-dimensional point cloud data to obtain a two-dimensional image.

[0063] A region detection submodule is configured to determine a three-dimensional recognition region in which the target object is located in the three-dimensional point cloud data by using a target detection algorithm.

[0064] A point cloud mapping image acquisition submodule is configured to map the three-dimensional recognition region to the two-dimensional image to obtain a first recognition region in which the target object is located in the two-dimensional image, and acquire an image in the first recognition region as the point cloud mapping image.

[0065] In one of the embodiments, the mapping submodule is specifically configured to map point cloud points in a preset range in the three-dimensional point cloud data from a three-dimensional coordinate system to a two-dimensional coordinate system according to a preset mapping relationship; obtain an overview map according to positions of the point cloud points in the two-dimensional coordinate system and height data of the point cloud points; obtain an intensity map according to the positions of the point cloud points in the two-dimensional coordinate system and intensity data of the point cloud points; obtain a density map according to a density of the point cloud points in the overview map; and perform merging processing on the overview map, the intensity map and the density map to obtain the two-dimensional image.

[0066] In one of the embodiments, the mapping submodule is specifically configured to determine pixel values corresponding to the point cloud points according to the height data of the point cloud points; and construct the overview map according to the positions of the point cloud points in the two-dimensional coordinate system and the pixel values corresponding to the point cloud points.

[0067] In one of the embodiments, the mapping submodule is specifically configured to determine pixel values corresponding to the point cloud points according to the intensity data of the point cloud points; and construct the intensity map according to the positions of the point cloud points in the two-dimensional coordinate system and the pixel values corresponding to the point cloud points.

[0068] In one of the embodiments, the mapping submodule is specifically configured to determine the pixel arrangement in the density map according to the pixel arrangement in the bird's eye view; calculate the density of the i-th pixel in the bird's eye view according to the number of point clouds of the i-th pixel, the minimum value of the number of point clouds of each pixel, and the maximum value of the number of point clouds of each pixel; i is a positive integer; determine the pixel value of each pixel according to the calculated density of each pixel; and construct the density map according to the pixel arrangement in the density map and the pixel value of each pixel.

[0069] In one of the embodiments, the mapping submodule is specifically configured to merge and process the bird's eye view, the intensity map, and the density map as images of R, G, and B three channels respectively to obtain a two-dimensional image.

[0070] In one of the embodiments, the image acquisition module comprises:

[0071] The photographed image acquisition submodule is configured to acquire a photographed image obtained by the monitoring camera photographing the target scene.

[0072] The monitoring image acquisition submodule is configured to map the stereoscopic recognition region to the photographed image according to the correspondence between the radar coordinates and the camera coordinates, to obtain a second recognition region in the photographed image where the target object is located, and to take the image in the second recognition region as the monitoring image.

[0073] In one of the embodiments, the feature extraction module is specifically configured to input the point cloud mapping image and the monitoring image into a pre-trained feature extraction network respectively to obtain a mapping feature map and a monitoring feature map output by the feature extraction network.

[0074] In one of the embodiments, the re-identification module comprises:

[0075] The searching submodule is configured to search the mapping feature map and the monitoring feature map in a pre-set feature map database respectively to obtain a plurality of first feature maps corresponding to the mapping feature map and a plurality of second feature maps corresponding to the monitoring feature map; the feature map database stores a plurality of candidate feature maps, and each candidate feature map has a corresponding object.

[0076] The target feature map determination submodule is configured to determine a target feature map according to the plurality of first feature maps and the plurality of second feature maps; wherein the object corresponding to the target feature map is the same object as the target object.

[0077] In one of the embodiments, the search submodule is specifically configured to calculate first similarities between the mapping feature map and the plurality of candidate feature maps respectively, and take the candidate feature map that meets a first preset condition as the first feature map; the first preset condition includes that the first similarity is greater than a preset threshold; calculate second similarities between the monitoring feature map and the plurality of candidate feature maps respectively, and take the candidate feature map that meets a second preset condition as the second feature map; the second preset condition includes that the second similarity is greater than the preset threshold.

[0078] In one of the embodiments, the target feature map determination submodule is specifically configured to filter the same feature maps from the plurality of first feature maps and the plurality of second feature maps as third feature maps; calculate target similarities corresponding to the third feature maps according to the first similarities and the second similarities corresponding to the third feature maps; sort the target similarities corresponding to the plurality of third feature maps, and determine the target feature map according to the sorting result.

[0079] In one of the embodiments, the target feature map determination submodule is specifically configured to, for each third feature map, perform weighted sum calculation on the first similarity and the second similarity to obtain the target similarity.

[0080] A computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0081] obtain a point cloud mapping image of a target object and a monitoring image of the target object; the point cloud mapping image is obtained by mapping three-dimensional point cloud data of the target object collected by a laser radar; the monitoring image is obtained by a monitoring camera;

[0082] perform feature extraction on the point cloud mapping image and the monitoring image respectively to obtain a mapping feature map corresponding to the point cloud mapping image and a monitoring feature map corresponding to the monitoring image;

[0083] perform re-identification processing on the target object based on the mapping feature map and the monitoring feature map.

[0084] A re-identification device, the device includes:

[0085] an image acquisition module configured to obtain a point cloud mapping image of a target object; the point cloud mapping image is obtained by mapping three-dimensional point cloud data of the target object collected by a laser radar;

[0086] a feature extraction module configured to perform feature extraction on the point cloud mapping image to obtain a mapping feature map corresponding to the point cloud mapping image;

[0087] a re-identification module configured to perform re-identification processing on the target object based on the mapping feature map.

[0088] A computer readable storage medium has a computer program stored thereon, the computer program being executed by a processor to implement the following steps:

[0089] A point cloud mapping image of the target object is acquired, and a monitoring image of the target object is acquired; the point cloud mapping image is mapped from three-dimensional point cloud data of the target object collected by a laser radar; the monitoring image is photographed by a monitoring camera;

[0090] Features are extracted from the point cloud mapping image and the monitoring image respectively to obtain a mapping feature map corresponding to the point cloud mapping image and a monitoring feature map corresponding to the monitoring image;

[0091] The target object is re-identified based on the mapping feature map and the monitoring feature map.

[0092] The above re-identification method, device, computer equipment and storage medium, the server acquires a point cloud mapping image of the target object, and acquires a monitoring image of the target object; features are extracted from the point cloud mapping image and the monitoring image respectively to obtain a mapping feature map corresponding to the point cloud mapping image and a monitoring feature map corresponding to the monitoring image; the target object is re-identified based on the mapping feature map and the monitoring feature map. Through the embodiment of the application, the three-dimensional point cloud data collected by the laser radar is combined with the monitoring image photographed by the monitoring camera to perform re-identification processing on the target object, the three-dimensional point cloud data is used to make up for the deficiency of the monitoring image, the data features in the re-identification processing can be increased, and the accuracy of the re-identification is improved. BRIEF DESCRIPTION OF DRAWINGS

[0093] Figure 1 It is an application environment diagram of the re-identification method in one embodiment;

[0094] Figure 2 It is a flowchart of the re-identification method in one embodiment;

[0095] Figure 3 It is a flowchart of the step of acquiring the point cloud mapping image of the target object in one embodiment;

[0096] Figure 4 It is a flowchart of the step of mapping the three-dimensional point cloud data to obtain a two-dimensional image in one embodiment;

[0097] Figure 5 It is a flowchart of the step of acquiring the monitoring image of the target object in one embodiment;

[0098] Figure 6 It is a flowchart of the step of re-identifying the target object in one embodiment;

[0099] Figure 7A flowchart of a process of a step of searching in a preset feature map database according to a mapping feature map and a monitoring feature map respectively in one embodiment;

[0100] Figure 8 A flowchart of a process of a step of determining a target feature map according to a plurality of first feature maps and a plurality of second feature maps in one embodiment;

[0101] Figure 9 A flowchart of a process of a re-identification method in another embodiment;

[0102] Figure 10 A flowchart of a process of a re-identification method in yet another embodiment;

[0103] Figure 11 A block diagram of a structure of a re-identification device in one embodiment;

[0104] Figure 12 A block diagram of a structure of a re-identification device in another embodiment;

[0105] Figure 13 An internal structure diagram of a computer device in one embodiment. DETAILED DESCRIPTION

[0106] In order to make the purposes, technical solutions and advantages of the present application clearer, further detailed description will be made to the present application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0107] The re-identification method provided by the present application can be applied in an application environment as shown in Figure 1 . The application environment includes a roadside perception device 102 and a server 104. The roadside perception device 102 communicates with the server 104 through a network, and transmits the collected data to the server 104. The roadside perception device can include a laser radar and a monitoring camera, and the server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0108] In one embodiment, as shown in Figure 2 , a re-identification method is provided. Taking the server in Figure 1 as an example, the method includes the following steps:

[0109] In step 201, the server acquires a point cloud mapping image of a target object, and acquires a monitoring image of the target object.

[0110] The point cloud mapping image is mapped from three-dimensional point cloud data of the target object collected by a laser radar; and the monitoring image is obtained by a monitoring camera.

[0111] The laser radar collects three-dimensional point cloud data of the target object and transmits the three-dimensional point cloud data to the server; the monitoring camera photographs the target object and transmits the photographed image to the server. The server performs mapping processing on the three-dimensional point cloud data to obtain a two-dimensional point cloud mapping image; and performs cropping processing on the photographed image to obtain a monitoring image. The target object is included in both the point cloud mapping image and the monitoring image.

[0112] In step 202, feature extraction is performed on the point cloud mapping image and the monitoring image respectively to obtain a mapping feature map corresponding to the point cloud mapping image and a monitoring feature map corresponding to the monitoring image.

[0113] After the server obtains the point cloud mapping image and the monitoring image, feature extraction is performed on the point cloud mapping image to obtain a mapping feature map corresponding to the point cloud mapping image; and feature extraction is performed on the monitoring image to obtain a monitoring feature map corresponding to the monitoring image. The mapping feature map and the monitoring feature map both include features of the target object. The feature extraction process can use a pre-trained feature extraction network, which is not limited in the embodiments of the present disclosure.

[0114] In step 203, the target object is subjected to re-identification processing based on the mapping feature map and the monitoring feature map.

[0115] A feature map database is pre-set. After the mapping feature map and the monitoring feature map are obtained, the mapping feature map and the monitoring feature map are respectively searched in the feature map database. A feature map similar to both the mapping feature map and the monitoring feature map is taken as a target feature map. The object corresponding to the target feature map is the same object as the target object, that is, the re-identification processing of the target object is completed.

[0116] In the above re-identification method, the server obtains a point cloud mapping image of a target object and a monitoring image of the target object; performs feature extraction on the point cloud mapping image and the monitoring image respectively to obtain a mapping feature map corresponding to the point cloud mapping image and a monitoring feature map corresponding to the monitoring image; and performs re-identification processing on the target object based on the mapping feature map and the monitoring feature map. According to the embodiments of the present disclosure, the three-dimensional point cloud data collected by the laser radar and the monitoring image photographed by the monitoring camera are combined for re-identification processing of the target object. The three-dimensional point cloud data can make up for the deficiency of the monitoring image, increase the data features in the re-identification processing, and thus improve the accuracy of re-identification.

[0117] In one embodiment, as shown in FIG. 1, the step of obtaining the point cloud mapping image of the target object includes: Figure 3

[0118] In step 301, the server obtains three-dimensional point cloud data obtained by scanning a target scene by a laser radar.

[0119] ​The target object is located in a target scene. For example, the target object is a target vehicle, the target vehicle is located on a road, a laser radar arranged on the roadside scans the road, and three-dimensional point cloud data scanned is transmitted to a server, and the server can obtain the three-dimensional point cloud data of the target vehicle.

[0120] In step 302, the three-dimensional point cloud data is mapped to obtain a two-dimensional image.

[0121] A corresponding relationship between the three-dimensional coordinate system and the two-dimensional coordinate system is established in advance, as shown in formula (1):

[0122]

[0123] wherein a and b represent the coordinates of a point cloud point in the three-dimensional coordinate system, a t , b t represent the coordinates of a point cloud point in the two-dimensional coordinate system after mapping, h represents the distance from the point cloud boundary to the y-axis, w represents the distance from the point cloud boundary to the x-axis, and the x-axis and the y-axis are coordinate axes with the laser radar as the origin.

[0124] According to the above corresponding relationship, each point cloud point in the three-dimensional point cloud data is mapped to the two-dimensional coordinate system to obtain a two-dimensional image.

[0125] In step 303, a target detection algorithm is used to determine a three-dimensional recognition region in which the target object is located in the three-dimensional point cloud data.

[0126] The target detection algorithm is used for target detection to obtain a three-dimensional bounding box of the target object, and the region contained in the three-dimensional bounding box is the three-dimensional recognition region in which the target object is located.

[0127] In step 304, the three-dimensional recognition region is mapped to the two-dimensional image to obtain a first recognition region in which the target object is located in the two-dimensional image, and an image in the first recognition region is taken as a point cloud mapping image.

[0128] According to the above corresponding relationship between the three-dimensional coordinate system and the two-dimensional coordinate system, eight vertices of the three-dimensional recognition region are mapped to the two-dimensional image to obtain a two-dimensional bounding box of the target object in the two-dimensional image. The region contained in the two-dimensional bounding box is the first recognition region in which the target object is located, and an image in the first recognition region is taken as a point cloud mapping image.

[0129] In the process of obtaining the point cloud mapping image of the target object, the server obtains three-dimensional point cloud data obtained by scanning a target scene by a laser radar; performs mapping processing on the three-dimensional point cloud data to obtain a two-dimensional image; determines a three-dimensional recognition region in which the target object is located in the three-dimensional point cloud data by using a target detection algorithm; maps the three-dimensional recognition region into the two-dimensional image to obtain a first recognition region in which the target object is located in the two-dimensional image, and takes an image in the first recognition region as the point cloud mapping image. According to the embodiment of the present disclosure, the three-dimensional point cloud data is mapped to obtain a two-dimensional image, and the three-dimensional recognition region in which the target object is located is mapped into the two-dimensional image to obtain a point cloud mapping image. In this way, the features of the target object are converted from a three-dimensional coordinate system to a two-dimensional coordinate system, and the re-identification processing can be combined with a monitoring image to improve the accuracy of re-identification.

[0130] In one embodiment, as shown in Figure 4 the step of performing mapping processing on the three-dimensional point cloud data to obtain a two-dimensional image includes:

[0131] In step 401, the server maps point cloud points in a preset range in the three-dimensional point cloud data from a three-dimensional coordinate system to a two-dimensional coordinate system according to a preset mapping relationship.

[0132] The point cloud points are sampling points for obtaining features of an object surface, and include three-dimensional coordinates (XYZ) and intensity information (Intensity). The intensity information is related to the material and roughness of the object surface, and the direction of the laser incidence angle, the emission energy of the instrument, and the wavelength of the laser.

[0133] In the process of mapping the three-dimensional point cloud data, the point cloud points in the preset range are intercepted. The preset range can be a range of ±70m with the laser radar as the origin and the x-axis and the y-axis.

[0134] After the point cloud points in the preset range are intercepted, each point cloud point is mapped according to formula (1) to obtain the position of each point cloud point in the two-dimensional coordinate system.

[0135] In step 402, a bird's eye view is obtained according to the position of each point cloud point in the two-dimensional coordinate system and the height data of each point cloud point.

[0136] First, the pixel value corresponding to each point cloud point is determined according to the height data of each point cloud point; and then the bird's eye view is constructed according to the position of each point cloud point in the two-dimensional coordinate system and the pixel value corresponding to each point cloud point.

[0137] In one of the embodiments, the pixel value corresponding to each point cloud point is determined according to the height data of the point cloud point, including: performing normalization on the height data of each point cloud point, normalizing the height data to 0-255, to obtain a height normalized value corresponding to each point cloud point, and taking the height normalized value as the pixel value corresponding to the point cloud point.

[0138] In one of the embodiments, the bird's eye view is constructed according to the position of each point cloud point in the two-dimensional coordinate system and the pixel value corresponding to each point cloud point, including: setting the pixel point arrangement mode of the bird's eye view according to the position of each point cloud point in the two-dimensional coordinate system; and filling each pixel point according to the pixel value corresponding to each point cloud point, to obtain the bird's eye view.

[0139] In actual application, when sequentially mapping each point cloud point from the three-dimensional coordinate system to the two-dimensional coordinate system, the situation that the coordinate of a later-mapped point cloud point is the same as that of an earlier-mapped point cloud point may occur. In this case, the pixel value of the later-mapped point cloud point is taken as the pixel value of the coordinate. That is, the pixel value of the later-mapped point cloud point is used to cover the pixel value of the earlier-mapped point cloud point.

[0140] In step 403, the intensity map is obtained according to the position of each point cloud point in the two-dimensional coordinate system and the intensity data of each point cloud point.

[0141] According to the intensity data of each point cloud point, the pixel value corresponding to each point cloud point is determined; and according to the position of each point cloud point in the two-dimensional coordinate system and the pixel value corresponding to each point cloud point, the intensity map is constructed.

[0142] In one of the embodiments, the pixel value corresponding to each point cloud point is determined according to the intensity data of the point cloud point, including: performing normalization on the intensity data of each point cloud point, normalizing the intensity data to 0-255, to obtain an intensity normalized value corresponding to each point cloud point, and taking the intensity normalized value as the pixel value corresponding to the point cloud point.

[0143] In one of the embodiments, the intensity map is constructed according to the position of each point cloud point in the two-dimensional coordinate system and the pixel value corresponding to each point cloud point, including: setting the pixel point arrangement mode of the intensity map according to the position of each point cloud point in the two-dimensional coordinate system; and filling each pixel point according to the pixel value corresponding to each point cloud point, to obtain the intensity map. The pixel point arrangement mode of the intensity map is the same as that of the bird's eye view.

[0144] In actual application, when sequentially mapping each point cloud point from the three-dimensional coordinate system to the two-dimensional coordinate system, the situation that the coordinate of a later-mapped point cloud point is the same as that of an earlier-mapped point cloud point may occur. Similarly, the pixel value of the later-mapped point cloud point is used to cover the pixel value of the earlier-mapped point cloud point.

[0145] In step 404, the density map is obtained according to the density of the point cloud points in the bird's eye view.

[0146] The pixel arrangement in the density map is determined based on the pixel arrangement in the bird's-eye view; the density of the i-th pixel is calculated based on the number of point clouds in the i-th pixel in the bird's-eye view, the minimum number of point clouds in each pixel, and the maximum number of point clouds in each pixel; i is a positive integer; the pixel value of each pixel is determined based on the calculated density of each pixel; and the density map is constructed based on the pixel arrangement and pixel value of each pixel in the density map.

[0147] The density of the i-th pixel can be calculated using formula (2):

[0148]

[0149] Where, ρ i Let c be the density of the i-th pixel. i Let c be the number of point clouds for the i-th pixel. min c is the minimum number of point clouds in each pixel. max This represents the maximum number of point clouds in each pixel.

[0150] By filling each pixel with its pixel value, a density map can be obtained.

[0151] Step 405: Merge the bird's-eye view, intensity map, and density map to obtain a two-dimensional image.

[0152] The bird's-eye view, intensity map, and density map are merged as R, G, and B channels respectively to obtain a two-dimensional image.

[0153] In the process of mapping 3D point cloud data to obtain a 2D image, the server maps point cloud points within a preset range from the 3D coordinate system to the 2D coordinate system according to a pre-set mapping relationship. A bird's-eye view is obtained based on the position and height data of each point cloud point in the 2D coordinate system. An intensity map is obtained based on the position and intensity data of each point cloud point in the 2D coordinate system. A density map is obtained based on the density of point cloud points in the bird's-eye view. The bird's-eye view, intensity map, and density map are then merged to obtain the 2D image. Through this embodiment, a bird's-eye view, intensity map, and density map are obtained from the 3D point cloud data, and then merged to obtain a 2D image. The 2D image contains various information about the point cloud points, thus preserving more features of the target object during subsequent feature extraction, which is more beneficial for re-identification processing.

[0154] In one embodiment, such as Figure 5 As shown, the steps for obtaining a monitoring image of the target object include:

[0155] Step 501: The server obtains the captured images of the target scene from the surveillance camera.

[0156] In actual application, the target object is a target vehicle, the target vehicle is located on a road, and a monitoring camera arranged on the roadside captures the road, so that a captured image of the target vehicle can be obtained. The monitoring camera transmits the captured image to a server, and the server can obtain the captured image of the target object.

[0157] In step 502, the stereoscopic recognition region is mapped into the captured image according to the correspondence between the radar coordinates and the camera coordinates, so as to obtain a second recognition region in which the target object is located in the captured image, and the image in the second recognition region is taken as the monitoring image.

[0158] When the laser radar and the monitoring camera are arranged on the roadside, the positional relationship between the laser radar and the monitoring camera, the scanning parameters of the laser radar, and the shooting parameters of the monitoring camera can be obtained. Then, the server can determine the correspondence between the radar coordinates and the camera coordinates according to the positional relationship, the scanning parameters, and the shooting parameters.

[0159] According to the correspondence between the radar coordinates and the camera coordinates, the eight vertices of the stereoscopic recognition region are mapped into the captured image, so as to obtain a two-dimensional bounding box of the target object in the captured image. The area contained in the two-dimensional bounding box is the second recognition region in which the target object is located, and the image in the second recognition region is taken as the monitoring image.

[0160] In the above process of obtaining the monitoring image of the target object, the server obtains a captured image of a target scene captured by a monitoring camera; the stereoscopic recognition region is mapped into the captured image according to the correspondence between the radar coordinates and the camera coordinates, so as to obtain a second recognition region in which the target object is located in the captured image, and the image in the second recognition region is taken as the monitoring image. According to the relationship between the laser radar and the monitoring camera, the stereoscopic recognition region in which the target object is located is mapped into the captured image to obtain the monitoring image. Since the three-dimensional point cloud data is not affected by distance and environment, even if the captured image is not clear enough due to the influence of distance or environment, the second recognition region in the captured image can be obtained through mapping, so that the target object can be extracted from the captured image for re-identification processing.

[0161] In one embodiment, as shown in FIG. 6, the step of performing re-identification processing on the target object based on the mapping feature map and the monitoring feature map includes: Figure 6

[0162] In step 601, the server respectively searches a pre-set feature map database according to the mapping feature map and the monitoring feature map, so as to obtain a plurality of first feature maps corresponding to the mapping feature map and a plurality of second feature maps corresponding to the monitoring feature map.

[0163] ​The pre-set feature map database stores a plurality of candidate feature maps, and each candidate feature map has a corresponding object. For example, the plurality of candidate feature maps are candidate feature Figure 1 maps Figure 2 … candidate feature map 100, candidate feature Figure 1 map corresponding to vehicle 1, candidate feature Figure 2 map corresponding to vehicle 2 … candidate feature map 100 corresponding to vehicle 100.

[0164] The candidate feature maps in the feature map database can be obtained in the same way as the monitoring feature map corresponding to the monitoring image. The embodiments of the present disclosure do not limit this.

[0165] According to the mapping feature map, a plurality of first feature maps similar to the mapping feature map are obtained by searching in the feature map database. According to the monitoring feature map, a plurality of second feature maps similar to the monitoring feature map are obtained by searching in the feature map database.

[0166] Step 602, determining a target feature map according to the plurality of first feature maps and the plurality of second feature maps.

[0167] The object corresponding to the target feature map is the same object as the target object.

[0168] After obtaining the plurality of first feature maps and the plurality of second feature maps, a target feature map with high similarity to both the mapping feature map and the monitoring feature map is selected from these feature maps.

[0169] In the above process of re-identifying the target object based on the mapping feature map and the monitoring feature map, the server searches in the pre-set feature map database according to the mapping feature map and the monitoring feature map respectively, obtains a plurality of first feature maps corresponding to the mapping feature map and a plurality of second feature maps corresponding to the monitoring feature map, and determines a target feature map according to the plurality of first feature maps and the plurality of second feature maps. According to the embodiments of the present disclosure, the target feature map is obtained by searching the feature map database according to the mapping feature map and the monitoring feature map, and the target object is determined according to the object corresponding to the target feature map. Since the mapping feature map obtained by processing the three-dimensional point cloud data can make up for the shortcomings of the monitoring image, the data features in the re-identification process can be increased, and the accuracy of the re-identification can be improved.

[0170] In one embodiment, as shown in Figure 7 the step of searching in the pre-set feature map database according to the mapping feature map and the monitoring feature map respectively to obtain a plurality of first feature maps corresponding to the mapping feature map and a plurality of second feature maps corresponding to the monitoring feature map, includes:

[0171] In step 6011, the server respectively calculates first similarities between the mapping feature map and the plurality of candidate feature maps, and takes the candidate feature map whose first similarity meets a first preset condition as a first feature map.

[0172] The first preset condition includes that the first similarity is greater than a preset threshold.

[0173] For example, the first similarity between the mapping feature map and the candidate feature map Figure 1 is 70%, the first similarity between the mapping feature map and the candidate feature map Figure 2 is 95%, and the first similarity between the mapping feature map and the candidate feature map Figure 3 is 91%. In this way, the first similarities between the mapping feature map and the plurality of candidate feature maps are obtained.

[0174] After the first similarities between the mapping feature map and the plurality of candidate feature maps are calculated, the candidate feature map whose first similarity is greater than the preset threshold is taken as the first feature map. For example, the preset threshold is 90%, and the candidate feature map Figure 2 and the candidate feature map Figure 3 are taken as the first feature map.

[0175] In step 6012, second similarities between the monitoring feature map and the plurality of candidate feature maps are respectively calculated, and the candidate feature map whose second similarity meets a second preset condition is taken as a second feature map.

[0176] The second preset condition includes that the second similarity is greater than the preset threshold.

[0177] For example, the second similarity between the monitoring feature map and the candidate feature map Figure 1 is 72%, the second similarity between the monitoring feature map and the candidate feature map Figure 2 is 94%, and the second similarity between the monitoring feature map and the candidate feature map Figure 3 is 89%. In this way, the second similarities between the monitoring feature map and the plurality of candidate feature maps are obtained. If the preset threshold is 90%, the candidate feature map Figure 2 is taken as the second feature map.

[0178] The above similarity calculation can be cosine similarity calculation or other similarity calculation, which is not limited in the embodiments of the present disclosure.

[0179] The order of steps 6011 and 6012 is not limited in the embodiments of the present disclosure.

[0180] It can be understood that the server respectively calculates the first similarity between the mapping feature map and the plurality of candidate feature maps, and takes the candidate feature map whose first similarity meets the first preset condition as the first feature map; respectively calculates the second similarity between the monitoring feature map and the plurality of candidate feature maps, and takes the candidate feature map whose second similarity meets the second preset condition as the second feature map. The feature map similar to the mapping feature map and the feature map similar to the monitoring feature map can be retrieved from the feature map database, so as to further determine the target feature map, and realize the re-identification processing.

[0181] In one of the embodiments, as shown in Figure 8 the step of determining the target feature map according to the plurality of first feature maps and the plurality of second feature maps comprises:

[0182] In step 6021, the server screens the same feature map from the plurality of first feature maps and the plurality of second feature maps as the third feature map.

[0183] For example, the candidate feature Figure 2 is both the first feature map and the second feature map, and the candidate feature Figure 2 is taken as the third feature map.

[0184] In step 6022, the target similarity corresponding to each third feature map is calculated according to the first similarity and the second similarity corresponding to each third feature map.

[0185] The first similarity and the second similarity are weighted and summed for each third feature map to obtain the target similarity. For example, the candidate feature Figure 2 is the third feature map, the first similarity corresponding to the candidate feature Figure 2 is 95%, and the second similarity corresponding to the candidate feature Figure 2 is 42%, and if the weights of the first similarity and the second similarity are 0.6 and 0.4 respectively, the target similarity can be calculated as 94.6%. The weight of the first similarity and the second similarity is not limited in the embodiment of the disclosure.

[0186] In step 6023, the target similarities corresponding to the plurality of third feature maps are sorted, and the target feature map is determined according to the sorting result.

[0187] If there is only one third feature map, the third feature map is taken as the target feature map; if there are a plurality of third feature maps, the plurality of third feature maps are sorted according to the target similarity from high to low, and the third feature map with the highest target similarity is taken as the target feature map.

[0188] Understandably, the server screens the same feature maps from the plurality of first feature maps and the plurality of second feature maps as third feature maps; calculates target similarities corresponding to each third feature map according to the first similarity and the second similarity corresponding to each third feature map; sorts the target similarities corresponding to the plurality of third feature maps, and determines a target feature map according to a sorting result, so that the target feature map with high similarity to both the mapping feature map and the monitoring feature map can be found, thereby realizing the re-identification processing. Moreover, the weights of the mapping feature map and the monitoring feature map can be set, so that more reliable data can be selected in the re-identification processing, and thus the accuracy of the re-identification can be improved.

[0189] In one embodiment, as shown in Figure 9 a re-identification method is provided, which can include:

[0190] Step 701, the server obtains three-dimensional point cloud data obtained by scanning a target scene by a laser radar.

[0191] Among them, the target object is located in the target scene.

[0192] Step 702, according to a pre-set mapping relationship, mapping the point cloud points in a preset range in the three-dimensional point cloud data from a three-dimensional coordinate system to a two-dimensional coordinate system.

[0193] Step 703, according to the positions of each point cloud point in the two-dimensional coordinate system and the height data of each point cloud point, obtaining an aerial view.

[0194] In one embodiment, according to the height data of each point cloud point, the pixel value corresponding to each point cloud point is determined; and according to the position of each point cloud point in the two-dimensional coordinate system and the pixel value corresponding to each point cloud point, the aerial view is constructed.

[0195] Step 704, according to the positions of each point cloud point in the two-dimensional coordinate system and the intensity data of each point cloud point, obtaining an intensity map.

[0196] In one embodiment, according to the intensity data of each point cloud point, the pixel value corresponding to each point cloud point is determined; and according to the position of each point cloud point in the two-dimensional coordinate system and the pixel value corresponding to each point cloud point, the intensity map is constructed.

[0197] Step 705, according to the density of the point cloud points in the aerial view, obtaining a density map.

[0198] In one of the embodiments, the pixel arrangement in the density map is determined according to the pixel arrangement in the bird's eye view; the density of the i-th pixel in the bird's eye view is calculated according to the number of point clouds in the i-th pixel, the minimum value of the number of point clouds in each pixel, and the maximum value of the number of point clouds in each pixel; i is a positive integer; the pixel value of each pixel is determined according to the calculated density of each pixel; and the density map is constructed according to the pixel arrangement in the density map and the pixel value of each pixel.

[0199] In step 706, the bird's eye view, the intensity map, and the density map are merged to obtain a two-dimensional image.

[0200] In one of the embodiments, the bird's eye view, the intensity map, and the density map are merged as R, G, and B channel images respectively to obtain a two-dimensional image.

[0201] In step 707, a target detection algorithm is used to determine a three-dimensional recognition region of a target object in the three-dimensional point cloud data.

[0202] In step 708, the three-dimensional recognition region is mapped to the two-dimensional image to obtain a first recognition region of the target object in the two-dimensional image, and the image in the first recognition region is taken as a point cloud mapping image.

[0203] In step 709, a shooting image of a target scene obtained by a monitoring camera is acquired.

[0204] In step 710, the three-dimensional recognition region is mapped to the shooting image according to a preset correspondence between the radar coordinates and the camera coordinates to obtain a second recognition region of the target object in the shooting image, and the image in the second recognition region is taken as a monitoring image.

[0205] In step 711, the point cloud mapping image and the monitoring image are respectively input into a pre-trained feature extraction network to obtain a mapping feature map and a monitoring feature map output by the feature extraction network.

[0206] The first feature extraction network is pre-trained for the point cloud mapping image, and the second feature extraction network is pre-trained for the monitoring image. The first feature extraction network and the second feature extraction network can both adopt a full convolutional neural network. The training of the feature extraction network is not limited in the embodiments of the present disclosure.

[0207] In the feature extraction process, the point cloud mapping image is input into the first feature extraction network to obtain a mapping feature map output by the first feature extraction network; and the monitoring image is input into the second feature extraction network to obtain a monitoring feature map output by the second feature extraction network.

[0208] Step 712, respectively according to the mapping feature map and the monitoring feature map, searching in the pre-set feature map database to obtain a plurality of first feature maps corresponding to the mapping feature map and a plurality of second feature maps corresponding to the monitoring feature map.

[0209] The feature map database stores a plurality of candidate feature maps, and each candidate feature map has a corresponding object.

[0210] Step 713, determining a target feature map according to the plurality of first feature maps and the plurality of second feature maps.

[0211] The object corresponding to the target feature map is the same object as the target object.

[0212] In the above re-identification method, the three-dimensional point cloud data collected by the laser radar is combined with the monitoring image captured by the monitoring camera to realize the re-identification processing of the target object. The three-dimensional point cloud data can make up for the deficiency of the monitoring image, increase the data features in the re-identification processing, and further improve the accuracy of the re-identification.

[0213] In one embodiment, as shown in Figure 10 , a re-identification method is provided. Taking the server in Figure 1 as an example, the method can include the following steps:

[0214] Step 801, the server acquires a point cloud mapping image of a target object.

[0215] The point cloud mapping image is mapped from three-dimensional point cloud data of the target object collected by a laser radar.

[0216] Step 802, feature extraction is performed on the point cloud mapping image to obtain a mapping feature map corresponding to the point cloud mapping image.

[0217] Step 803, re-identification processing is performed on the target object based on the mapping feature map.

[0218] The specific implementation of the embodiments of the present disclosure can refer to the above embodiments, which will not be described here. Through the embodiments of the present disclosure, the re-identification processing of the target object is performed according to the three-dimensional point cloud data collected by the laser radar. Since the three-dimensional point cloud data is not affected by environmental factors, the characteristics of the target object can be preserved, thereby improving the accuracy of the re-identification.

[0219] In one embodiment, a feature map database is provided, which stores a plurality of candidate feature maps, and each candidate feature map has a corresponding object. The feature map database is used for re-identification processing of a target object based on a mapping feature map.

[0220] It should be understood that, although Figures 2-10The steps in the flowcharts are shown in sequence according to the arrows, but the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the steps are not strictly limited in sequence, and the steps can be executed in other orders. Moreover, Figures 2-10 At least part of the steps in the flowcharts can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of the steps or stages is not necessarily sequential, but can be alternately executed with other steps or steps or stages in other steps.

[0221] In one embodiment, as shown in Figure 11 A re-identification device is provided, comprising:

[0222] An image acquisition module 10 is configured to acquire a point cloud mapping image of a target object and acquire a monitoring image of the target object. The point cloud mapping image is obtained by mapping three-dimensional point cloud data of the target object collected by a laser radar. The monitoring image is obtained by a monitoring camera.

[0223] A feature extraction module 11 is configured to perform feature extraction on the point cloud mapping image and the monitoring image respectively to obtain a mapping feature map corresponding to the point cloud mapping image and a monitoring feature map corresponding to the monitoring image.

[0224] A re-identification module 12 is configured to perform re-identification processing on the target object based on the mapping feature map and the monitoring feature map.

[0225] In one embodiment, the image acquisition module 10 comprises:

[0226] A point cloud data acquisition submodule is configured to acquire three-dimensional point cloud data obtained by scanning a target scene by a laser radar. The target object is located in the target scene.

[0227] A mapping submodule is configured to perform mapping processing on the three-dimensional point cloud data to obtain a two-dimensional image.

[0228] A region detection submodule is configured to determine a three-dimensional recognition region in which the target object is located in the three-dimensional point cloud data by using a target detection algorithm.

[0229] A point cloud mapping image acquisition submodule is configured to map the three-dimensional recognition region to the two-dimensional image to obtain a first recognition region in which the target object is located in the two-dimensional image, and acquire an image in the first recognition region as the point cloud mapping image.

[0230] In one of the embodiments, the mapping submodule is specifically configured to map the point cloud points in the preset range in the three-dimensional point cloud data from the three-dimensional coordinate system to the two-dimensional coordinate system according to the preset mapping relationship; obtain the bird's eye view according to the positions of the point cloud points in the two-dimensional coordinate system and the height data of the point cloud points; obtain the intensity map according to the positions of the point cloud points in the two-dimensional coordinate system and the intensity data of the point cloud points; obtain the density map according to the density of the point cloud points in the bird's eye view; and perform merging processing on the bird's eye view, the intensity map and the density map to obtain the two-dimensional image.

[0231] In one of the embodiments, the mapping submodule is specifically configured to determine the pixel value corresponding to each point cloud point according to the height data of the point cloud points; and construct the bird's eye view according to the positions of the point cloud points in the two-dimensional coordinate system and the pixel value corresponding to each point cloud point.

[0232] In one of the embodiments, the mapping submodule is specifically configured to determine the pixel value corresponding to each point cloud point according to the intensity data of the point cloud points; and construct the intensity map according to the positions of the point cloud points in the two-dimensional coordinate system and the pixel value corresponding to each point cloud point.

[0233] In one of the embodiments, the mapping submodule is specifically configured to determine the arrangement of the pixel points in the density map according to the arrangement of the pixel points in the bird's eye view; calculate the density of the i th pixel point in the bird's eye view according to the number of point cloud points of the i th pixel point, the minimum value of the number of point cloud points of each pixel point and the maximum value of the number of point cloud points of each pixel point; i is a positive integer; determine the pixel value of each pixel point according to the calculated density of each pixel point; and construct the density map according to the arrangement of the pixel points in the density map and the pixel value of each pixel point.

[0234] In one of the embodiments, the mapping submodule is specifically configured to perform merging processing on the bird's eye view, the intensity map and the density map as images of R, G and B three channels respectively to obtain the two-dimensional image.

[0235] In one of the embodiments, the image acquisition module 10 comprises:

[0236] The photographed image acquisition submodule is configured to acquire a photographed image obtained by the monitoring camera when photographing the target scene.

[0237] The monitoring image acquisition submodule is configured to map the stereoscopic recognition region to the photographed image according to the preset corresponding relationship between the radar coordinate and the camera coordinate to obtain a second recognition region in the photographed image where the target object is located, and take the image in the second recognition region as the monitoring image.

[0238] In one of the embodiments, the feature extraction module 11 is specifically configured to input the point cloud mapping image and the monitoring image into the pre-trained feature extraction network respectively to obtain the mapping feature map and the monitoring feature map output by the feature extraction network.

[0239] In one of the embodiments, the re-identification module 12 includes:

[0240] A searching sub-module is configured to search the mapping feature map and the monitoring feature map in a pre-set feature map database respectively to obtain a plurality of first feature maps corresponding to the mapping feature map and a plurality of second feature maps corresponding to the monitoring feature map; the feature map database stores a plurality of candidate feature maps, and each candidate feature map has a corresponding object.

[0241] A target feature map determining sub-module is configured to determine a target feature map according to the plurality of first feature maps and the plurality of second feature maps; the object corresponding to the target feature map is the same object as the target object.

[0242] In one of the embodiments, the searching sub-module is specifically configured to calculate a first similarity between the mapping feature map and the plurality of candidate feature maps respectively, and take the candidate feature map whose first similarity meets a first preset condition as a first feature map; the first preset condition includes that the first similarity is greater than a preset threshold; calculate a second similarity between the monitoring feature map and the plurality of candidate feature maps respectively, and take the candidate feature map whose second similarity meets a second preset condition as a second feature map; the second preset condition includes that the second similarity is greater than the preset threshold.

[0243] In one of the embodiments, the target feature map determining sub-module is specifically configured to filter out the same feature maps from the plurality of first feature maps and the plurality of second feature maps as third feature maps; calculate a target similarity corresponding to each third feature map according to the first similarity and the second similarity corresponding to each third feature map; sort the target similarities corresponding to the plurality of third feature maps, and determine the target feature map according to the sorting result.

[0244] In one of the embodiments, the target feature map determining sub-module is specifically configured to, for each third feature map, perform weighted sum calculation on the first similarity and the second similarity to obtain the target similarity.

[0245] In one of the embodiments, as shown in Figure 12 a re-identification device is provided, which includes:

[0246] An image acquisition module 20 is configured to acquire a point cloud mapping image of a target object; the point cloud mapping image is obtained by mapping three-dimensional point cloud data of the target object collected by a laser radar;

[0247] The feature extraction module 21 is configured to perform feature extraction on the point cloud mapping image to obtain a mapping feature map corresponding to the point cloud mapping image.

[0248] The re-identification module 22 is configured to perform re-identification processing on the target object based on the mapping feature map.

[0249] For specific limitations of the re-identification device, refer to the limitations of the re-identification method described above, which will not be repeated here. Each module in the above re-identification device can be realized by software, hardware and their combination in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so that the processor calls and executes the operations corresponding to the above modules.

[0250] In one embodiment, a computer device is provided, which can be a server, and its internal structure diagram can be as shown in Figure 13 The computer device includes a processor, a memory and a network interface connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store re-identification data. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a re-identification method.

[0251] Those skilled in the art can understand that Figure 13 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0252] In one embodiment, a computer device is provided, which includes a memory and a processor, and the memory stores a computer program. The processor executes the computer program to implement the following steps:

[0253] Obtain a point cloud mapping image of a target object and a monitoring image of the target object. The point cloud mapping image is obtained by mapping three-dimensional point cloud data of the target object collected by a laser radar. The monitoring image is obtained by a monitoring camera;

[0254] Perform feature extraction on the point cloud mapping image and the monitoring image respectively to obtain a mapping feature map corresponding to the point cloud mapping image and a monitoring feature map corresponding to the monitoring image;

[0255] Based on the mapping feature map and the monitoring feature map, the target object is re-identified.

[0256] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0257] Obtaining three-dimensional point cloud data obtained by scanning a target scene by a laser radar; the target object is located in the target scene;

[0258] Mapping processing is performed on the three-dimensional point cloud data to obtain a two-dimensional image;

[0259] A three-dimensional recognition region in which the target object is located in the three-dimensional point cloud data is determined by using a target detection algorithm;

[0260] The three-dimensional recognition region is mapped into the two-dimensional image to obtain a first recognition region in which the target object is located in the two-dimensional image, and an image in the first recognition region is taken as a point cloud mapping image.

[0261] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0262] According to a preset mapping relationship, point cloud points in a preset range in the three-dimensional point cloud data are mapped from a three-dimensional coordinate system to a two-dimensional coordinate system;

[0263] According to the positions of the point cloud points in the two-dimensional coordinate system and the height data of the point cloud points, an overview map is obtained;

[0264] According to the positions of the point cloud points in the two-dimensional coordinate system and the intensity data of the point cloud points, an intensity map is obtained;

[0265] According to the density of the point cloud points in the overview map, a density map is obtained;

[0266] The overview map, the intensity map and the density map are merged to obtain a two-dimensional image.

[0267] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0268] According to the height data of the point cloud points, pixel values corresponding to the point cloud points are determined;

[0269] According to the positions of the point cloud points in the two-dimensional coordinate system and the pixel values corresponding to the point cloud points, an overview map is constructed.

[0270] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0271] According to the intensity data of the point cloud points, pixel values corresponding to the point cloud points are determined;

[0272] According to the positions of the point cloud points in the two-dimensional coordinate system and the pixel values corresponding to the point cloud points, an intensity map is constructed.

[0273] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0274] According to the arrangement of the pixel points in the bird's eye view, the arrangement of the pixel points in the density map is determined.

[0275] According to the number of point clouds of the i-th pixel point in the bird's eye view, the minimum value of the number of point clouds in each pixel point, and the maximum value of the number of point clouds in each pixel point, the density of the i-th pixel point is calculated; i is a positive integer.

[0276] According to the calculated densities of the pixel points, the pixel values of the pixel points are determined.

[0277] According to the arrangement of the pixel points in the density map and the pixel values of the pixel points, a density map is constructed.

[0278] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0279] The bird's eye view, the intensity map and the density map are respectively merged as images of R, G and B channels to obtain a two-dimensional image.

[0280] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0281] A shooting image obtained by a monitoring camera shooting a target scene is acquired.

[0282] According to a pre-set correspondence between radar coordinates and camera coordinates, a stereoscopic recognition region is mapped to the shooting image to obtain a second recognition region in the shooting image where the target object is located, and an image in the second recognition region is taken as a monitoring image.

[0283] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0284] The point cloud mapping image and the monitoring image are respectively input into a pre-trained feature extraction network to obtain a mapping feature map and a monitoring feature map output by the feature extraction network.

[0285] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0286] The mapping feature map and the monitoring feature map are respectively searched in a pre-set feature map database to obtain a plurality of first feature maps corresponding to the mapping feature map and a plurality of second feature maps corresponding to the monitoring feature map; the feature map database stores a plurality of candidate feature maps, and each candidate feature map has a corresponding object.

[0287] The target feature map is determined according to the plurality of first feature maps and the plurality of second feature maps; wherein the object corresponding to the target feature map is the same object as the target object.

[0288] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0289] The first similarity between the mapping feature map and the plurality of candidate feature maps is calculated respectively, and the candidate feature map whose first similarity meets a first preset condition is taken as the first feature map; the first preset condition includes that the first similarity is greater than a preset threshold.

[0290] The second similarity between the monitoring feature map and the plurality of candidate feature maps is calculated respectively, and the candidate feature map whose second similarity meets a second preset condition is taken as the second feature map; the second preset condition includes that the second similarity is greater than a preset threshold.

[0291] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0292] The same feature maps are screened out from the plurality of first feature maps and the plurality of second feature maps as the third feature maps;

[0293] The target similarity corresponding to each third feature map is calculated according to the first similarity and the second similarity corresponding to each third feature map;

[0294] The target similarities corresponding to the plurality of third feature maps are sorted, and the target feature map is determined according to the sorting result.

[0295] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0296] The first similarity and the second similarity are weighted and summed for each third feature map to obtain the target similarity.

[0297] In one embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the following steps:

[0298] A point cloud mapping image of the target object is obtained, and a monitoring image of the target object is obtained; the point cloud mapping image is obtained by mapping three-dimensional point cloud data of the target object collected by a laser radar; the monitoring image is obtained by a monitoring camera;

[0299] Feature extraction is performed on the point cloud mapping image and the monitoring image respectively to obtain a mapping feature map corresponding to the point cloud mapping image and a monitoring feature map corresponding to the monitoring image;

[0300] The target object is re-identified based on the mapping feature map and the monitoring feature map.

[0301] In one embodiment, the computer program, when executed by the processor, implements the following steps:

[0302] Obtaining three-dimensional point cloud data obtained by a laser radar scanning a target scene; a target object is located in the target scene;

[0303] Mapping the three-dimensional point cloud data to obtain a two-dimensional image;

[0304] Determining a three-dimensional recognition region in which the target object is located in the three-dimensional point cloud data by using a target detection algorithm;

[0305] Mapping the three-dimensional recognition region to the two-dimensional image to obtain a first recognition region in which the target object is located in the two-dimensional image, and taking an image in the first recognition region as a point cloud mapping image.

[0306] In one embodiment, the computer program, when executed by the processor, implements the following steps:

[0307] Mapping the point cloud points in a preset range in the three-dimensional point cloud data from a three-dimensional coordinate system to a two-dimensional coordinate system according to a preset mapping relationship;

[0308] Obtaining an overview map according to the positions of the point cloud points in the two-dimensional coordinate system and the height data of the point cloud points;

[0309] Obtaining an intensity map according to the positions of the point cloud points in the two-dimensional coordinate system and the intensity data of the point cloud points;

[0310] Obtaining a density map according to the density of the point cloud points in the overview map;

[0311] Merging the overview map, the intensity map, and the density map to obtain a two-dimensional image.

[0312] In one embodiment, the computer program, when executed by the processor, implements the following steps:

[0313] Determining pixel values corresponding to the point cloud points according to the height data of the point cloud points;

[0314] Constructing an overview map according to the positions of the point cloud points in the two-dimensional coordinate system and the pixel values corresponding to the point cloud points.

[0315] In one embodiment, the computer program, when executed by the processor, implements the following steps:

[0316] Determining pixel values corresponding to the point cloud points according to the intensity data of the point cloud points;

[0317] Constructing an intensity map according to the positions of the point cloud points in the two-dimensional coordinate system and the pixel values corresponding to the point cloud points.

[0318] In one embodiment, the computer program, when executed by the processor, implements the following steps:

[0319] determine the pixel arrangement in the density map according to the pixel arrangement in the bird's eye view;

[0320] calculate the density of the i-th pixel in the bird's eye view according to the number of point clouds in each pixel, the minimum value of the number of point clouds in each pixel, and the maximum value of the number of point clouds in each pixel; i is a positive integer;

[0321] determine the pixel value of each pixel according to the calculated density of each pixel;

[0322] construct the density map according to the pixel arrangement in the density map and the pixel value of each pixel.

[0323] In one embodiment, the computer program is executed by the processor to implement the following steps:

[0324] merge the bird's eye view, the intensity map and the density map as R, G and B channel images respectively to obtain a two-dimensional image.

[0325] In one embodiment, the computer program is executed by the processor to implement the following steps:

[0326] obtain a shooting image obtained by a monitoring camera shooting a target scene;

[0327] map the stereoscopic recognition region to the shooting image according to the correspondence between the radar coordinates and the camera coordinates to obtain a second recognition region in the shooting image where the target object is located, and take the image in the second recognition region as a monitoring image.

[0328] In one embodiment, the computer program is executed by the processor to implement the following steps:

[0329] respectively input the point cloud mapping image and the monitoring image into the pre-trained feature extraction network to obtain the mapping feature map and the monitoring feature map output by the feature extraction network.

[0330] In one embodiment, the computer program is executed by the processor to implement the following steps:

[0331] respectively search the mapping feature map and the monitoring feature map in the pre-set feature map database to obtain a plurality of first feature maps corresponding to the mapping feature map and a plurality of second feature maps corresponding to the monitoring feature map; the feature map database stores a plurality of candidate feature maps, and each candidate feature map has a corresponding object;

[0332] determine a target feature map according to the plurality of first feature maps and the plurality of second feature maps; wherein the object corresponding to the target feature map is the same object as the target object.

[0333] In one embodiment, the computer program, when executed by the processor, implements the following steps:

[0334] The first similarity between the mapping feature map and each of the plurality of candidate feature maps is calculated respectively, and the candidate feature map meeting a first preset condition is taken as the first feature map; the first preset condition includes that the first similarity is greater than a preset threshold.

[0335] The second similarity between the monitoring feature map and each of the plurality of candidate feature maps is calculated respectively, and the candidate feature map meeting a second preset condition is taken as the second feature map; the second preset condition includes that the second similarity is greater than a preset threshold.

[0336] In one embodiment, the computer program, when executed by the processor, implements the following steps:

[0337] The same feature maps are selected from the plurality of first feature maps and the plurality of second feature maps as the third feature maps;

[0338] The target similarity corresponding to each third feature map is calculated according to the first similarity and the second similarity corresponding to the third feature map;

[0339] The target similarities corresponding to the plurality of third feature maps are sorted, and the target feature map is determined according to the sorting result.

[0340] In one embodiment, the computer program, when executed by the processor, implements the following steps:

[0341] The first similarity and the second similarity are weighted and summed for each third feature map to obtain the target similarity.

[0342] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0343] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, but as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0344] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A re-identification method, characterized in that, The method comprises: acquiring a point cloud mapping image of a target object and acquiring a monitoring image of the target object; the point cloud mapping image is obtained by mapping three-dimensional point cloud data of the target object collected by a laser radar; the monitoring image is obtained by a monitoring camera; respectively extracting features of the point cloud mapping image and the monitoring image to obtain a mapping feature map corresponding to the point cloud mapping image and a monitoring feature map corresponding to the monitoring image; based on the mapping feature map and the monitoring feature map, performing re-identification processing on the target object; wherein the acquiring of the point cloud mapping image of the target object comprises: acquiring the three-dimensional point cloud data obtained by scanning a target scene by the laser radar; the target object is located in the target scene; performing mapping processing on the three-dimensional point cloud data to obtain a two-dimensional image; determining a three-dimensional recognition area where the target object is located in the three-dimensional point cloud data by using a target detection algorithm; mapping the three-dimensional recognition area to the two-dimensional image to obtain a first recognition area where the target object is located in the two-dimensional image, and taking an image in the first recognition area as the point cloud mapping image.

2. The method of claim 1, wherein, The mapping processing on the three-dimensional point cloud data to obtain a two-dimensional image comprises: mapping point cloud points in a preset range in the three-dimensional point cloud data from a three-dimensional coordinate system to a two-dimensional coordinate system according to a preset mapping relationship; obtaining an aerial view according to positions of the point cloud points in the two-dimensional coordinate system and height data of the point cloud points; obtaining an intensity map according to the positions of the point cloud points in the two-dimensional coordinate system and intensity data of the point cloud points; obtaining a density map according to densities of point cloud points in the aerial view; performing merging processing on the aerial view, the intensity map and the density map to obtain the two-dimensional image.

3. The method of claim 2, wherein, The obtaining of the aerial view according to the positions of the point cloud points in the two-dimensional coordinate system and the height data of the point cloud points comprises: determining pixel values corresponding to the point cloud points according to the height data of the point cloud points; constructing the aerial view according to the positions of the point cloud points in the two-dimensional coordinate system and the pixel values corresponding to the point cloud points.

4. The method of claim 2, wherein, The obtaining of the intensity map according to the positions of the point cloud points in the two-dimensional coordinate system and the intensity data of the point cloud points comprises: determining pixel values corresponding to the point cloud points according to the intensity data of the point cloud points; constructing the intensity map according to the positions of the point cloud points in the two-dimensional coordinate system and the pixel values corresponding to the point cloud points.

5. The method of claim 2, wherein, The obtaining of the density map according to the densities of point cloud points in the aerial view comprises: determining pixel point arrangement in the density map according to pixel point arrangement in the aerial view; calculating a density of an i-th pixel point in the aerial view according to a number of point cloud points of the i-th pixel point, a minimum value of the number of point cloud points of each pixel point and a maximum value of the number of point cloud points of each pixel point; i is a positive integer; determining pixel values of the pixel points according to the calculated densities of the pixel points; constructing the density map according to the pixel point arrangement in the density map and the pixel values of the pixel points.

6. The method of claim 2, wherein, The merging processing of the bird's eye view, the intensity map and the density map obtains the two-dimensional image, including: The bird's eye view, the intensity map and the density map are merged as R, G and B three channel images to obtain the two-dimensional image.

7. The method of claim 1, wherein, The monitoring image of the target object is obtained, including: A shooting image obtained by the monitoring camera shooting the target scene is obtained. According to a preset corresponding relationship between radar coordinates and camera coordinates, the stereoscopic recognition region is mapped into the shooting image to obtain a second recognition region in which the target object is located in the shooting image, and an image in the second recognition region is taken as the monitoring image.

8. The method of claim 1, wherein, The feature extraction is performed on the point cloud mapping image and the monitoring image respectively to obtain a mapping feature map corresponding to the point cloud mapping image and a camera feature map corresponding to the camera image, including: The point cloud mapping image and the monitoring image are respectively input into a pre-trained feature extraction network to obtain the mapping feature map and the monitoring feature map output by the feature extraction network.

9. The method of claim 1, wherein, The re-identification processing is performed on the target object based on the mapping feature map and the monitoring feature map, including: According to the mapping feature map and the monitoring feature map, a plurality of first feature maps corresponding to the mapping feature map and a plurality of second feature maps corresponding to the monitoring feature map are obtained by searching in a pre-set feature map database; the feature map database stores a plurality of candidate feature maps, and each candidate feature map has a corresponding object. A target feature map is determined according to the plurality of first feature maps and the plurality of second feature maps; wherein the object corresponding to the target feature map is the same object as the target object.

10. The method of claim 9, wherein, The searching in the pre-set feature map database according to the mapping feature map and the monitoring feature map to obtain the plurality of first feature maps corresponding to the mapping feature map and the plurality of second feature maps corresponding to the monitoring feature map, including: First similarities between the mapping feature map and the plurality of candidate feature maps are calculated respectively, and a candidate feature map with a first similarity meeting a first preset condition is taken as the first feature map; the first preset condition includes that the first similarity is greater than a first preset threshold. Second similarities between the monitoring feature map and the plurality of candidate feature maps are calculated respectively, and a candidate feature map with a second similarity meeting a second preset condition is taken as the second feature map; the second preset condition includes that the second similarity is greater than a second preset threshold.

11. The method of claim 9, wherein, The determination of the target feature map according to the plurality of first feature maps and the plurality of second feature maps, including: Identical feature maps are screened from the plurality of first feature maps and the plurality of second feature maps as third feature maps; Target similarities corresponding to each of the third feature maps are calculated according to first similarities and second similarities corresponding to each of the third feature maps; The target similarities corresponding to the plurality of third feature maps are sorted, and the target feature map is determined according to a sorting result.

12. The method of claim 11, wherein, The target similarity of each third feature map is calculated according to the first similarity and the second similarity corresponding to each third feature map, and the target similarity of each third feature map is calculated according to the first similarity and the second similarity corresponding to each third feature map, and the target similarity of each third feature map is calculated according to the first similarity and the second similarity corresponding to each third feature map. The first similarity and the second similarity are weighted and summed to obtain the target similarity for each third feature map.

13. A re-identification method, characterized by, The method comprises: Obtaining a point cloud mapping image of a target object; the point cloud mapping image is obtained by mapping three-dimensional point cloud data of the target object collected by a laser radar; Feature extraction is performed on the point cloud mapping image to obtain a mapping feature map corresponding to the point cloud mapping image; Based on the mapping feature map, the target object is re-identified; Wherein, the point cloud mapping image of the target object comprises: The three-dimensional point cloud data obtained by scanning the target scene by the laser radar; the target object is located in the target scene; Mapping processing is performed on the three-dimensional point cloud data to obtain a two-dimensional image; A three-dimensional recognition area where the target object is located in the three-dimensional point cloud data is determined by using a target detection algorithm; Map the three-dimensional recognition area to the two-dimensional image to obtain a first recognition area where the target object is located in the two-dimensional image, and take the image in the first recognition area as the point cloud mapping image.

14. A feature map database, characterized by, The feature map database stores a plurality of candidate feature maps, and each candidate feature map has a corresponding object. The feature map database is used for re-identifying a target object based on a mapping feature map.

15. A re-identification apparatus characterized by comprising: The device comprises: An image acquisition module is configured to acquire a point cloud mapping image of a target object and a monitoring image of the target object; the point cloud mapping image is obtained by mapping three-dimensional point cloud data of the target object collected by a laser radar; and the monitoring image is obtained by a monitoring camera; A feature extraction module is configured to perform feature extraction on the point cloud mapping image and the monitoring image respectively to obtain a mapping feature map corresponding to the point cloud mapping image and a monitoring feature map corresponding to the monitoring image; A re-identification module is configured to perform re-identification processing on the target object based on the mapping feature map and the monitoring feature map; The image acquisition module comprises: A point cloud data acquisition submodule is configured to acquire the three-dimensional point cloud data obtained by scanning a target scene by the laser radar; the target object is located in the target scene; A mapping submodule is configured to perform mapping processing on the three-dimensional point cloud data to obtain a two-dimensional image; A region detection submodule is configured to determine a three-dimensional recognition area where the target object is located in the three-dimensional point cloud data by using a target detection algorithm; A point cloud mapping image acquisition submodule is configured to map the three-dimensional recognition area to the two-dimensional image to obtain a first recognition area where the target object is located in the two-dimensional image, and take the image in the first recognition area as the point cloud mapping image.

16. A re-identification apparatus characterized by comprising: The device comprises: An image acquisition module is configured to acquire a point cloud mapping image of a target object; the point cloud mapping image is obtained by mapping three-dimensional point cloud data of the target object collected by a laser radar; The feature extraction module is configured to perform feature extraction on the point cloud mapping image to obtain a mapping feature map corresponding to the point cloud mapping image. The re-identification module is configured to perform re-identification processing on the target object based on the mapping feature map. The image acquisition module comprises: The point cloud data acquisition submodule is configured to acquire the three-dimensional point cloud data obtained by scanning a target scene by the laser radar; the target object is located in the target scene. The mapping submodule is configured to perform mapping processing on the three-dimensional point cloud data to obtain a two-dimensional image. The region detection submodule is configured to determine a three-dimensional recognition region in which the target object is located in the three-dimensional point cloud data by using a target detection algorithm. The point cloud mapping image acquisition submodule is configured to map the three-dimensional recognition region into the two-dimensional image to obtain a first recognition region in which the target object is located in the two-dimensional image, and take an image in the first recognition region as the point cloud mapping image. 17.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-16. The processor executes the computer program to implement the steps of the method in any one of claims 1 to 13.

18. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 13.

Images