Identity binding method and apparatus, terminal device, and storage medium

Abstract

The application is suitable for the technical field of data processing, and provides an identity binding method and device, terminal equipment and storage medium, wherein the method first acquires point cloud data and image data of a target area, then tracks a target object in the target area according to the point cloud data to generate tracking information corresponding to the target object, then determines positioning information of the target object and identity information of the target object according to the image data, and finally fuses the tracking information and the positioning information of the target object, and binds the fused result and the identity information. Thus, on the basis of fusing the tracking information and the positioning information of the target object, the identity binding of the target object is further completed, so that each object in the target area corresponds to unique identity information, which facilitates on-site management. In addition, the tile map is also used to realize accurate positioning of the target object.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, and in particular relates to an identity binding method, apparatus, terminal device and storage medium. Background Technology

[0002] In specific road scenarios such as urban traffic intersections, highway toll stations, and tunnel entrances and exits, various sensing devices (such as lidar and cameras) are usually installed to identify targets such as vehicles and pedestrians, which helps to improve traffic management efficiency.

[0003] In related technologies, different types of sensing data may not provide consistent identification information for the same target object. This is especially true when there are many objects in the sensing area, which can easily lead to confusion in identification information and hinder on-site management. Summary of the Invention

[0004] This application provides an identity binding method, device, terminal equipment, and storage medium, which can solve the problem that the identity information identified by different types of sensing data for the same target object is not consistent. In particular, when there are many objects in the sensing area, the identity information is easily confused, which is not conducive to on-site management.

[0005] The first aspect of this application provides an identity binding method, including:

[0006] Acquire point cloud data and image data of the target area;

[0007] The target objects within the target area are tracked based on point cloud data to generate tracking information corresponding to the target objects.

[0008] Based on image data and tile maps, the location information of the target object is determined, wherein the image data and tile maps have a pre-established mapping relationship;

[0009] Determine the identity information of the target object;

[0010] The tracking information and location information of the target object are fused together, and the fused result is bound to the identity information.

[0011] Optionally, in one possible implementation of the first aspect, the above-mentioned tracking of target objects within the target area based on point cloud data to generate tracking information corresponding to the target objects includes:

[0012] Point cloud detection algorithms are used to detect target objects in point cloud data to generate point cloud detection information corresponding to the target objects.

[0013] Tracking algorithms are used to track point cloud detection information in order to generate tracking information corresponding to the target object.

[0014] Optionally, in one possible implementation of the first aspect, the point cloud detection algorithm mentioned above is any one of the VoxelNet algorithm, SECOND algorithm, PointPillars algorithm, and PV-RCNN algorithm.

[0015] Optionally, in one possible implementation of the first aspect, the above tracking algorithm is a 3D-Sort tracking algorithm.

[0016] Optionally, in one possible implementation of the first aspect, the image data includes a plurality of first pixels, and determining the location information of the target object based on the image data and the tile map includes:

[0017] Based on the tile map, a perspective transformation is performed on the image data so that the view plane of the image data is consistent with the view plane of the tile map. The tile map includes multiple second pixels, and each second pixel corresponds to a location information.

[0018] In the image data after perspective transformation, determine the first pixel points corresponding to each second pixel point;

[0019] Based on the positioning information corresponding to each second pixel, determine the positioning information of each first pixel corresponding to each second pixel;

[0020] The location information of the target object is determined based on the location information of the first pixel corresponding to each second pixel.

[0021] Optionally, in one possible implementation of the first aspect, the above-mentioned perspective transformation of the image data based on the tile map, so that the view plane of the image data coincides with the view plane of the tile map, includes:

[0022] Four first feature pixels are identified, where the four first feature pixels are the first pixels at any four different positions in the tile image;

[0023] In each second pixel, identify four second feature pixels that correspond one-to-one with the four first feature pixels;

[0024] Generate a perspective transformation matrix based on four first feature pixels and four second feature pixels;

[0025] Based on the perspective transformation matrix, the image data is transformed by perspective so that the view plane of the image data is consistent with the view plane of the tile image.

[0026] Optionally, in one possible implementation of the first aspect, the aforementioned identity binding method further includes:

[0027] Acquire video data for the target area;

[0028] Video detection algorithms are used to process video data to determine whether a target object exists within the target area.

[0029] Optionally, in one possible implementation of the first aspect, the aforementioned video detection algorithm is either the Faster R-CNN algorithm or the YOLO algorithm.

[0030] Optionally, in one possible implementation of the first aspect, the video data of the target area is acquired by a camera, and the identification information of the target object includes:

[0031] Video data is processed using video detection algorithms to generate image detection boxes corresponding to the target objects. The image detection boxes and capture commands are then sent to the camera, enabling the camera to identify the target objects within the image detection boxes based on the capture commands, thereby determining the identity information of the target objects.

[0032] Optionally, in one possible implementation of the first aspect, the type of the target object is a vehicle, and the identity information of the target object is generated based on the license plate information of the target object.

[0033] Optionally, in one possible implementation of the first aspect, the fusion of the target object's tracking information and positioning information includes:

[0034] Convert the positioning information to the preset point cloud coordinate system corresponding to the tracking information;

[0035] Determine the distance between the location information and the tracking information;

[0036] When the distance between the positioning information and the tracking information is less than a preset distance threshold, the tracking information and positioning information of the target object are fused based on the point cloud coordinate system.

[0037] Optionally, in one possible implementation of the first aspect, when the distance between the positioning information and the tracking information is less than a preset distance threshold, the tracking information and positioning information of the target object are fused based on the point cloud coordinate system, including:

[0038] When the distance between the location information and the tracking information is less than a preset distance threshold, determine the intersection-over-union ratio between the visual detection box corresponding to the location information and the point cloud detection box corresponding to the tracking information.

[0039] When the cross-union ratio (CUNR) is greater than the preset CUNR threshold, the tracking information and positioning information of the target object are fused based on the point cloud coordinate system.

[0040] A second aspect of this application provides an identity binding device, comprising:

[0041] The acquisition module is used to acquire point cloud data and image data of the target area;

[0042] The generation module is used to track target objects within the target area based on point cloud data, so as to generate tracking information corresponding to the target objects;

[0043] The mapping module is used to determine the location information of the target object based on image data and tile map, wherein the image data and tile map have a pre-established mapping relationship;

[0044] The identification module is used to determine the identity information of the target object;

[0045] The binding module is used to fuse the tracking information and positioning information of the target object, and bind the fused result with the identity information.

[0046] A third aspect of this application provides a terminal device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the identity binding method of the first aspect described above.

[0047] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the identity binding method described in the first aspect.

[0048] The fifth aspect of this application provides a computer program product that, when run on a terminal device, causes the terminal device to execute the identity binding method described in the first aspect.

[0049] The beneficial effects of this application's embodiments compared to existing technologies are as follows: This application discloses an identity binding method, apparatus, terminal device, and storage medium. The method first acquires point cloud data and image data of a target area. Then, it tracks target objects within the target area based on the point cloud data to generate tracking information corresponding to the target objects. Next, it determines the location information and identity information of the target objects based on the image data and tile maps. Finally, it fuses the tracking information and location information of the target objects and binds the fused result with the identity information. Thus, by fusing the target object's tracking and location information, the identity binding of the target objects is completed, ensuring that each object within the target area has a unique identity, facilitating on-site management. Furthermore, it utilizes tile maps to achieve precise positioning of the target objects. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating an identity binding method provided in Embodiment 1 of this application;

[0052] Figure 2 This is a schematic diagram of a tile map containing a target area provided in Embodiment 1 of this application;

[0053] Figure 3 This is a schematic diagram of image data captured by a camera according to Embodiment 1 of this application;

[0054] Figure 4 This is a flowchart illustrating a method for determining whether a target object exists within a target area, as provided in Embodiment 2 of this application.

[0055] Figure 5 This is a schematic diagram of the structure of an identity binding device provided in Embodiment 3 of this application;

[0056] Figure 6 This is a schematic diagram of the structure of a terminal device provided in Embodiment 4 of this application. Detailed Implementation

[0057] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0058] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0059] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0060] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0061] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0062] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0063] It should be understood that the sequence number of each step in this embodiment does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0064] In related technologies, different types of sensing data may not provide consistent identification information for the same target object. This is especially true when there are many objects in the sensing area, which can easily lead to confusion in identification information and hinder on-site management.

[0065] In view of this, embodiments of this application provide an identity binding method, apparatus, terminal device, and storage medium. By fusing target object tracking information and positioning information, the identity binding of the target object is completed, ensuring that each object within the target area has a unique identity, facilitating on-site management. Furthermore, tile mapping is utilized to achieve precise positioning of the target object.

[0066] The following examples illustrate the application scenarios of the identity binding method provided in this application. This application can be applied to vehicle capture scenarios at checkpoints, such as urban traffic intersections, highway toll stations, and tunnel entrances and exits, to capture, process, and record passing motor vehicles and drivers. It is primarily used to detect and identify vehicle information, including license plates, vehicle colors, and vehicle types. Utilizing the identity binding method provided in this application, vehicle identity information identified from various sensor data can be unified, thereby quickly and accurately binding vehicle body information with identity information. This facilitates road traffic management, assists in handling traffic accidents, and improves traffic management efficiency.

[0067] To illustrate the technical solution of this application, specific embodiments are described below.

[0068] Reference Figure 1 The diagram illustrates a flowchart of an identity binding method provided in Embodiment 1 of this application. Figure 1 As shown, the identity binding method may include the following steps:

[0069] Step 101: Obtain point cloud data and image data of the target area.

[0070] It should be noted that the target area is a pre-set area that requires identity binding. Taking the vehicle capture scenario at the checkpoint as an example, the target area can be a fixed area in a specific scenario such as a toll station or a tunnel, and the type of target object can be a vehicle, a pedestrian, etc.

[0071] In this embodiment of the application, point cloud data and image data of the target area can be acquired by a sensing device. Specifically, a gantry can be set up in the target area, and a sensing device can be installed on the gantry so that the detection range of the sensing device can cover the entire target area.

[0072] As one possible implementation, in order to acquire point cloud data and image data of the target area, the sensing device may include a lidar and a camera. The lidar is set up to acquire point cloud data, and the camera is set up to acquire image data.

[0073] Step 102: Track the target object within the target area based on the point cloud data to generate tracking information corresponding to the target object.

[0074] In this embodiment, point cloud target perception technology can be used to detect and track target objects entering the target area. Furthermore, the point cloud target perception technology mainly includes two steps: point cloud detection and point cloud target tracking. Specifically, in one possible implementation of this embodiment, step 102 may include:

[0075] Point cloud detection algorithms are used to detect target objects in point cloud data to generate point cloud detection information corresponding to the target objects.

[0076] Tracking algorithms are used to track point cloud detection information in order to generate tracking information corresponding to the target object.

[0077] It should be noted that point cloud reading is required before point cloud detection can be performed on the point cloud sensing device. Point cloud reading involves parsing the point cloud data using the corresponding output protocol of each type of point cloud sensing device. Taking LiDAR as an example, the corresponding output protocol is the LiDAR output protocol.

[0078] In order to detect the point cloud detection information of the target object, the point cloud data needs to be processed by the point cloud detection algorithm. As one possible implementation of this application, the point cloud detection algorithm can be any one of the VoxelNet algorithm, SECOND algorithm, PointPillars algorithm and PV-RCNN algorithm. This application does not limit the specific algorithm of the point cloud detection algorithm.

[0079] In this embodiment, target tracking involves establishing the positional relationship of the target to be tracked within a continuous data sequence to obtain the target's complete motion trajectory. Typically, given the target's positional features from the previous frame, the position and bounding box size of the target are predicted in the next frame. Specifically, the tracking algorithm can be a 3D-Sort tracking algorithm, etc., and this application does not limit the specific algorithm used.

[0080] Step 103: Determine the location information of the target object based on the image data and tile map.

[0081] In this process, the image data and the tile map are pre-mapped.

[0082] In this embodiment of the application, the image data is composed of multiple pixels, where each pixel only corresponds to pixel coordinates and does not correspond to positioning information itself. Therefore, before determining the positioning information of the target object, it is necessary to use the tile map containing positioning information to determine the positioning information corresponding to some pixels in the image data. Then, the positioning information of the target object can be indirectly obtained based on the pixel with positioning information where the target object is located in the image data.

[0083] It should be noted that tile maps can be obtained from a database query or generated in advance through data collection and operations such as 3D modeling and image rendering. Each point in a tile map contains latitude and longitude information. However, since the pixels in a tile map may not be identical to the pixels in the collected image data, there is no one-to-one correspondence between the pixels in the image data and the pixels in the tile map.

[0084] Furthermore, based on a tile map containing three-dimensional information of the target region, and by performing perspective transformation on the image data, the first pixel points corresponding to each second pixel point can be determined. This further determines the first pixel point on the image data corresponding to the second pixel point where the target object is located. Combined with the mapping relationship, the precise positioning information of the target object can be obtained. That is, in one possible implementation of this application embodiment, step 103 may include:

[0085] Based on the tile map, a perspective transformation is performed on the image data so that the view plane of the image data is consistent with the view plane of the tile map. The tile map includes multiple second pixels, and each second pixel corresponds to a location information.

[0086] In the image data after perspective transformation, determine the first pixel points corresponding to each second pixel point;

[0087] Based on the positioning information corresponding to each second pixel, determine the positioning information of each first pixel corresponding to each second pixel;

[0088] The location information of the target object is determined based on the location information of the first pixel corresponding to each second pixel.

[0089] The essence of perspective transformation is to project the current image onto a new view plane.

[0090] It should be noted that since the pixels in the image data do not correspond one-to-one with the pixels in the tile image, it is impossible to determine the location information of all first pixels in the image data. This embodiment first determines each first pixel corresponding to a second pixel, then determines the location information of each first pixel corresponding to a second pixel, and finally obtains the location information of the target object based on the first pixel with corresponding location information.

[0091] It is important to understand that when a target object occupies only one first pixel with corresponding positioning information in the image data, the positioning information corresponding to that first pixel is directly determined as the positioning information of the target object. When a target object occupies more than one first pixel with corresponding positioning information in the image data, the positioning information corresponding to the first pixel in the middle position can be determined as the positioning information of the target object.

[0092] Furthermore, since perspective transformation requires first determining four sets of corresponding pixels on the image data and the tile image respectively, and then constructing a perspective transformation matrix, and then obtaining the mapping relationship between the image data and the tile image through the perspective transformation matrix, that is, in one possible implementation of this application embodiment, the above-mentioned perspective transformation of the image data based on the tile image to make the view plane of the image data consistent with the view plane of the tile image may include:

[0093] Four first feature pixels are identified, where the four first feature pixels are the first pixels at any four different positions in the tile image;

[0094] In each second pixel, identify four second feature pixels that correspond one-to-one with the four first feature pixels;

[0095] Generate a perspective transformation matrix based on four first feature pixels and four second feature pixels;

[0096] Based on the perspective transformation matrix, the image data is transformed by perspective so that the view plane of the image data is consistent with the view plane of the tile image.

[0097] As one possible implementation, in order to ensure the accuracy of the association between the first feature pixel and the second feature pixel, a fixed special object in the target area can be selected, and the first pixel of the special object in the image data can be determined as the first feature pixel, and the second pixel of the special object in the tile image can be determined as the second feature pixel.

[0098] As another possible implementation, since the field of view captured by the camera is usually a matrix, the image data is usually also rectangular. Therefore, the four corner points of the image data can be directly determined as the first feature pixel points, and the four corner points of the target area contained in the tile image can be determined as the second feature pixel points. This setting is more accurate, convenient and fast.

[0099] For example, taking a vehicle capture scenario at a checkpoint as an example, see... Figure 2 and Figure 3 ,in Figure 2 For a tile map containing the target area, Figure 3 For image data captured by the camera, firstly, four pairs of pixels are found in the tile image and image data. Then, the perspective transformation matrix is ​​solved. Based on the perspective transformation matrix, the image data is transformed into the perspective of the tile image, generating the latitude and longitude information corresponding to the first pixel in the camera's field of view, i.e., the positioning information.

[0100] Step 104: Determine the identity information of the target object.

[0101] It should be noted that video detection algorithms such as Faster R-CNN or YOLO can be used to process the video data of the target area, generate image detection boxes corresponding to the target objects, and trigger the camera to take pictures. The target objects within the image detection boxes are then identified to obtain the identity information of the target objects. In one possible implementation of this application embodiment, determining the identity information of the target objects may include:

[0102] Video data is processed using video detection algorithms to generate image detection boxes corresponding to the target objects. The image detection boxes and capture commands are then sent to the camera, enabling the camera to identify the target objects within the image detection boxes based on the capture commands, thereby determining the identity information of the target objects.

[0103] Among them, the capture command can be the capture time. After receiving the capture command, the camera takes a picture when the capture time is reached, thereby obtaining the image data of the area where the image detection box is located, and then the identity information of the target object can be identified.

[0104] As one possible approach, the identity information of a target object can be determined by combining the type of the target object with the attributes specific to that type. Taking a vehicle capture scenario at a checkpoint as an example, the type of the target object can be a vehicle, and the attribute specific to a vehicle is its license plate number. Therefore, the license plate information of the target vehicle can be obtained through the aforementioned method for identifying the identity information of the target object, thereby generating the identity information of the target vehicle.

[0105] Step 105: Fuse the tracking information and positioning information of the target object, and bind the fused result with the identity information.

[0106] After obtaining the tracking information, positioning information, and identity information of the target object, the tracking information and positioning information are fused together. Based on the positioning information obtained in the previous steps, the identity of the target object within the target area is bound together. This unifies the object identity information identified by different types of perception data, making on-site management more convenient.

[0107] Furthermore, since the coordinate systems of image data and tile images are different, coordinate system transformation is required before fusing tracking information and positioning information. Therefore, positioning information can be transformed to the preset point cloud coordinate system corresponding to tracking information to achieve coordinate system synchronization. Then, by determining whether the distance between positioning information and tracking information in the preset point cloud coordinate system meets the requirements, fusion and identity binding are performed, improving the matching accuracy of positioning information and tracking information and the success rate of identity binding. This is one possible implementation in this application embodiment. Step 105 above may include:

[0108] Convert the positioning information to the preset point cloud coordinate system corresponding to the tracking information;

[0109] Determine the distance between the location information and the tracking information;

[0110] When the distance between the positioning information and the tracking information is less than a preset distance threshold, the tracking information and positioning information of the target object are fused based on the point cloud coordinate system.

[0111] The preset point cloud coordinate system is generated based on the tile map. The preset distance threshold can be set according to the actual application scenario, and this application does not impose any restrictions on it.

[0112] It should be noted that if the distance between the location information and the tracking information is greater than or equal to the preset distance threshold, the location information or tracking information may be inaccurate and needs to be re-determined.

[0113] Furthermore, since the tracking information of the target object can be in the form of a point cloud detection box, and the positioning information of the target object can be in the form of a visual detection box, in order to improve the matching accuracy of the positioning information and the tracking information and the success rate of identity binding, it is possible to determine whether the intersection over union (IoU) between the point cloud detection box and the visual detection box of the target object meets the requirements, and then perform fusion and identity binding. That is, in one possible implementation of this application embodiment, when the distance between the positioning information and the tracking information is less than a preset distance threshold, the fusion of the tracking information and the positioning information of the target object based on the point cloud coordinate system may include:

[0114] When the distance between the location information and the tracking information is less than a preset distance threshold, determine the intersection-over-union ratio between the visual detection box corresponding to the location information and the point cloud detection box corresponding to the tracking information.

[0115] When the cross-union ratio (CUNR) is greater than the preset CUNR threshold, the tracking information and positioning information of the target object are fused based on the point cloud coordinate system.

[0116] It should be noted that the preset intersection-union ratio (IURR) threshold can be set according to the actual application scenario, and this application does not impose any restrictions on it. If the IURR between the visual detection bounding box and the point cloud detection bounding box is greater than or equal to the preset IURR threshold, the localization or tracking information may be inaccurate and needs to be re-determined.

[0117] The identity binding method disclosed in the above embodiments of this application first acquires point cloud data and image data of the target area. Then, it tracks the target object within the target area based on the point cloud data to generate tracking information corresponding to the target object. Next, it determines the positioning information and identity information of the target object based on the image data. Finally, it fuses the tracking information and positioning information of the target object and binds the fused result with the identity information. Thus, by fusing the tracking and positioning information of the target object, the identity binding of the target object is completed, ensuring that each object within the target area has a unique identity, facilitating on-site management. Furthermore, it utilizes tile maps to achieve precise positioning of the target object.

[0118] In one possible implementation of this application, since the identity binding is triggered when a target object exists in the target area, the target object can be identified by visual target perception technology. Specifically, the camera acquires video data of the target area in real time, and then the video data can be processed by video detection algorithms to accurately and timely determine whether a target object exists in the target area.

[0119] See Figure 4 The diagram illustrates a flowchart of a method for determining whether a target object exists within a target area, as provided in Embodiment 2 of this application. Figure 4 As shown, this judgment method may include the following steps:

[0120] Step 401: Obtain video data for the target area.

[0121] In one possible implementation, in order to enable the camera's detection range to cover the entire target area, a gantry can be set up in the target area and a camera can be mounted on the gantry, thereby enabling the accurate detection of all objects entering the target area from outside the target area.

[0122] Step 402: Use a video detection algorithm to process the video data to determine whether there is a target object in the target area.

[0123] It should be noted that the embodiments of this application use visual target perception technology to detect objects entering the target area from outside the target area. The visual target perception technology specifically includes two steps: video reading and target detection. In the video reading step, the video data collected by the camera can be parsed through the corresponding video output protocol, and then the video data is processed by the video detection algorithm to perform target perception. After the target perception step, it can be determined whether there is a target object in the current target area.

[0124] As one possible implementation, the video detection algorithm can be Faster R-CNN, YOLO, etc. This application does not limit the specific algorithm for video detection.

[0125] The above embodiments of this application disclose a method for determining whether a target object exists within a target area. First, video data of the target area is acquired using a camera. Then, a video detection algorithm is used to process the video data to determine whether a target object exists within the target area. This method can accurately and timely determine whether a target object exists within the target area. Applying this method to an identity binding method can effectively improve the real-time performance of the identity binding method.

[0126] See Figure 5The diagram shows a structural schematic of an identity binding device provided in Embodiment 3 of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0127] The identity binding device may specifically include the following modules:

[0128] The acquisition module 501 is used to acquire point cloud data and image data of the target area.

[0129] The generation module 502 is used to track target objects within the target area based on point cloud data, so as to generate tracking information corresponding to the target objects.

[0130] The mapping module 503 is used to determine the location information of the target object based on the image data and tile map.

[0131] In this process, the image data and the tile map are pre-mapped.

[0132] The determination module 504 is used to determine the identity information of the target object.

[0133] The binding module 505 is used to fuse the tracking information and positioning information of the target object, and bind the fused result with the identity information.

[0134] The identity binding device disclosed in the above embodiments of this application first acquires point cloud data and image data of the target area. Then, it tracks the target object within the target area based on the point cloud data to generate tracking information corresponding to the target object. Next, it determines the positioning information and identity information of the target object based on the image data. Finally, it fuses the tracking information and positioning information of the target object and binds the fused result with the identity information. Thus, by fusing the tracking and positioning information of the target object, the identity binding of the target object is completed, ensuring that each object within the target area has a unique identity, facilitating on-site management. Furthermore, it utilizes tile mapping to achieve precise positioning of the target object.

[0135] In one possible implementation of Embodiment 3 of this application, the generation module 502 may specifically include the following sub-modules:

[0136] The first generation submodule is used to detect target objects in point cloud data using point cloud detection algorithms, so as to generate point cloud detection information corresponding to the target objects.

[0137] The second generation submodule is used to track the point cloud detection information using a tracking algorithm to generate tracking information corresponding to the target object.

[0138] In one possible implementation of Embodiment 3 of this application, the point cloud detection algorithm is any one of the VoxelNet algorithm, SECOND algorithm, PointPillars algorithm, and PV-RCNN algorithm.

[0139] In one possible implementation of Embodiment 3 of this application, the above tracking algorithm is a 3D-Sort tracking algorithm.

[0140] In one possible implementation of Embodiment 3 of this application, the image data includes a plurality of first pixels, and the mapping module 503 may specifically include the following sub-modules:

[0141] The perspective transformation submodule is used to perform perspective transformation on image data based on the tile map, so that the view plane of the image data is consistent with the view plane of the tile map.

[0142] The tile map includes multiple second pixels, and each second pixel corresponds to a location information.

[0143] The first determination submodule is used to determine each first pixel point corresponding to the second pixel point in the image data after perspective transformation.

[0144] The second determining submodule is used to determine the positioning information of each first pixel corresponding to each second pixel based on the positioning information corresponding to each second pixel.

[0145] The positioning determination submodule is used to determine the positioning information of the target object based on the positioning information of each first pixel corresponding to the second pixel.

[0146] In one possible implementation of Embodiment 3 of this application, the perspective transformation submodule may specifically include the following units:

[0147] The first determining unit is used to determine four first feature pixels.

[0148] Among them, the four first feature pixels are the first pixels at any four different positions in the tile image.

[0149] The second determining unit is used to determine four second feature pixels that correspond one-to-one with the four first feature pixels in each second pixel.

[0150] The first generation unit is used to generate a perspective transformation matrix based on four first feature pixels and four second feature pixels.

[0151] The perspective transformation unit is used to perform perspective transformation on the image data according to the perspective transformation matrix, so that the view plane of the image data is consistent with the view plane of the tile image.

[0152] In one possible implementation of Embodiment 3 of this application, the above-mentioned identity binding method may further include the following modules:

[0153] The first acquisition module is used to acquire video data of the target area.

[0154] The first judgment module is used to process video data using video detection algorithms to determine whether a target object exists within the target area.

[0155] In one possible implementation of Embodiment 3 of this application, the video detection algorithm is either the Faster R-CNN algorithm or the YOLO algorithm.

[0156] In one possible implementation of Embodiment 3 of this application, the video data of the target area is acquired by a camera, and the determination module 504 may specifically include the following sub-modules:

[0157] The third determination submodule is used to process video data using video detection algorithms to generate image detection boxes corresponding to the target object, and send the image detection boxes and capture instructions to the camera so that the camera can identify the target object within the image detection box according to the capture instructions, thereby determining the identity information of the target object.

[0158] In one possible implementation of Embodiment 3 of this application, the type of the target object is a vehicle, and the identity information of the target object is generated based on the license plate information of the target object.

[0159] In one possible implementation of Embodiment 3 of this application, the binding module 505 may specifically include the following sub-modules:

[0160] The first conversion submodule is used to convert the positioning information to the preset point cloud coordinate system corresponding to the tracking information.

[0161] The fourth determination submodule is used to determine the distance between the location information and the tracking information.

[0162] The binding submodule is used to fuse the tracking information and positioning information of the target object based on the point cloud coordinate system when the distance between the positioning information and the tracking information is less than a preset distance threshold.

[0163] In one possible implementation of Embodiment 3 of this application, the aforementioned binding submodule may specifically include the following units:

[0164] The third determining unit is used to determine the intersection-over-union ratio between the visual detection box corresponding to the positioning information and the point cloud detection box corresponding to the tracking information when the distance between the positioning information and the tracking information is less than a preset distance threshold.

[0165] The fusion unit is used to fuse the tracking information and positioning information of the target object based on the point cloud coordinate system when the cross-union ratio is greater than the preset cross-union ratio threshold.

[0166] The identity binding device disclosed in the above embodiments of this application first uses a camera to acquire video data of the target area, and then uses a video detection algorithm to process the video data to determine whether there is a target object in the target area. This allows for accurate and timely determination of whether there is a target object in the target area. Applying this determination method to the identity binding method can effectively improve the real-time performance of the identity binding method.

[0167] The identity binding device provided in this application embodiment can be applied in the foregoing method embodiment. For details, please refer to the description of the above method embodiment, which will not be repeated here.

[0168] Figure 6 This is a schematic diagram of the terminal device provided in Embodiment 4 of this application. Figure 6 As shown, the terminal device 600 of this embodiment includes: at least one processor 610 ( Figure 6 (Only one is shown) a processor, a memory 620, and a computer program 621 stored in the memory 620 and executable on the at least one processor 610, wherein the processor 610 executes the computer program 621 to implement the steps in the above-described identity binding method embodiments.

[0169] The terminal device 600 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. This terminal device may include, but is not limited to, a processor 610 and a memory 620. Those skilled in the art will understand that... Figure 6 This is merely an example of terminal device 600 and does not constitute a limitation on terminal device 600. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0170] The processor 610 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0171] In some embodiments, the memory 620 may be an internal storage unit of the terminal device 600, such as a hard disk or memory of the terminal device 600. In other embodiments, the memory 620 may be an external storage device of the terminal device 600, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the terminal device 600. Furthermore, the memory 620 may include both internal and external storage units of the terminal device 600. The memory 620 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 620 can also be used to temporarily store data that has been output or will be output.

[0172] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0173] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0174] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0175] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0176] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0177] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0178] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0179] The implementation of all or part of the processes in the methods of the above embodiments can also be accomplished by a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the various method embodiments described above.

[0180] The embodiments described above are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An identity binding method, characterized in that, include: Acquire point cloud data and image data of the target area; The target object within the target area is tracked based on the point cloud data to generate tracking information corresponding to the target object; Based on the image data and tile map, the location information of the target object is determined, wherein the image data and the tile map have a pre-established mapping relationship; the image data includes multiple first pixels; determining the location information of the target object based on the image data and tile map includes: performing a perspective transformation on the image data based on the tile map so that the view plane of the image data is consistent with the view plane of the tile map, wherein the tile map includes multiple second pixels, each second pixel corresponding to a location information; determining each first pixel corresponding to a second pixel in the perspective-transformed image data; determining the location information of each first pixel corresponding to a second pixel based on the location information corresponding to each second pixel; and determining the location information of the target object based on the location information of each first pixel corresponding to a second pixel. Determine the identity information of the target object; The tracking information and positioning information of the target object are fused together, and the fused result is bound to the identity information.

2. The identity binding method as described in claim 1, characterized in that, The step of tracking target objects within the target area based on the point cloud data to generate tracking information corresponding to the target objects includes: The target object in the point cloud data is detected using a point cloud detection algorithm to generate point cloud detection information corresponding to the target object; The point cloud detection information is tracked using a tracking algorithm to generate the tracking information corresponding to the target object.

3. The identity binding method as described in claim 2, characterized in that, The point cloud detection algorithm is any one of the following: VoxelNet algorithm, SECOND algorithm, PointPillars algorithm, and PV-RCNN algorithm.

4. The identity binding method as described in claim 2, characterized in that, The tracking algorithm is the 3D-Sort tracking algorithm.

5. The identity binding method as described in claim 1, characterized in that, The step of performing a perspective transformation on the image data based on the tile map, so that the view plane of the image data is consistent with the view plane of the tile map, includes: Four first feature pixels are determined, wherein the four first feature pixels are first pixels at any four different positions in the tile image; In each of the second pixels, four second feature pixels are identified that correspond one-to-one with the four first feature pixels. A perspective transformation matrix is ​​generated based on the four first feature pixels and the four second feature pixels; According to the perspective transformation matrix, the image data is subjected to perspective transformation so that the view plane of the image data is consistent with the view plane of the tile image.

6. The identity binding method as described in claim 1, characterized in that, Also includes: Acquire video data of the target area; The video data is processed using a video detection algorithm to determine whether the target object exists within the target area.

7. The identity binding method as described in claim 6, characterized in that, The video detection algorithm is either the Faster R-CNN algorithm or the YOLO algorithm.

8. The identity binding method as described in claim 6, characterized in that, The video data of the target area is collected by a camera, and determining the identity information of the target object includes: The video data is processed using the video detection algorithm to generate an image detection box corresponding to the target object. The image detection box and the capture command are then sent to the camera so that the camera can identify the target object within the image detection box according to the capture command, thereby determining the identity information of the target object.

9. The identity binding method as described in claim 8, characterized in that, The target object is a vehicle, and its identity information is generated based on its license plate information.

10. The identity binding method as described in claim 1, characterized in that, The process of fusing the tracking information and positioning information of the target object includes: The positioning information is converted to the preset point cloud coordinate system corresponding to the tracking information; Determine the distance between the location information and the tracking information; When the distance between the positioning information and the tracking information is less than a preset distance threshold, the tracking information and positioning information of the target object are fused based on the point cloud coordinate system.

11. The identity binding method as described in claim 10, characterized in that, When the distance between the positioning information and the tracking information is less than a preset distance threshold, the tracking information and positioning information of the target object are fused based on the point cloud coordinate system, including: When the distance between the positioning information and the tracking information is less than the preset distance threshold, the intersection-over-union ratio between the visual detection box corresponding to the positioning information and the point cloud detection box corresponding to the tracking information is determined. When the intersection-to-union ratio (CUIR) is greater than the preset CUIR threshold, the tracking information and positioning information of the target object are fused based on the point cloud coordinate system.

12. An identity binding device, characterized in that, include: The acquisition module is used to acquire point cloud data and image data of the target area; The generation module is used to track target objects within the target area based on the point cloud data, so as to generate tracking information corresponding to the target objects; A mapping module is used to determine the location information of the target object based on the image data and the tile map, wherein the image data and the tile map have a pre-established mapping relationship; the image data includes multiple first pixels, and the mapping module includes the following sub-modules: a perspective transformation sub-module, used to perform perspective transformation on the image data based on the tile map so that the view plane of the image data is consistent with the view plane of the tile map, wherein the tile map includes multiple second pixels, and each second pixel corresponds to a location information; a first determination sub-module, used to determine each first pixel corresponding to a second pixel in the perspective-transformed image data; a second determination sub-module, used to determine the location information of each first pixel corresponding to a second pixel based on the location information corresponding to each second pixel; and a location determination sub-module, used to determine the location information of the target object based on the location information of each first pixel corresponding to a second pixel. The determination module is used to determine the identity information of the target object; The binding module is used to fuse the tracking information and positioning information of the target object, and bind the fused result with the identity information.

13. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 11.

14. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 11.

Images