Cross-camera trajectory determination method and device, equipment and storage medium

Abstract

The application relates to a cross-camera trajectory determination method and device, equipment and a storage medium, and relates to the technical field of automobiles. The method comprises the following steps: acquiring a video stream photographed by a target camera, and determining a trajectory segment of an object under the target camera. Further, it is determined whether a target historical trajectory exists in a plurality of historical trajectories. In the plurality of trajectory segments included in the target historical trajectory, at least one trajectory segment includes a local identifier which is the same as a local identifier of the object. In the case where the target historical trajectory exists in the plurality of historical trajectories, the target historical trajectory is updated according to the trajectory segment of the object. In the case where the target historical trajectory does not exist in the plurality of historical trajectories, target similarity degrees of the trajectory segment of the object and each trajectory segment in a plurality of trajectory segments under other cameras are determined. In the case where a target trajectory segment exists in the plurality of trajectory segments, a cross-camera trajectory of the object is determined. Thus, the object in a plurality of single cameras is associated through cross cameras, and the trajectory of the object is determined.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automobiles, in particular to the technical field of automatic driving image recognition, and specifically relates to a cross-camera trajectory determination method and device, equipment and a storage medium. BACKGROUND

[0002] In the scenario of vehicle automatic driving, in order to obtain more abundant visual information, multiple cameras are usually required to take pictures at the same time, so the detection and tracking of the interested object need to be performed for each camera, and then the same object under the cross-camera is associated and connected to obtain the complete running trajectory of the object, so as to subsequently comprehensively process and realize continuous and stable tracking of the object across the cameras according to the images of each angle of the object.

[0003] However, due to a series of problems such as occlusion and image jitter in actual situations, the correlation accuracy of the cross-camera trajectory is insufficient, which brings hidden dangers to the driving safety in the scenario of automatic driving. SUMMARY

[0004] One of the purposes of the present application is to provide a cross-camera trajectory determination method, device, equipment and storage medium for realizing the association of objects in multiple single cameras and determining the trajectory of the object.

[0005] In order to achieve the above-mentioned purpose, the technical solution adopted by the present application is as follows:

[0006] According to the first aspect of the present application, a cross-camera trajectory determination method is provided, which comprises: a cross-camera trajectory determination device acquires a video stream taken by a target camera, the target camera being any one of multiple cameras deployed on the same vehicle; and determines a trajectory segment of an object under the target camera; the trajectory segment of the object comprises a local identifier, a detection box, a trajectory in a world coordinate system and an appearance feature of the object. Further, the cross-camera trajectory determination device determines whether there is a target historical trajectory in multiple historical trajectories, the historical trajectory being obtained by merging multiple trajectory segments, and the target historical trajectory comprising multiple trajectory segments, at least one of which comprises the same local identifier as that of the object; and in the case where there is a target historical trajectory in the multiple historical trajectories, the target historical trajectory is updated according to the trajectory segment of the object. Further, in the case where there is no target historical trajectory in the multiple historical trajectories, the cross-camera trajectory determination device determines the target similarity between the trajectory segment of the object and each trajectory segment of multiple trajectory segments under other cameras; and in the case where there is a target trajectory segment in the multiple trajectory segments, the cross-camera trajectory of the object is determined, the cross-camera trajectory comprising the trajectory segment of the object and the target trajectory segment, the target trajectory segment being the trajectory segment in the multiple trajectory segments having a target similarity greater than a preset similarity with the trajectory segment of the object.

[0007] According to the technical means, the method for determining a cross-camera track provided in the present application comprises the following steps: obtaining a track segment of an object; determining whether there is a target historical track in a plurality of historical tracks; in the case that there is a target historical track, updating the target historical track based on the track segment of the object; in the case that there is no target historical track, determining whether there is a track segment with a target similarity greater than a preset similarity to the track segment of the object in a plurality of track segments under other cameras; in the case that there is a track segment, merging the track segment of the object and the target track segment to determine the track of the object, i.e., to track the object.

[0008] In a possible implementation, the method for determining the track segment of the object under the target camera comprises the following steps: performing object detection on the video stream based on a preset algorithm to obtain a local identifier of the object and a detection box; determining a track of the object in a world coordinate system according to the local identifier of the object and the detection box; determining target images including the object in the video stream, the target images being a plurality of images with an object quality score higher than a preset score, the object quality score being used to represent the definition, completeness and orientation of the object in the image; determining an appearance feature of the object according to the target images; and determining the track segment of the object under the target camera according to the local identifier of the object, the detection box, the track in the world coordinate system and the appearance feature.

[0009] According to the technical means, the present application provides an implementation method for determining a track segment of an object, and the feature quality of the track segment can be guaranteed.

[0010] In a possible implementation, the method for determining whether there is a target historical track in a plurality of historical tracks comprises the following steps: determining a local identifier set, the local identifier set comprising a local identifier corresponding to each historical track in the plurality of historical tracks; in the case that the local identifier set comprises the local identifier of the object, determining that there is a target historical track in the plurality of historical tracks; and in the case that the local identifier set does not comprise the local identifier of the object, determining that there is no target historical track in the plurality of historical tracks.

[0011] In a possible implementation, the method for updating the target historical track based on the track segment of the object comprises the following steps: determining a lowest object quality score in object quality scores of a plurality of images in the target historical track; and in the case that the object quality score of the target image is greater than the lowest object quality score, replacing an image corresponding to the lowest object quality score according to the target image.

[0012] According to the technical means, the method for determining a cross-camera track provided in the present application can avoid determining the same object as different objects, which affects the tracking of the object.

[0013] In a possible implementation, the determining the target similarity between the trajectory segment of the object and each of the plurality of trajectory segments under the other cameras includes: determining an appearance similarity and a spatio-temporal similarity between the trajectory segment of the object and each of the plurality of trajectory segments under the other cameras; and determining the target similarity according to the appearance similarity and the spatio-temporal similarity.

[0014] According to the above technical means, in the cross-camera trajectory determination method provided by the present application, the correlation of the trajectory segment is evaluated from multiple dimensions based on the appearance similarity and the spatio-temporal similarity, and the accuracy of object association is improved.

[0015] In a possible implementation, the cross-camera trajectory determination method includes: in a case where the trajectory segment of the object is in the overlapping region of the target camera and the other cameras, determining the target similarity according to the appearance similarity, the spatio-temporal similarity, and a first preset weight, the first preset weight including a weight of the appearance similarity and a weight of the spatio-temporal similarity, and the weight of the appearance similarity being less than the weight of the spatio-temporal similarity.

[0016] According to the above technical means, in the cross-camera trajectory determination method provided by the present application, for the case where there is an overlapping region, the current tracking result of the same object under different cameras should have similar world coordinates, the historical tracking result should have similar trajectory start and end positions, similar trajectory shapes, speeds, directions, etc., and therefore the weight of the spatio-temporal similarity is set to be greater than the weight of the appearance similarity, which helps to improve the accuracy of the target similarity.

[0017] In a possible implementation, the cross-camera trajectory determination method includes: in a case where the trajectory segment of the object is not in the overlapping region of the target camera and the other cameras, determining the target similarity according to the appearance similarity, the spatio-temporal similarity, and a second preset weight, the second preset weight including a weight of the appearance similarity and a weight of the spatio-temporal similarity, and the weight of the appearance similarity being greater than the weight of the spatio-temporal similarity.

[0018] According to the above technical means, in the cross-camera trajectory determination method provided by the present application, for the case where there is no overlapping region, since there is no restriction in space and time, in order to improve the accuracy of the target similarity, the weight of the spatio-temporal similarity is set to be less than the weight of the appearance similarity, so as to ensure the accuracy of trajectory determination.

[0019] In a possible implementation, the cross-camera trajectory determination method includes: in a case where the target trajectory segment does not exist in the plurality of trajectory segments, updating the plurality of trajectory segments, and the plurality of trajectory segments after the update including the trajectory segment of the object.

[0020] According to the above technical means, in the cross-camera trajectory determination method provided by the present application, in the case that there is no target trajectory segment in the plurality of trajectory segments, i.e., the cross-camera trajectory determination device does not find a trajectory segment matching the trajectory segment of the object, the trajectory segment of the object is stored and determined as one of the plurality of trajectory segments, so that when the object appears again subsequently, the matching is performed to better determine the motion trajectory of the object.

[0021] In a possible implementation, the cross-camera trajectory determination method includes: in the case that any one of the plurality of historical trajectories is not updated for more than a preset time length, deleting the any one of the historical trajectories.

[0022] According to the above technical means, in the cross-camera trajectory determination method provided by the present application, the cross-camera trajectory determination device monitors the plurality of stored historical trajectories, and in the case that any one of the historical trajectories is not updated for more than a preset time length, i.e., the object corresponding to the historical trajectory has temporarily disappeared, the historical trajectory of the object is discarded, so as to avoid long-time occupation of the storage resource and the computing resource of the cross-camera trajectory determination device and reduce the resource consumption.

[0023] According to the second aspect provided by the present application, a cross-camera trajectory determination device is provided, which includes an acquisition unit, a determination unit, and a processing unit. The acquisition unit is configured to acquire a video stream captured by a target camera, the target camera being any one of a plurality of cameras deployed on the same vehicle. The determination unit is configured to determine a trajectory segment of an object under the target camera, the trajectory segment of the object including a local identifier, a bounding box, a trajectory in a world coordinate system, and an appearance feature. The determination unit is further configured to determine whether there is a target historical trajectory in a plurality of historical trajectories, the historical trajectory being obtained by merging a plurality of trajectory segments, and the target historical trajectory including a plurality of trajectory segments, at least one of which includes the same local identifier as the local identifier of the object. The processing unit is configured to update the target historical trajectory according to the trajectory segment of the object in the case that there is the target historical trajectory in the plurality of historical trajectories. The determination unit is further configured to determine a target similarity between the trajectory segment of the object and each trajectory segment of a plurality of trajectory segments under other cameras in the case that there is no target historical trajectory in the plurality of historical trajectories. The determination unit is further configured to determine a cross-camera trajectory of the object in the case that there is a target trajectory segment in the plurality of trajectory segments, the cross-camera trajectory including the trajectory segment of the object and the target trajectory segment, and the target trajectory segment being a trajectory segment in the plurality of trajectory segments having a target similarity greater than a preset similarity with the trajectory segment of the object.

[0024] In a possible implementation, the acquisition unit is further configured to perform object detection on the video stream based on a preset algorithm, to acquire the local identifier and the detection frame of the object; the determination unit is further configured to determine the trajectory of the object in the world coordinate system according to the local identifier and the detection frame of the object; to determine a target image including the object in the video stream, the target image being a plurality of images in which the object quality score of the object is higher than a preset score, the object quality score being used to represent the definition, completeness and orientation of the object in the image; to determine the appearance feature of the object according to the target image; and to determine the trajectory segment of the object under the target camera according to the local identifier, the detection frame, the trajectory of the object in the world coordinate system and the appearance feature of the object.

[0025] In a possible implementation, the determination unit is further configured to determine a local identifier set, the local identifier set including the local identifier corresponding to each historical trajectory in the plurality of historical trajectories; in a case where the local identifier of the object exists in the local identifier set, to determine that the target historical trajectory exists in the plurality of historical trajectories; and in a case where the local identifier of the object does not exist in the local identifier set, to determine that the target historical trajectory does not exist in the plurality of historical trajectories.

[0026] In a possible implementation, the determination unit is further configured to determine the lowest object quality score in the object quality scores of the plurality of images in the target historical trajectory; and the processing unit is further configured to replace the image corresponding to the lowest object quality score according to the target image in a case where the object quality score of the target image is greater than the lowest object quality score.

[0027] In a possible implementation, the determination unit is further configured to determine the appearance similarity and the spatio-temporal similarity between the trajectory segment of the object and each trajectory segment of the plurality of trajectory segments under the other cameras; and to determine the target similarity according to the appearance similarity and the spatio-temporal similarity.

[0028] In a possible implementation, the determination unit is further configured to determine the target similarity according to the appearance similarity, the spatio-temporal similarity and a first preset weight in a case where the trajectory segment of the object is within the overlapping region of the target camera and the other cameras, the first preset weight including a weight of the appearance similarity and a weight of the spatio-temporal similarity, the weight of the appearance similarity being less than the weight of the spatio-temporal similarity.

[0029] In a possible implementation, the determination unit is further configured to determine the target similarity according to the appearance similarity, the spatio-temporal similarity and a second preset weight in a case where the trajectory segment of the object is not within the overlapping region of the target camera and the other cameras, the second preset weight including a weight of the appearance similarity and a weight of the spatio-temporal similarity, the weight of the appearance similarity being greater than the weight of the spatio-temporal similarity.

[0030] In a possible implementation, the processing unit is further configured to update the plurality of trajectory segments in a case where the target trajectory segment does not exist in the plurality of trajectory segments, and the updated plurality of trajectory segments include the trajectory segment of the object.

[0031] In a possible implementation, the processing unit is further configured to delete any one of the plurality of historical trajectories in a case where the any one of the plurality of historical trajectories is not updated for a preset time length.

[0032] According to a third aspect provided in the present application, a cross-camera trajectory determination device is provided, which is deployed on a vehicle. The cross-camera trajectory determination device comprises a memory and a processor, which are coupled; the memory is configured to store computer program code, the computer program code comprising computer instructions; when the processor executes the computer instructions, the cross-camera trajectory determination device performs the cross-camera trajectory determination method provided in the first aspect and any possible implementation thereof.

[0033] According to a fourth aspect provided in the present application, a computer readable storage medium is provided, which stores instructions, when the instructions run on the cross-camera trajectory determination device, the cross-camera trajectory determination device performs the cross-camera trajectory determination method provided in the first aspect and any possible implementation thereof.

[0034] According to a fifth aspect provided in the present application, a vehicle is provided, which comprises the cross-camera trajectory determination device provided in the third aspect.

[0035] According to a sixth aspect provided in the present application, a computer program product is provided, which comprises computer instructions, when the computer instructions run on the cross-camera trajectory determination device, the cross-camera trajectory determination device performs the cross-camera trajectory determination method provided in the first aspect and any possible implementation thereof.

[0036] Therefore, the above technical features of the present application have the following beneficial effects:

[0037] (1) In the cross-camera trajectory determination method provided in the present application, after the trajectory segment of the object is obtained, firstly, it is determined whether the local identifier of the object is associated with any one of the plurality of historical trajectories, in a case where the local identifier of the object is associated with any one of the plurality of historical trajectories, the target historical trajectory is updated based on the trajectory segment of the object, and the trajectory determination of the object is realized; in a case where the local identifier of the object is not associated with any one of the plurality of historical trajectories, it is determined whether there is a trajectory segment in the plurality of trajectory segments under other cameras, which has a target similarity greater than a preset similarity with the trajectory segment of the object, in a case where there is, the trajectory segment of the object and the target trajectory segment are merged, and the trajectory determination of the object is realized, that is, the tracking of the object is realized.

[0038] (2) Based on the appearance similarity and the space-time similarity, the correlation of the track segments is evaluated from multiple dimensions, and the accuracy of the object association is improved.

[0039] (3) For the case where there is an overlapping area, the current tracking results of the same object in different cameras should have similar world coordinates, the historical tracking results should have similar track start and end positions, similar track shapes, speeds, directions, etc. Therefore, the weight of the space-time similarity is set to be greater than the appearance similarity, which helps to improve the accuracy of the target similarity.

[0040] (4) According to the above technical means, in the cross-camera track determination method provided by the application, for the case where there is no overlapping area, due to the lack of space-time restriction, in order to improve the accuracy of the target similarity, the weight of the space-time similarity is set to be less than the appearance similarity, which ensures the accuracy of the track determination.

[0041] (5) In the case where there is no target track segment in multiple track segments, i.e. the cross-camera track determination device does not find a track segment matching the track segment of the object, the track segment of the object is stored and determined as one of the multiple track segments, so that when the object appears again subsequently, the matching is performed to better determine the motion track of the object.

[0042] (6) The cross-camera track determination device monitors the stored multiple historical tracks, and in the case where any one of them is not updated for more than a predetermined time, it indicates that the object corresponding to the historical track has temporarily disappeared, and the historical track of the object is discarded, avoiding long-term occupation of the storage resources and the computing resources of the cross-camera track determination device, and reducing the resource consumption. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 is a structural schematic diagram of a cross-camera track determination system according to an example embodiment;

[0044] Figure 2 is a flowchart of a cross-camera track determination method according to an example embodiment;

[0045] Figure 3 is a flowchart of another cross-camera track determination method according to an example embodiment;

[0046] Figure 4 is a flowchart of another cross-camera track determination method according to an example embodiment;

[0047] Figure 5 is a structural schematic diagram of another cross-camera track determination system according to an example embodiment;

[0048] Figure 6is a flowchart of yet another cross-camera trajectory determination method according to an example embodiment;

[0049] Figure 7 is a block diagram of a cross-camera trajectory determination apparatus according to an example embodiment;

[0050] Figure 8 is a block diagram of a cross-camera trajectory determination device according to an example embodiment. DETAILED DESCRIPTION

[0051] Other advantages and effects of the present application can be easily understood by those skilled in the art from the description of the present application. The present application can also be implemented or applied through other different specific embodiments, and the details in the description can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, but not for limiting the protection scope of the present application.

[0052] It should be noted that the drawings provided in the following embodiments only schematically illustrate the basic concept of the present application, and only the components related to the present application are shown in the drawings, but not drawn according to the number, shape and size of the components in actual implementation. The shape, number and ratio of each component in actual implementation can be arbitrarily changed, and the layout pattern of the components can be more complex.

[0053] In the description of the embodiments, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this document only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone. In addition, "at least one" "multiple" means two or more. "First", "second", etc. do not limit the number and execution order, and "first", "second", etc. do not necessarily mean different.

[0054] In the related art, first, the tracking result of the obstacle of each single camera is obtained, the local identification of the obstacle is determined, after the trigger condition of the main camera for cross-camera obstacle fusion is met, the mapping relationship between the local identification and the global identification is established, the global identification is detected for association consistency (position consistency, contour consistency and volume consistency), and then matched according to the similarity, and finally the global identification of the same obstacle is determined.

[0055] In the related art, due to the complexity of the actual scene, the outlines of the same object imaged under different angle positions of the camera are inconsistent, and in actual practice, the object is blocked under one camera and not blocked under another camera, thereby causing the check of not meeting the outline consistency and the volume consistency, and being easily identified as two objects, and the correlation accuracy is low. Moreover, the method proposed in the related art is only applicable to the position consistency detection of the overlapping area, and cannot be applied to the object correlation in the non-overlapping area, and the accuracy of the determination of the object trajectory is poor.

[0056] To solve the above technical problems, the present application provides a cross-camera trajectory determination method, device, equipment and storage medium, the method comprising: a cross-camera trajectory determination device acquires a video stream photographed by a target camera, the target camera being any one of a plurality of cameras deployed on the same vehicle; and determines a trajectory segment of an object under the target camera; the trajectory segment of the object includes a local identifier, a detection box, a trajectory in a world coordinate system, and an appearance feature. Further, the cross-camera trajectory determination device determines whether there is a target historical trajectory in a plurality of historical trajectories, the historical trajectory being obtained by merging a plurality of trajectory segments, and at least one trajectory segment included in the target historical trajectory including the same local identifier as the local identifier of the object; and in the case that there is a target historical trajectory in the plurality of historical trajectories, the target historical trajectory is updated according to the trajectory segment of the object. Further, in the case that there is no target historical trajectory in the plurality of historical trajectories, the cross-camera trajectory determination device determines a target similarity between the trajectory segment of the object and each trajectory segment of a plurality of trajectory segments under other cameras; and in the case that there is a target trajectory segment in the plurality of trajectory segments, a cross-camera trajectory of the object is determined, the cross-camera trajectory including the trajectory segment of the object and the target trajectory segment, the target trajectory segment being a trajectory segment in the plurality of trajectory segments having a target similarity greater than a preset similarity with the trajectory segment of the object.

[0057] In this way, in the cross-camera trajectory determination method provided by the present application, after the trajectory segment of the object is acquired, it is first determined whether the local identifier of the object is associated with a historical trajectory in the plurality of historical trajectories that have been determined, in the case that there is an association, the target historical trajectory is updated based on the trajectory segment of the object, and the determination of the trajectory of the object is realized; in the case that there is no association, it is determined whether there is a trajectory segment in the plurality of trajectory segments under other cameras having a target similarity greater than a preset similarity with the trajectory segment of the object, in the case that there is, the trajectory segment of the object and the target trajectory segment are merged, and the determination of the trajectory of the object, i.e. the tracking of the object, is realized.

[0058] Figure 1 A cross-camera trajectory determination system is shown, the cross-camera trajectory determination method provided by the present application can be applied to the case where Figure 1The cross-camera trajectory determination system shown is used to realize cross-camera association of objects in multiple single cameras, and determine the trajectory of the objects. As shown in Figure 1 As shown, the cross-camera trajectory determination system 10 comprises a cross-camera trajectory determination device 11, a camera 12, and a camera 13.

[0059] The cross-camera trajectory determination device 11 is connected with the camera 12 and the camera 13 respectively, which can be connected in a wired manner or in a wireless manner, and the present application does not make specific limitation thereon.

[0060] The cross-camera trajectory determination device 11, the camera 12, and the camera 13 are deployed on a vehicle.

[0061] It should be noted that Figure 1 Two cameras, i.e. the camera 12 and the camera 13, are exemplarily shown, which are used to represent that the cross-camera trajectory determination method provided by the embodiments of the present application is applicable to the multi-camera scene on the vehicle, and does not constitute a limitation on the number of cameras. In the case of deploying 3, 4, 5, etc. cameras on the vehicle, the cross-camera trajectory determination method provided by the embodiments of the present application can also be applicable.

[0062] The camera 12 and the camera 13 can be used to shoot videos of corresponding angles based on the deployed positions, and can also be used to send the shot videos to the cross-camera trajectory determination device 11 in the form of video streams.

[0063] It should be noted that the camera 12 and the camera 13 can be depth cameras, i.e. can obtain RGB and corresponding depth maps. The operation and maintenance personnel of the cross-camera trajectory determination system 10 pre-sets the intrinsic and extrinsic parameters of the cameras, and time synchronization calibration.

[0064] The camera extrinsic parameter is a transformation matrix from the camera coordinate system to the vehicle coordinate system, and the origin of the vehicle coordinate system can be defined as the projection point of the vehicle center point to the ground. The world coordinate system can be the vehicle coordinate system at the initial position of the vehicle, and is fixed. When the object detection algorithm of the camera detects the regression box of the object, the lower edge end point of the regression box is selected as the image position of the object, and the position of the object in the world coordinate system is determined through coordinate transformation. The object is a target or obstacle related to autonomous driving, such as a pedestrian, a vehicle, etc., and the embodiments of the present application do not make specific limitation thereon.

[0065] The cross-camera trajectory determination device 11 can be used to obtain the video streams sent by the camera 12 and the camera 13.

[0066] The cross-camera trajectory determination device 11 can also be used to detect, instance segment, and track the object in the video stream, and determine the trajectory segment of the object under the camera. The trajectory segment of the object comprises local identification, a detection box, a trajectory in the world coordinate system, and an appearance feature of the object.

[0067] The cross-camera trajectory determination apparatus 11 can also be configured to determine whether the target historical trajectory exists in the plurality of historical trajectories.

[0068] The historical trajectory is obtained by merging a plurality of trajectory segments, and the target historical trajectory includes a plurality of trajectory segments, at least one of which includes a local identifier identical to the local identifier of the object.

[0069] The cross-camera trajectory determination apparatus 11 can also be configured to update the target historical trajectory according to the trajectory segment of the object in the case where the target historical trajectory exists in the plurality of historical trajectories.

[0070] The cross-camera trajectory determination apparatus 11 can also be configured to determine the target similarity of the trajectory segment of the object to each of the plurality of trajectory segments under other cameras in the case where the target historical trajectory does not exist in the plurality of historical trajectories.

[0071] The cross-camera trajectory determination apparatus 11 can also be configured to determine the cross-camera trajectory of the object in the case where the target trajectory segment exists in the plurality of trajectory segments.

[0072] The cross-camera trajectory includes the trajectory segment of the object and the target trajectory segment, and the target trajectory segment is a trajectory segment in the plurality of trajectory segments having a target similarity to the trajectory segment of the object greater than a preset similarity.

[0073] The cross-camera trajectory determination apparatus 11 can also be configured to store the historical trajectory and the unassociated trajectory segment.

[0074] Figure 2 FIG. 1 is a flow diagram of a cross-camera trajectory determination method according to some example embodiments. In some embodiments, the cross-camera trajectory determination method described above can be applied to the cross-camera trajectory determination apparatus 11 in the cross-camera trajectory determination system 10 as shown in FIG. 1. Hereinafter, the embodiments of the present application take the cross-camera trajectory determination method applied to the cross-camera trajectory determination apparatus 11 as an example to describe the cross-camera trajectory determination method described above. Figure 1

[0075] As shown in FIG. 1, the cross-camera trajectory determination method provided by the embodiments of the present application includes the following S201-S206. Figure 2

[0076] S201, the cross-camera trajectory determination apparatus acquires a video stream captured by a target camera.

[0077] The target camera is any one of a plurality of cameras deployed on the same vehicle.

[0078] ​​As a possible implementation manner, the multiple cameras deployed on the same vehicle start photographing after the vehicle starts, and based on a preset period, the photographed video is sent to the cross-camera trajectory determination apparatus in the form of a video stream.

[0079] Correspondingly, the cross-camera trajectory determination apparatus receives the video stream sent by each of the multiple cameras, and obtains the video stream photographed by the target camera.

[0080] It should be noted that the preset period can be set in the camera in advance by an operator of the cross-camera trajectory determination system, for example, can be 1 minute, 2 minutes, etc., and the embodiments of the present application do not make specific limitations.

[0081] S202, the cross-camera trajectory determination apparatus determines the trajectory segment of the object under the target camera.

[0082] The trajectory segment of the object includes the local identity, the detection box, the trajectory in the world coordinate system, and the appearance feature of the object.

[0083] As a possible implementation manner, the cross-camera trajectory determination apparatus uses the Mask R-CNN algorithm to perform object detection and segmentation on the video stream, to determine the local identity and the detection box of the object. Further, the cross-camera trajectory determination apparatus uses the FairMOT algorithm to track the detected object, to determine the trajectory of the object in the vehicle coordinate system, and then according to the coordinate conversion, to determine the trajectory of the object in the world coordinate system. Further, the cross-camera trajectory determination apparatus also determines the appearance feature of the object according to a preset number of object images in the video stream, and then determines the trajectory segment of the object under the target camera according to the local identity, the detection box, the trajectory in the world coordinate system, and the appearance feature of the object.

[0084] In some embodiments, in order to improve the accuracy of the appearance feature of the object, the cross-camera trajectory determination apparatus determines the trajectory segment of the object under the target camera, as shown in Figure 3 It also includes S2021-S2025.

[0085] S2021, the cross-camera trajectory determination apparatus performs object detection on the video stream based on a preset algorithm, to obtain the local identity and the detection box of the object.

[0086] It should be noted that the preset algorithm can be the Mask R-CNN algorithm, or the Faster R-CNN algorithm, or other algorithms for object detection, which can be set in the cross-camera trajectory determination apparatus in advance by an operator of the cross-camera trajectory determination system, and the embodiments of the present application do not make specific limitations.

[0087] S2022, the cross-camera trajectory determination apparatus determines the trajectory of the object in the world coordinate system according to the local identity and the detection box of the object.

[0088] As a possible implementation manner, the cross-camera trajectory determination apparatus determines the midpoint of the bottom edge of the detection frame as the position of the object based on the local identification of the object determined in step S2021 and the detection frame. Further, the cross-camera trajectory determination apparatus determines the trajectory of the object in the vehicle coordinate system based on the change of the position of the object in the vehicle coordinate system, and further determines the trajectory of the object in the world coordinate system according to the conversion matrix of the vehicle coordinate system and the world coordinate system.

[0089] S2023, the cross-camera trajectory determination apparatus determines a target image including the object in the video stream.

[0090] The target image is a plurality of images with object quality scores higher than a preset score, and the object quality score is used to represent the definition, completeness and orientation of the object in the image.

[0091] As a possible implementation manner, the cross-camera trajectory determination apparatus determines the image including the object in the video stream, and for each image, calculates the object quality score of the image based on the definition, completeness and orientation of the object in each image. Further, the cross-camera trajectory determination apparatus compares the object quality score of each image with a preset score, and determines the image with the object quality score greater than the preset score as the target image.

[0092] In some embodiments, the preset score can be determined by the cross-camera trajectory determination apparatus according to the object quality scores of each image obtained in descending order, and the object quality score of the image in the preset order is determined as the preset score, that is, the image before the preset order is determined as the target image.

[0093] It should be noted that the preset score can be determined by the cross-camera trajectory determination apparatus according to the object quality score of the image in a video stream, or can be set as a fixed value in the cross-camera trajectory determination apparatus by the operation and maintenance personnel of the cross-camera trajectory determination system, and the embodiments of the present application do not make specific limitation.

[0094] S2024, the cross-camera trajectory determination apparatus determines the appearance feature of the object according to the target image.

[0095] As a possible implementation manner, the cross-camera trajectory determination apparatus uses a preset convolutional neural network to extract the appearance feature of the object from the target image determined in step S2023.

[0096] It should be noted that the preset convolutional neural network can be set as a fixed value in the cross-camera trajectory determination apparatus by the operation and maintenance personnel of the cross-camera trajectory determination system, and the embodiments of the present application do not make specific limitation.

[0097] S2025、The cross-camera trajectory determination apparatus determines a trajectory segment of the object under the target camera according to the local identifier of the object, the bounding box, the trajectory in the world coordinate system, and the appearance feature.

[0098] S203、The cross-camera trajectory determination apparatus determines whether the target historical trajectory exists in the plurality of historical trajectories.

[0099] The historical trajectory is obtained by merging a plurality of trajectory segments, and the target historical trajectory includes a plurality of trajectory segments, at least one of which includes the same local identifier as the local identifier of the object.

[0100] As a possible implementation, the cross-camera trajectory determination apparatus determines a plurality of historical trajectories stored by the cross-camera trajectory determination apparatus, a trajectory segment corresponding to each historical trajectory, and a local identifier corresponding to the trajectory segment. Further, the cross-camera trajectory determination apparatus determines whether there is a local identifier identical to the local identifier of the object in the local identifier corresponding to each historical trajectory in the plurality of historical trajectories.

[0101] In the case where it is determined that there is a local identifier identical to the local identifier of the object, it is determined that the target historical trajectory exists in the plurality of historical trajectories; in the case where it is determined that there is no local identifier identical to the local identifier of the object, it is determined that the target historical trajectory does not exist in the plurality of historical trajectories.

[0102] In some embodiments, the cross-camera trajectory determination apparatus determines a local identifier set, wherein the local identifier set includes a local identifier corresponding to each historical trajectory in the plurality of historical trajectories. Further, the cross-camera trajectory determination apparatus determines whether the local identifier of the object exists in the local identifier set.

[0103] In the case where the local identifier of the object exists in the local identifier set, the cross-camera trajectory determination apparatus determines that the target historical trajectory exists in the plurality of historical trajectories; in the case where the local identifier of the object does not exist in the local identifier set, the cross-camera trajectory determination apparatus determines that the target historical trajectory does not exist in the plurality of historical trajectories.

[0104] S204、The cross-camera trajectory determination apparatus updates the target historical trajectory according to the trajectory segment of the object in the case where the target historical trajectory exists in the plurality of historical trajectories.

[0105] It should be noted that in the embodiments of the present application, each trajectory segment has a local identifier configured by the cross-camera trajectory determination apparatus, and each historical trajectory has a global identifier configured by the cross-camera trajectory determination apparatus.

[0106] As a possible implementation, in the case that the cross-camera trajectory determination apparatus determines that the target historical trajectory exists in the plurality of historical trajectories based on the step S203, the cross-camera trajectory determination apparatus merges the trajectory segment of the object with the target historical trajectory, and incorporates the appearance feature of the object into the feature of the target historical trajectory, so as to associate the recognized trajectory segment of the object with other trajectory segments.

[0107] In some embodiments, in order to guarantee the feature quality of the target historical trajectory, the cross-camera trajectory determination apparatus updates the target historical trajectory according to the trajectory segment of the object, such as Figure 4 As shown, S2041-S2042 are further included.

[0108] S2041, the cross-camera trajectory determination apparatus determines the lowest object quality score in the object quality scores of the plurality of images in the target historical trajectory.

[0109] As a possible implementation, the cross-camera trajectory determination apparatus determines the object quality scores of the images used to describe the object in the target historical trajectory, and determines the lowest object quality score and the image corresponding to the lowest object quality score.

[0110] S2042, in the case that the object quality score of the target image is greater than the lowest object quality score, the cross-camera trajectory determination apparatus replaces the image corresponding to the lowest object quality score with the target image.

[0111] As a possible implementation, the cross-camera trajectory determination apparatus judges the size of the object quality score of the target image and the lowest object quality score based on the target image determined in the step S2023 and the object quality score of the target image. Further, in the case that the object quality score of the target image is greater than the lowest object quality score, the cross-camera trajectory determination apparatus replaces the image corresponding to the lowest object quality score with the target image to describe the object in the target historical trajectory.

[0112] S205, in the case that the target historical trajectory does not exist in the plurality of historical trajectories, the cross-camera trajectory determination apparatus determines the target similarity between the trajectory segment of the object and each of the plurality of trajectory segments under other cameras.

[0113] As a possible implementation, in the case that the cross-camera trajectory determination apparatus determines that the target historical trajectory does not exist in the plurality of historical trajectories based on the step S203, the cross-camera trajectory determination apparatus respectively determines the target similarity between the trajectory segment of the object and each of the plurality of trajectory segments under other cameras, to obtain a plurality of target similarities corresponding to the plurality of trajectory segments under other cameras.

[0114] It should be noted that the target similarity can be a spatio-temporal similarity, an appearance similarity, and / or a weighted sum of the spatio-temporal similarity and the appearance similarity of the two trajectory segments.

[0115] In some embodiments, the cross-camera trajectory determination apparatus determines the target similarity of the trajectory segment of the object and the plurality of trajectory segments under the other cameras can include: determining, by the cross-camera trajectory determination apparatus, the appearance similarity and the spatio-temporal similarity of the trajectory segment of the object and each of the plurality of trajectory segments under the other cameras; and determining the target similarity according to the appearance similarity and the spatio-temporal similarity.

[0116] Specifically, in the case that the trajectory segment of the object is in the overlapping region of the target camera and the other cameras, the cross-camera trajectory determination apparatus determines the target similarity according to the appearance similarity, the spatio-temporal similarity, and a first preset weight.

[0117] The first preset weight includes a weight of the appearance similarity and a weight of the spatio-temporal similarity, and the weight of the appearance similarity is less than the weight of the spatio-temporal similarity.

[0118] For example, the first preset weight includes a weight of the appearance similarity of 0.2 and a weight of the spatio-temporal similarity of 0.8. If the appearance similarity is A and the spatio-temporal similarity is B, then the target similarity S = 0.2A + 0.8B.

[0119] It can be understood that, for the case of the overlapping region, the current tracking result of the same object under different cameras should have similar world coordinates, the historical tracking result should have similar trajectory start and end positions, similar trajectory shapes, speeds, directions, etc. Therefore, the weight of the spatio-temporal similarity is set to be greater than the weight of the appearance similarity, which helps to improve the accuracy of the target similarity, but the specific size of the weight is not limited in the embodiments of the present application.

[0120] In the case that the trajectory segment of the object is not in the overlapping region of the target camera and the other cameras, the cross-camera trajectory determination apparatus determines the target similarity according to the appearance similarity, the spatio-temporal similarity, and a second preset weight.

[0121] The second preset weight includes a weight of the appearance similarity and a weight of the spatio-temporal similarity, and the weight of the appearance similarity is greater than the weight of the spatio-temporal similarity.

[0122] For example, the second preset weight includes a weight of the appearance similarity of 0.65 and a weight of the spatio-temporal similarity of 0.35. If the appearance similarity is A and the spatio-temporal similarity is B, then the target similarity S = 0.65A + 0.35B.

[0123] It can be understood that, for the case where there is no overlapping area, in order to improve the accuracy of the target similarity, the weight of the space-time similarity should be set to be less than the weight of the appearance similarity due to the lack of space-time restriction, but the specific size of the weight is not limited in the embodiments of the present application.

[0124] It should be noted that the first preset weight and the second preset weight can be set in the cross-camera trajectory determination device by an operation and maintenance personnel of the cross-camera trajectory determination system in advance, and the embodiments of the present application do not make specific limitations.

[0125] In S206, the cross-camera trajectory determination device determines the cross-camera trajectory of the object when the target trajectory segment exists in the plurality of trajectory segments.

[0126] The cross-camera trajectory includes the trajectory segment of the object and the target trajectory segment, and the target trajectory segment is a trajectory segment in the plurality of trajectory segments that has a target similarity greater than a preset similarity with the trajectory segment of the object.

[0127] As a possible implementation manner, the cross-camera trajectory determination device determines whether there is a target similarity greater than a preset similarity based on the plurality of target similarities determined in the step S205. Further, when there is a target similarity greater than a preset similarity, the trajectory segment corresponding to the target similarity is determined as the target trajectory segment, the target trajectory segment and the trajectory segment of the object are combined to determine the cross-camera trajectory of the object, a global identifier is configured, and the appearance feature of the cross-camera trajectory of the object is determined by selecting a preset number of images with the highest object quality score from the target trajectory segment and the trajectory segment of the object.

[0128] In some embodiments, the cross-camera trajectory determination device updates the plurality of trajectory segments when the target trajectory segment does not exist in the plurality of trajectory segments, and the updated plurality of trajectory segments includes the trajectory segment of the object.

[0129] It can be understood that, when the target trajectory segment does not exist in the plurality of trajectory segments, i.e., when the cross-camera trajectory determination device does not find a trajectory segment matching the trajectory segment of the object, the trajectory segment of the object is stored and determined as one of the plurality of trajectory segments, so that the object can be matched and the motion trajectory of the object can be better determined when the object appears again subsequently.

[0130] In some embodiments, the cross-camera trajectory determination device deletes any one of the plurality of historical trajectories when the any one of the plurality of historical trajectories is not updated for more than a preset time length.

[0131] It can be understood that the cross-camera trajectory determination apparatus monitors the stored plurality of historical trajectories, and in the case that any one of the historical trajectories is not updated for more than a preset time length, it indicates that the object corresponding to the historical trajectory has temporarily disappeared, and the historical trajectory of the object is discarded, so as to avoid occupying the storage resource and the computing resource of the cross-camera trajectory determination apparatus for a long time, and to reduce the resource consumption.

[0132] It can be understood that in the cross-camera trajectory determination method provided in the application, after the trajectory segment of the object is acquired, it is first judged whether the local identifier of the object is associated with the historical trajectory in the plurality of historical trajectories that have been determined, in the case that there is an association, the target historical trajectory is updated based on the trajectory segment of the object, and the trajectory determination of the object is realized; in the case that there is no association, it is determined whether there is a trajectory segment with a target similarity greater than a preset similarity in the plurality of trajectory segments under other cameras, in the case that there is, the object trajectory segment and the target trajectory segment are merged, and the determination of the object trajectory is realized, that is, the tracking of the object is realized.

[0133] In one design, in combination with the cross-camera trajectory determination system 10 as shown in Figure 1 , the cross-camera trajectory determination apparatus 11 includes a single-camera object tracking module 111, an object quality evaluation module 112, a cross-camera association module 113, and an object archive module 114, as shown in Figure 5 .

[0134] The single-camera object tracking module 111 can be configured to acquire a plurality of video streams captured by a plurality of cameras, detect, instance segment, and track the object in each single-camera video stream, and acquire the local identifier (Local_id) and the trajectory segment of the object under the single camera.

[0135] The object quality evaluation module 112 can be configured to determine the object quality score of each image in the video stream captured by the single camera, and determine the features extracted from the target image with a higher score, which represent the appearance features of the trajectory segment of the object under the single camera.

[0136] The cross-camera association module 113 can be configured to associate the trajectory segments.

[0137] The object archive module 114 can be configured to maintain and update the global identifier (Global_id) and the local identifier (Local_id) of the object.

[0138] In combination with the cross-camera trajectory determination system as shown in Figure 5 , another flowchart of the cross-camera trajectory determination method provided in the embodiment of the application includes S1-S11, as shown in Figure 6 .

[0139] S1, the single-camera object tracking module acquires a plurality of single-camera tracking results.

[0140] S2, the object quality evaluation module scores the object quality of each single camera tracking result.

[0141] S3, the cross-camera association module determines whether the Global_id in the object profile module has the Local_id of the object's trajectory segment under the current multiple single cameras.

[0142] It should be noted that in the case where the Local_id of the object's trajectory segment under the current multiple single cameras exists, step S4 is performed, and in the case where the Local_id of the object's trajectory segment under the current multiple single cameras does not exist, step S5 is performed.

[0143] S4, the cross-camera association module determines that the Global_id with the same Local_id in the Local_id of the object's trajectory segment under the current multiple single cameras is merged.

[0144] S5, the cross-camera association module determines whether the object is in the overlapping area of the current single camera and other cameras.

[0145] It should be noted that in the case where it is determined that the object is in the overlapping area of the current single camera and other cameras, step S6 is performed, and in the case where it is determined that the object is not in the overlapping area of the current single camera and other cameras, step S8 is performed.

[0146] S6, the cross-camera association module determines the candidate trajectory segment.

[0147] Wherein, the candidate trajectory segment is the trajectory segment under the other camera that can shoot the current overlapping area.

[0148] S7, the cross-camera association module calculates the target similarity of different camera Local_id, and merges the trajectory segment according to the target similarity.

[0149] S8, the cross-camera association module determines the historical candidate trajectory segment.

[0150] Wherein, the historical candidate trajectory segment is the trajectory segment stored in the object profile module.

[0151] S9, the cross-camera association module calculates the target similarity of Local_id and the historical candidate trajectory segment, and merges or adds Global_id according to the target similarity.

[0152] S10, the object profile module updates the object profile according to the object quality score.

[0153] S11, the object profile module merges the low-quality profile.

[0154] It should be noted that the specific implementation of the cross-camera trajectory determination method by the modules in the cross-camera trajectory determination apparatus can refer to the description of the above embodiments of the present application, and will not be described here.

[0155] The above describes the solutions provided by the embodiments of the present application mainly from the perspective of methods. In order to implement the above functions, the cross-camera trajectory determination apparatus or the cross-camera trajectory determination device includes hardware structures and / or software modules corresponding to each function. Those skilled in the art should easily realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is implemented in hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0156] The embodiments of the present application can divide the functional modules of the cross-camera trajectory determination apparatus or the cross-camera trajectory determination device according to the above method, for example, the cross-camera trajectory determination apparatus or the cross-camera trajectory determination device can include functional modules corresponding to each functional division, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or software functional module. It should be noted that the division of modules in the embodiments of the present application is illustrative, and is only a logical functional division. Actual implementation can have another division method.

[0157] Figure 7 A structure diagram of a cross-camera trajectory determination apparatus provided by an embodiment of the present application is shown. The cross-camera trajectory determination apparatus is used to execute the above cross-camera trajectory determination method. As shown in the figure, the cross-camera trajectory determination apparatus 30 includes an acquisition unit 301, a determination unit 302, and a processing unit 303. Figure 7

[0158] The acquisition unit 301 is configured to acquire a video stream captured by a target camera, the target camera being any one of a plurality of cameras deployed on the same vehicle.

[0159] The determination unit 302 is configured to determine a trajectory segment of an object under the target camera, the trajectory segment of the object including a local identifier of the object, a detection box, a trajectory in a world coordinate system, and an appearance feature.

[0160] The determination unit 302 is further configured to determine whether a target historical trajectory exists in a plurality of historical trajectories, the historical trajectory being obtained by merging a plurality of trajectory segments, and the target historical trajectory including a plurality of trajectory segments, at least one of which includes a local identifier identical to the local identifier of the object.​

[0161] The processing unit 303 is configured to update the target historical trajectory according to the trajectory segment of the object in a case where the target historical trajectory exists in the plurality of historical trajectories.

[0162] The determining unit 302 is further configured to determine a target similarity between the trajectory segment of the object and each trajectory segment of the plurality of trajectory segments under other cameras in a case where the target historical trajectory does not exist in the plurality of historical trajectories.

[0163] The determining unit 302 is further configured to determine a cross-camera trajectory of the object in a case where the target trajectory segment exists in the plurality of trajectory segments, the cross-camera trajectory including the trajectory segment of the object and the target trajectory segment, the target trajectory segment being a trajectory segment of the plurality of trajectory segments having a target similarity with the trajectory segment of the object greater than a preset similarity.

[0164] Optionally, the obtaining unit 301 is further configured to perform object detection on the video stream based on a preset algorithm to obtain the local identifier of the object and the detection frame.

[0165] The determining unit 302 is further configured to determine a trajectory of the object in a world coordinate system according to the local identifier of the object and the detection frame; determine a target image including the object in the video stream, the target image being a plurality of images having an object quality score higher than a preset score, the object quality score being used to represent a definition, completeness and orientation of the object in the image; determine an appearance feature of the object according to the target image; and determine the trajectory segment of the object under the target camera according to the local identifier of the object, the detection frame, the trajectory of the object in the world coordinate system and the appearance feature.

[0166] Optionally, the determining unit 302 is further configured to determine a local identifier set, the local identifier set including a local identifier corresponding to each historical trajectory of the plurality of historical trajectories; determine that the target historical trajectory exists in the plurality of historical trajectories in a case where the local identifier of the object exists in the local identifier set; and determine that the target historical trajectory does not exist in the plurality of historical trajectories in a case where the local identifier of the object does not exist in the local identifier set.

[0167] Optionally, the determining unit 302 is further configured to determine a lowest object quality score of the object quality scores of the plurality of images in the target historical trajectory.

[0168] The processing unit 303 is further configured to replace an image corresponding to the lowest object quality score according to the target image in a case where the object quality score of the target image is greater than the lowest object quality score.

[0169] Optionally, the determining unit 302 is further configured to determine an appearance similarity and a spatio-temporal similarity between the trajectory segment of the object and each trajectory segment of the plurality of trajectory segments under other cameras; and determine the target similarity according to the appearance similarity and the spatio-temporal similarity.

[0170] Optionally, the determining unit 302 is further configured to determine the target similarity according to the appearance similarity, the space-time similarity, and a first preset weight in a case that the trajectory segment of the object is in the overlapping area of the target camera and the other cameras, the first preset weight including a weight of the appearance similarity and a weight of the space-time similarity, the weight of the appearance similarity being less than the weight of the space-time similarity.

[0171] Optionally, the determining unit 302 is further configured to determine the target similarity according to the appearance similarity, the space-time similarity, and a second preset weight in a case that the trajectory segment of the object is not in the overlapping area of the target camera and the other cameras, the second preset weight including the weight of the appearance similarity and the weight of the space-time similarity, the weight of the appearance similarity being greater than the weight of the space-time similarity.

[0172] Optionally, the processing unit 303 is further configured to update the plurality of trajectory segments in a case that the target trajectory segment does not exist in the plurality of trajectory segments, the updated plurality of trajectory segments including the trajectory segment of the object.

[0173] Optionally, the processing unit 303 is further configured to delete any one of the plurality of historical trajectories in a case that the any one of the plurality of historical trajectories is not updated for a preset time length.

[0174] Figure 8 is a block diagram of a cross-camera trajectory determination device according to an example embodiment. As shown in Figure 8 The cross-camera trajectory determination device 40 includes but is not limited to a processor 401 and a memory 402.

[0175] The memory 402 is configured to store executable instructions of the processor 401. It can be understood that the processor 401 is configured to execute the instructions to implement the cross-camera trajectory determination method in the above embodiments.

[0176] It should be noted that those skilled in the art can understand that the cross-camera trajectory determination device structure shown in Figure 8 does not constitute a limitation to the cross-camera trajectory determination device. The cross-camera trajectory determination device can include more or less components than those shown in Figure 8 , or combine some components, or different component arrangements.

[0177] The processor 401 is a control center of the cross-camera trajectory determination device, connects each part of the cross-camera trajectory determination device through various interfaces and lines, performs various functions of the cross-camera trajectory determination device and processes data by running or executing software programs and / or modules stored in the memory 402 and calling data stored in the memory 402, thereby monitoring the cross-camera trajectory determination device as a whole. The processor 401 can include one or more processing units. Optionally, the processor 401 can integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 401.

[0178] The memory 402 can be used to store software programs and various data. The memory 402 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs (such as a determination unit, a processing unit, etc.) required by at least one function module, and the like. In addition, the memory 402 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0179] In the example embodiment, a computer readable storage medium including instructions is also provided, for example, the memory 402 including instructions, and the instructions can be executed by the processor 401 of the cross-camera trajectory determination device 40 to implement the cross-camera trajectory determination method in the above-mentioned embodiment.

[0180] In actual implementation, Figure 7 The functions of the acquisition unit 301, the determination unit 302, and the processing unit 303 in the above-mentioned embodiment can be realized by the processor 401 calling the computer program stored in the memory 402. The specific execution process can refer to the description of the cross-camera trajectory determination method in the above-mentioned embodiment, and will not be described here. Figure 8 The functions of the acquisition unit 301, the determination unit 302, and the processing unit 303 in the above-mentioned embodiment can be realized by the processor 401 calling the computer program stored in the memory 402. The specific execution process can refer to the description of the cross-camera trajectory determination method in the above-mentioned embodiment, and will not be described here.

[0181] Optionally, the computer readable storage medium can be a non-transitory computer readable storage medium, for example, the non-transitory computer readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0182] In the example embodiment, the embodiment of the present application also provides a vehicle including the above-mentioned cross-camera trajectory determination device.

[0183] In the example embodiment, the embodiment of the present application also provides a computer program product including one or more instructions executable by the processor 401 of the cross-camera trajectory determination device to complete the cross-camera trajectory determination method in the above embodiment.

[0184] It should be noted that the instructions in the above computer readable storage medium or the one or more instructions in the computer program product are executed by the processor of the cross-camera trajectory determination device to realize the processes of the above cross-camera trajectory determination method embodiment, and achieve the same technical effects as the above cross-camera trajectory determination method. To avoid repetition, it will not be described here.

[0185] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of functional modules is taken as an example for illustration. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete the above-described full classification or part of the function.

[0186] In several embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiment described above is only illustrative, for example, the division of modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be through some interface, indirect coupling or communication connection between the units or devices, which can be electrical, mechanical or other forms.

[0187] The units described as separate components can or can not be physically separated, and the components shown as units can be one physical unit or multiple physical units, that is, they can be located in one place or distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0188] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0189] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application essentially or say the part that contributes to the prior art or the whole classification part or part of the technical solutions can be embodied in the form of a software product. The software product is stored in a storage medium, including a plurality of instructions to make a device (which can be a single-chip microcomputer, a chip, etc.) or a processor execute the whole classification part or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, and various storage program codes.

[0190] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any change or replacement within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A cross-camera trajectory determination method, characterized in that, The method comprises: acquiring a video stream captured by a target camera, the target camera being any one of a plurality of cameras deployed on the same vehicle; determining a track segment of an object under the target camera, the track segment of the object comprising a local identifier, a bounding box, a track in a world coordinate system, and an appearance feature of the object; determining whether a target historical track exists in a plurality of historical tracks, the historical track being obtained by merging a plurality of track segments, and the target historical track comprising a plurality of track segments, at least one of which comprises the same local identifier as the local identifier of the object; in a case where the target historical track exists in the plurality of historical tracks, updating the target historical track according to the track segment of the object; in a case where the target historical track does not exist in the plurality of historical tracks, determining an appearance similarity and a spatio-temporal similarity between the track segment of the object and each track segment of a plurality of track segments under other cameras; in a case where the track segment of the object is within an overlapping region of the target camera and the other cameras, determining a target similarity according to the appearance similarity, the spatio-temporal similarity, and a first preset weight, the first preset weight comprising a weight of the appearance similarity and a weight of the spatio-temporal similarity, the weight of the appearance similarity being less than the weight of the spatio-temporal similarity; in a case where the track segment of the object is not within the overlapping region of the target camera and the other cameras, determining the target similarity according to the appearance similarity, the spatio-temporal similarity, and a second preset weight, the second preset weight comprising the weight of the appearance similarity and the weight of the spatio-temporal similarity, the weight of the appearance similarity being greater than the weight of the spatio-temporal similarity; in a case where a target track segment exists in the plurality of track segments, determining a cross-camera track of the object, the cross-camera track comprising the track segment of the object and the target track segment, the target track segment being a track segment in the plurality of track segments having a target similarity with the track segment of the object greater than a preset similarity.

2. The cross-camera trajectory determination method of claim 1, wherein, The method comprises: acquiring a video stream captured by a target camera, the target camera being any one of a plurality of cameras deployed on the same vehicle; determining a track segment of an object under the target camera, the track segment of the object comprising a local identifier, a bounding box, a track in a world coordinate system, and an appearance feature of the object; determining whether a target historical track exists in a plurality of historical tracks, the historical track being obtained by merging a plurality of track segments, and the target historical track comprising a plurality of track segments, at least one of which comprises the same local identifier as the local identifier of the object; in a case where the target historical track exists in the plurality of historical tracks, updating the target historical track according to the track segment of the object; in a case where the target historical track does not exist in the plurality of historical tracks, determining an appearance similarity and a spatio-temporal similarity between the track segment of the object and each track segment of a plurality of track segments under other cameras; 3. The cross-camera trajectory determination method of claim 1, wherein, in a case where the track segment of the object is within an overlapping region of the target camera and the other cameras, determining a target similarity according to the appearance similarity, the spatio-temporal similarity, and a first preset weight, the first preset weight comprising a weight of the appearance similarity and a weight of the spatio-temporal similarity, the weight of the appearance similarity being less than the weight of the spatio-temporal similarity; in a case where the track segment of the object is not within the overlapping region of the target camera and the other cameras, determining the target similarity according to the appearance similarity, the spatio-temporal similarity, and a second preset weight, the second preset weight comprising the weight of the appearance similarity and the weight of the spatio-temporal similarity, the weight of the appearance similarity being greater than the weight of the spatio-temporal similarity; in a case where a target track segment exists in the plurality of track segments, determining a cross-camera track of the object, the cross-camera track comprising the track segment of the object and the target track segment, the target track segment being a track segment in the plurality of track segments having a target similarity with the track segment of the object greater than a preset similarity. The method comprises: acquiring a video stream captured by a target camera, the target camera being any one of a plurality of cameras deployed on the same vehicle; determining a track segment of an object under the target camera, the track segment of the object comprising a local identifier, a bounding box, a track in a world coordinate system, and an appearance feature of the object; determining whether a target historical track exists in a plurality of historical tracks, the historical track being obtained by merging a plurality of track segments, and the target historical track comprising a plurality of track segments, at least one of which comprises the same local identifier as the local identifier of the object; in a case where the target historical track exists in the plurality of historical tracks, updating the target historical track according to the track segment of the object; in a case where the target historical track does not exist in the plurality of historical tracks, determining an appearance similarity and a spatio-temporal similarity between the track segment of the object and each track segment of a plurality of track segments under other cameras; in a case where the track segment of the object is within an overlapping region of the target camera and the other cameras, determining a target similarity according to the appearance similarity, the spatio-temporal similarity, and a first preset weight, the first preset weight comprising a weight of the appearance similarity and a weight of the spatio-temporal similarity, the weight of the appearance similarity being less than the weight of the spatio-temporal similarity; in a case where the track segment of the object is not within the overlapping region of the target camera and the other cameras, determining the target similarity according to the appearance similarity, the spatio-temporal similarity, and a second preset weight, the second preset weight comprising the weight of the appearance similarity and the weight of the spatio-temporal similarity, the weight of the appearance similarity being greater than the weight of the spatio-temporal similarity; in a case where a target track segment exists in the plurality of track segments, determining a cross-camera track of the object, the cross-camera track comprising the track segment of the object and the target track segment, the target track segment being a track segment in the plurality of track segments having a target similarity with the track segment of the object greater than a preset similarity. determining that the target historical trajectory exists in the plurality of historical trajectories in a case where the local identifier of the object exists in the set of local identifiers; determining that the target historical trajectory does not exist in the plurality of historical trajectories in a case where the local identifier of the object does not exist in the set of local identifiers.

4. The cross-camera trajectory determination method of claim 2, wherein, updating the target historical trajectory according to the trajectory segment of the object comprises: determining a lowest object quality score in object quality scores of a plurality of images in the target historical trajectory; replacing an image corresponding to the lowest object quality score according to the target image in a case where the object quality score of the target image is greater than the lowest object quality score.

5. The cross camera trajectory determination method of claim 1, wherein, The method further comprises: updating the plurality of trajectory segments in a case where the target trajectory segment does not exist in the plurality of trajectory segments, the plurality of trajectory segments after being updated comprising the trajectory segment of the object.

6. The cross-camera trajectory determination method of claim 1, wherein, The method further comprises: deleting any one historical trajectory in the plurality of historical trajectories in a case where the any one historical trajectory is not updated for more than a preset time length.

7. A cross-camera trajectory determination apparatus, characterized by, comprise an acquisition unit, a determination unit, and a processing unit; The acquisition unit is configured to acquire a video stream captured by a target camera, the target camera being any one of a plurality of cameras deployed on the same vehicle; The determination unit is configured to determine a trajectory segment of an object under the target camera, the trajectory segment of the object comprising a local identifier of the object, a bounding box, a trajectory in a world coordinate system, and an appearance feature. The determination unit is further configured to determine whether a target historical trajectory exists in a plurality of historical trajectories, the historical trajectory being obtained by merging a plurality of trajectory segments, and the target historical trajectory comprising a plurality of trajectory segments, at least one trajectory segment of the plurality of trajectory segments comprising a local identifier identical to the local identifier of the object. The processing unit is configured to update the target historical trajectory according to the trajectory segment of the object in a case where the target historical trajectory exists in the plurality of historical trajectories. The determination unit is further configured to determine an appearance similarity and a spatio-temporal similarity between the trajectory segment of the object and each trajectory segment of a plurality of trajectory segments under other cameras in a case where the target historical trajectory does not exist in the plurality of historical trajectories, to determine a target similarity according to the appearance similarity, the spatio-temporal similarity, and a first preset weight in a case where the trajectory segment of the object is within an overlapping region of the target camera and the other cameras, the first preset weight comprising a weight of the appearance similarity and a weight of the spatio-temporal similarity, the weight of the appearance similarity being less than the weight of the spatio-temporal similarity, and to determine the target similarity according to the appearance similarity, the spatio-temporal similarity, and a second preset weight in a case where the trajectory segment of the object is not within the overlapping region of the target camera and the other cameras, the second preset weight comprising the weight of the appearance similarity and the weight of the spatio-temporal similarity, the weight of the appearance similarity being greater than the weight of the spatio-temporal similarity. ​ The determining unit is further configured to, in a case where there is a target track segment in the plurality of track segments, determine a cross-camera track of the object, the cross-camera track including the track segment of the object and a target track segment, the target track segment being a track segment in the plurality of track segments that has a target similarity to the track segment of the object greater than a preset similarity.

8. A cross-camera trajectory determination device, comprising: Deployed on a vehicle, comprising a memory and a processor; The memory and the processor are coupled; The memory is configured to store computer program code, the computer program code comprising computer instructions; When the processor executes the computer instructions, the cross-camera track determination device executes the cross-camera track determination method according to any one of claims 1-6.

9. A computer-readable storage medium having stored therein instructions, the computer-readable storage medium comprising: When the instructions run on the cross-camera track determination device, the cross-camera track determination device executes the cross-camera track determination method according to any one of claims 1-6.

10. A vehicle characterized by comprising: The cross-camera track determination device according to claim 8.

Images